That's nice, dear

This blog post is about programming, for people that don't program - at least that's the effect I'm aiming for. The title is in recognition of my tolerant wife, The Inelegant Gardener, who has learnt the appropriate response to my enthusiastic displays of the results of my programming: "That's nice, dear"!.

I started programming a long time ago - in around 1980, at the school computer club, when I was 10. Since then I've been taught odd bits of programming by scientists, and done quite a lot of programming as part of my scientific job. I've started to get more interested in proper software engineering in the last few years. This is a roundabout way of saying I am an enthusiastic amateur.

People associate programming with the mathematically minded, but this isn't necessarily the case: the codebreakers at Bletchley Park, who were amongst the first users of electronic computers, had a range of skills - amongst them were linguists and crossword wizards. I was talking to a Fellow in linguistics, who'd helped write his college's library software - as he pointed out: a very logical view of language is a great benefit for a programmer. Programming is about giving an idiot very exact instructions, if the instructions concern maths then you need to know maths - otherwise you don't.

The core of programming is still what I learned years ago, data (numbers or letters) is stored in "variables" that have names. There are conditional statements: "If [something is true] Then [do this] or else [do the other]". There are looping statements: "Do this 100 times". And there are functions: "add 2 to this number, square it, add the number you first thought of and tell me the answer" or "how many times does the letter a occur in this sentence".

These simple statements are being buried under an increasing depth of additional ideas. Since the 80's the big thing in programming has been "object-orientation". In object-orientated programming you package up data of a particular sort with functions that relate to that data. So if you had data modelling an octopus you would include functions such as "wave-tentacles" and "change colour", such functions would be useless for data describing a horse. The real benefit to this is comprehending larger software systems, because a sea of functions and data is grouped together into logical islands. Beyond this there are design patterns - reoccurring systems of objects which I haven't entirely go the hang of.

In addition to the changes in language, there are changes in the tools used to program: syntax highlighting is nice, it amounts to colouring the verbs, nouns and proper names in programming in different colours - makes it easier to spot mistakes. Auto-completion is another handy tool, in a well-designed language there are only a limited number of next possible statements when you are programming - auto-completion presents you with them as you type. Sites like Stackoverflow are great for asking programming questions, and there no end of function libraries available on the web to help you out.

I have a number of little software projects on the go, you can see them in much the same way as woodworking projects, suduko or crosswords: they keep me out the way, muttering quietly to myself and exercising my brain. It doesn't matter that what I'm doing isn't groundbreaking and new.

Programming does lead to some odd habits; when I started programming it was useful to know binary and hexadecimal number systems, as a consequence I believe that numbers such as 1024 and 128 are nice and round. I've come to appreciate a wide range of bracket styles [] (){} since they are all used for different things and the semi-colon is one of the most important pieces of punctuation in my life. If I program for too long in a stretch I start to forget how to speak to people.

And just to show off the results of my latest fiddlings: maps of the UK election results. I got interested in doing this just after the General Election. The Guardian has published a lovely spreadsheet of election results, including data on every single candidate. You see lots of maps of data of this sort, I wanted to know how it was done. (Technical details beyond the maps.)

First of all the gender of MP's by constituency: constituencies represented by ladies are marked pink, those by men marked blue:

The black constituency in northern England is Thirsk and Malton, which held its election on 27th May, following the death of one of the candidates during the general election campaign.

The population of each constituency is also interesting, here I have coloured the constituencies with 9 different shades of green, the palest shade corresponds to a voting population of between 20,000 and 30,000, the darkest shade corresponds to a population of between 100,000 and 110,000:

The Western Isles (now known by it's Gaelic name: Na h-Eileanan an Iar) has the smallest population at about 22,000 and the Isle of Wight has the largest population with just under 110,000 potential voters. I used ColorBrewer to find a nice set of colours.

Finally here's a map of which party came second in each constituency in the 2010 General Election:

Red for Labour, blue for the Conservatives, orange for Liberal Democrats, yellow for Scottish Nationalists, pale green for Plaid Cymru, dark green for Sinn Fein, blue for Ulster Conservatives and Unionists, and there are a few independents and minor Northern Island parties which are all coloured white. 

Footnotes

So the task is to get the spreadsheet data into a map: To get started I did a bit of memory trawling and googling, a couple of people have written about colouring in maps: this one uses shapefile format map data and the R programming language, whilst this one uses SVG format map data and Python (another programming language). It turns out the shapefile format data for constituencies is a little difficult to get - you have to fill in forms! However enterprising people on Wikipedia have made SVG format constituency maps available. SVG stands for Scaleable Vector Graphics, it's an XML format which means it's plaintext and there are standard means to extract data from it and manipulate it. The only real problem is that the constituency names in the spreadsheet don't exactly match the names inside the SVG format map - I had to resort to some horrible constituency by constituency coding for a load of them. To do this I used the C# programming language, largely because Visual Studio Express C# is a very nice, free development environment which I've used before. To view the SVG maps inside my application I used the Webkit .NET library to provide a webbrowser control (which wraps up the rendering engine used in the Safari and Google Chrome browsers) - the native C# webbrowser control is based on Internet Explorer - which doesn't render SVG. Output to bitmaps is a bit clumsy, Inkscape (a free SVG editor) wasn't keen on displaying the original constituency map, so I resorted to viewing the map in Google Chrome and taking a screen shot (a terrible bodge).

Book review: The World of Gerard Mercator

Once again I have been reading, this time "The World of Gerard Mercator" by Andrew Taylor. As before this blog post could be viewed as a review or, alternatively, as some notes to remind of what I have read. Overall I enjoyed the book, it provides the right amount of background information and doesn't bang on interminably about minutiae. I would have liked to have seen some better illustrations, but I suspect good illustrations of maps of this period are hard to come by and a full description of Mercator's projection was probably not appropriate.

The book starts off with some scene setting: at the beginning of the 16th century the Catholic church were still keen on Ptolemy's interpretation of world geography in fact to defy this interpretation was a heresy and could be severely punished. Ptolemy had put down his thoughts in Geographia produced around 150AD, which combined a discussion of the methods of cartography with a map of the known world. As a precedent Ptolemy's work was excellent, however by the time of the 16th century it was beginning to show it's antiquity. Geographical data, in Ptolemy's time, from beyond the Roman Empire was a little fanciful, and since the known world was a relatively small fraction of the surface of the globe the problems associated with showing the surface of a 3D object on a 2D map were not pressing. Ptolemy was well aware of the spherical nature of the world, Eratothenes had calculated the size of the earth in around 240BC, he stated that a globe would be the best way of displaying a map of the world. However, a globe large enough to display the whole world at sufficient detail would have to be very large, and thus difficult to construct and transport.

Truly global expeditions were starting to occur in the years before Mercator's birth: Columbus had "discovered"  the West Indies in 1492, John Cabot made landfall on the North American landmass in 1497. Bartolomeu Dias had sailed around the Southern tip of Africa in 1488, Vasco da Gama had continued on to India in 1497, around the Cape of Good Hope. The state of the art in geography could be found in Waldseemüller's map of 1507, showing a recognisable view of most of our world. Magellan's expedition would make the first circumnavigation of the globe in the early years of Mercator's life (1519-1522).

Mercator was born in Rupelmonde in Flanders on 5 March 1512, he died 2 December 1594 in Duisburg in what is now Germany at the age of 82. This was a pretty turbulent time in the Netherlands, the country was ruled by Charles V (of Spain) and there appears to have been significant repression of the somewhat rebellious and potentially Protestant population. Mercator was imprisoned for heresy in Rupelmonde in February 1543, remaining in custody until September, many in similar circumstances were executed, however Mercator seems to have avoided this by a combination of moderately powerful friends and a lack of any evidence of heresy.

Mercator's skill was in the collation and interpretation of geographical data from a wide range of sources including his own surveys. In addition he was clearly a very skilled craftsman in the preparation of copperplate engravings. He was commercially successful, manufacturing his globe throughout his life, as well as many maps and scientific instruments for cartographers. He also had a clear insight into the power of patronage.

His early work was in the preparation of maps of the Holy Land (in 1537) and Europe (in 1554), along with a globe produced in 1541. The globe seems to be popular amongst reproducers of antiquities, you can see details of it on the Harvard Map Collection Website.

Mercator is best known for his "projection", in this context a projection is a way of converting the world - which is found on the surface of a 3D sphere into a flat, 2D map. Mercator introduced his eponymous projection for his 1569 map of the world, illustrated at the top of this post. The particular feature of this projection is that if you follow a fixed compass bearing you find yourself following a straight line on the Mercator projected map. This is good news for navigators! The price you pay for this property is that, although all regions are in the correct places relative to each other, their areas are distorted so those regions near the poles appear much larger than those near the equator. Mercator seems to have made little of this discovery, nor described the method by which the projection is constructed - this was done some time later, in 1599, by Edward Wright. Prior to maps following Mercator's projection navigation was a bit hit and miss, basically you headed up to a convenient latitude and then followed it back to your destination - an inefficient way to plan your course. If you're interested in the maths behind the projection see here.

In terms of it's content the 1569 map shows Europe, Africa and a large fraction of Asia much as we would see it today, certainly in terms of outline. The Eastern coast of North and South America is fairly recognisable. The map fails in it's representation of the West coast of America - although to give credit where it is due, it at least has a west coast. The landmasses indicated at the northern and southern poles are close to pure fantasy. The Southern continent had been proposed by Ptolemy as a counterbalance to the known Northern continents - with no supporting evidence. Exploration of the far North was starting to occur during Mercator's life, with expedition such as that of Frobisher.

Mercator is also responsible for the word "atlas" to describe a book containing a set of maps, in this instance he coined the term to describe the volumes of maps he was preparing towards the end of his life, the last of which was published published posthumously by his son, Rumold, in 1595.

Following my efforts on Joseph Banks, I thought I'd make a map of significant locations in Mercator's life. You can find them here in Google Maps, zoom out and you will see the world in Mercator projection - a legacy from a man that lived nearly 500 years ago.

Understanding mayonnaise

Some time ago I wrote a post on confocal microscopy - a way of probing 3D structure at high spatial resolution. This post is about using confocal microscope to understand mayonnaise (and a bunch of other things)

As young scientists we are introduced to the ideas of solids, liquids and gases very early on. We make these distinctions, amongst other things, to understand their mechanical properties, to answer questions such as: How thick do I have to make the legs of my chair to support my weight? How fast will liquid run out of a bucket? How high will my balloon fly?

But what is mayonnaise? It's very soft, and can be made to flow but it's not a proper liquid - you can make a pile of mayonnaise. How do we describe grain in a silo, or an avalanche? In some senses they have properties similar to a liquid: they flow - yet they form heaps which is something a solid does. What about foams -  a pile of shaving foam looks pretty similar to mayonnaise? Starch paste is an even weirder example, it acts like a liquid if you treat it gently but a solid if you try anything quick. (This is known as shear thickening). These mixed systems are known as colloids.

The programme for understanding solids, liquids, gases and these odd systems is to understand the interactions between the "fundamental" particles in the system. For our early courses in solids, liquids, and gases this means understanding what the atoms (or molecules) are doing - how many of them are there in a unit volume, how are they ordered, how do they move and how they interact. Typically there are many "fundamental" particles in whatever you're looking at so rather than trying to work out in detail what all of them are up to you resort to "statistical mechanics": finding the right statistical properties of your collection of particles to inform you of their large scale behaviour.

The distinguishing feature of all of our new systems (mayonnaise, grain piles, avalanches, foams, starch paste) is that they are made from lumps of one phase (gas, liquid, solid) in another. Avalanches and grain piles are solid particles in a gas; mayonnaise is an emulsion: liquid droplets (oil) inside another liquid (water); foams are air inside a liquid and starch paste is a solid inside a liquid. These systems are more difficult to analyse than our traditional gases, solids and liquids: firstly their component parts aren't all simple and aren't all the same. Particles most likely have different sizes and shapes. Atoms and molecules are all the same size and all the same shape. Secondly, they're athermal - ambient temperatures don't jiggle all their bits around to make nice averages.

Confocal microscopy looked like an interesting way to answer some of these important questions about the structures to be found in these complex systems. Mayonnaise turns out not to be a good model system to work with - you can't see through it. However, you can make an emulsion of different combinations of oil and water, and if you're cunning you can make an emulsion with over 50% of droplets by volume which is still transparent. Using even more cunning you can make the distribution of droplet sizes relatively small.

Having spent a fair bit of time getting the emulsions transparent with reasonable droplet size distributions, my student, Jasna, came in with some pictures of an emulsion from the confocal microscope: where the oil droplets touched each other the image was brighter, you can see this in the image at the top of this post. This was rather unexpected, and useful. The thing about squishy balls, is that the amount by which they are squished tells you something about how hard they are being squeezed. The size of the little patches tells you how much force each droplet is feeling. So all we have to do to find the force network in an emulsions is measure the size of the bright patches between them.

In the end our work measured the forces between droplets in a compressed emulsion and we found that these measurements agreed with a theory and some computer simulations. Criticisms of the work were that the relationship between luminous patch size and force was more complicated than we had assumed, and that the force distribution was all very well but the interesting thing was the arrangement of those forces. These criticisms are fair enough. Must have been pretty good though, because someone wrote a paper for Science claiming to have done it first, whilst citing our paper (they had to publish a correction)!

Footnotes
This work can be found in this paper:

Brujic, J., S. F. Edwards, D. V. Grinev, I. Hopkinson, D. Brujic, and H. A. Makse. “3D bulk measurements of the force distribution in a compressed emulsion system.” Faraday  Discussions 123, (2003), 207-220.  (pdf file on Scribd)

Jasna Brujic was the PhD student who did the experimental work, Sir Sam Edwards is a theoretician who works on granular materials, Dmitri Grinev worked with Sir Sam on the theory, I supervised Jasna, Djordje Brujic is Jasna's dad and wrote the image analysis code and Hernan Makse is a computer simulator of granular materials.

Go back to your constituencies and prepare for government!

I think this will be my final political blog post for a while. I've written quite a few over the past month: I started with an explanation as to why I was a member of the Liberal Democrats here. I hunted out some data on the occupations of MP's and made a graph here (what a lot barristers!). Then I wrote a blog post about the statistical errors in opinion polls, which I think was a little prescient (latest reports suggest that the voters that didn't turn out were predominantly LibDem - hence biasing the pre-election polls). Approaching the finishing line, I wrote a post on my plans to stay up late on election night, followed by a post on my crashing gloom of the night itself. Then I made a post on what it might mean as the negotiations for coalition continued.

This post is about coalition, and a look at the debris of the election. My friends at work from Labour and Conservative tendencies called me a "turncoat weasel" today (in a friendly manner) so I have illustrated this post with a picture I took of a weasel.

First up, my take on the agreement between the Liberal Democrat and Conservative Party: remind me not to negotiate with Nick Clegg for anything, I think he's played a blinder on this one. The agreement contains a mix of LibDem and Tory policies, and five positions in cabinet. It's quite clear that the LibDems are a junior partner, but then that's what you'd expect. I'm also pleased Nick Clegg did what he said he would do before the election: go to the party with the largest electoral mandate in the first instance.

Despite the massive apparent differences on Europe, I suspect it won't be a big problem for the coalition. Liberal Democrats have a principled view that power should dissociate to the appropriate level - some things are best handled at a local level, some at a national level, and some to the European level. My guess is that over the next five years there will not be another major treaty moving power towards Europe, and joining the Euro in the next five years, as we all crawl out of recession, would be unwise as far as anyone is concerned. Perhaps David Cameron will value a coalition partner who has not isolated itself in Europe.

I'm wondering whether the Labour Party are trying to take the mantle of "the nasty party", in the background to David Cameron's speech in Downing Street could be heard a chant of "Tory scum". David Blunkett described Nick Clegg as a harlot, amusing coming from an adulterer. And this morning Nick Cohen, left-wing columnist for the Observer and various other places, tweeted:

Here's my writing sorted out, Will crucify every fucker who voted Lib Dem, one by one, in the national press. Better you had voted Tory.

I suppose I should proviso this last one, on the whole I don't believe a single tweet means national news, perhaps Cohen was joking, perhaps he was tired and emotional. But it picks up the tenor of quite a few of the tweets I've seen from Labour supporters today.

Maybe this is why I'm a natural Liberal Democrat, I feel nothing like a visceral hatred for either of the two other main parties. I believe that people are largely in politics for the right reasons, I may think they are wrong but I don't question their faith in what they believe is for the best. Getting a glimpse of how elections work via twitter, I see just how much work is involved in getting elected as an MP (at least in a contestable seat).

It's tickled my fancy that much of the discussion of the last few days has centred around the percentage share of the vote that a different alliances would get, that's lovely and I agree entirely with that line of argument but your past record: opposing proportional representation, tells me that you're a hypocrite if you raise it just now.

I'm still glad I voted Liberal Democrat, I didn't vote Tory because of the social conservatives, Margaret Thatchers "no such thing as society", Michael Howard's blind faith in market solutions, isolationism in Europe, Peter Lilley and his little list, John Major's denial of support to the Iraq rebels after the first Gulf War and the moves of senior Tories into directorships of companies benefiting from their privatisations. The Tories claim to have changed, perhaps they have, perhaps the LibDems will reinforce that strand.

I hope the move for fair votes takes off, in a sense this is one of the key things I'm after long term in politics. Democracy is important, when I go to the ballot box I want my vote to count just as much as yours. Sorting out the deficit is ephemeral, fair votes will be a lasting legacy.

Perhaps the coalition will fall apart before the end of its term, perhaps the LibDems will suffer for this coalition in the next election (there's some suggestion that this happens to smaller coalition parties). But for the first time in 70 years people who call themselves "liberal" are in government.

May you live in interesting times...

It turns out that a chunk of my audience for my last blog post were my colleagues at work, they thought it a bit of a gloomy rant. These days I'm a bit more perky: in contrast to every election since 1974, this time the Liberal Democrats (my party) have something to be cheery about following the despair of election night! Usually post-election we are most definitely not in government, returning wearily to our constituencies to prepare for more time in opposition. This time it's different!

Watching the comments on twitter as events have unfolded has raised a few questions, and clear misconceptions which I thought might try to address from my point of view as a long (21 years) term party member.

What are the Liberal Democrats?
Some Liberal Democrats were carried over from the old style Liberals, some Liberal Democrats split off from the Labour party as the Social Democrat Party, since 1988 they were simply Liberal Democrats. I've always been a Liberal Democrat but my political origins are probably closer to the soft-right of the Tory party. I've never been tribal Labour (or Tory) for that matter. It's fair to say that the majority of the Liberal Democrats are left of centre, but we're in the party for a reason - we don't want to be in any other party.

What is coalition government?
The way people talk you might get the impression that the Liberal Democrats in coalition would simply be there to prop up their coalition partners. Labour seem to view this almost as a right, that the Liberal Democrats are a little turbocharger for those elections where they didn't quite win in their own right. Consequently they believe that a LibCon coalition would simply prop up a Tory government with a Tory agenda. This misses the point of coalition entirely, why on earth would we sign up to such a deal? The point of coalition is to get at least some of your agenda implemented, if you're not in the governing coalition then none of your agenda is implemented.

Proportional Representation
A lot of the discussion at the moment is around proportional representation, personally I think it should be around the economy first: massive deficits don't get reduced by themselves. I don't intend to discuss proportional representation properly here, but simply highlight three systems:
The pure Alternative Vote system is the one proposed by those that don't actually want proportional representation, it doesn't actually provide a proportional output. The Jenkins Commission, set up by the Labour government following the 1997 election, recommended Alternative Vote plus top up (AV+). In AV+ there are constituency elections with a top-up from party lists that provides proportionality, the benefit here is that there are still relatively small constituencies. The output should be pretty proportional. The Electoral Reform Society prefers Single Transferable Vote, this provides broadly proportional output, but requires the use of large constituencies to work.

Labour and Proportional Representation
Labour's new-found enthusiasm for proportional representation leads to hollow laughter amongst Liberal Democrats. For why? Go have a look at the evolution of the Labour commitment to a referendum. Basically a referendum was promised at the 1997 election, this referendum never happened and although it remained in the manifesto for subsequent elections the commitment became ever weaker. You can see why Liberal Democrats don't trust Labour on proportional representation.

I'd like to present a slightly heretical opinion for a Liberal Democrat: an absence of a commitment for a referendum on proportional representation should not be a deal breaker. My reasoning: I don't believe either Tory or Labour party could currently deliver a majority in parliament for such a referendum. It is possible that a referendum would not require a parliamentary vote, but let's assume it does. A commission on electoral reform means that at least the Tories will have to start thinking about it on their own terms, something they haven't been doing, even if it is a self-evident kick into the long grass. The next time there's a hung parliament we will then have fruitless electoral reform documents from both Labour and Tory parties, but here's the good thing: that means that they can't really ask for another one. Furthermore there appears to be a groundswell of opinion in favour of electoral reform, and I don't think it's party political. Over the coming parliament, and at the next election I really hope this groundswell is directed into contact with politicians, we shouldn't be hearing "This isn't an issue on the doorstep" next time.

Under proportional representation coalition government is likely to become a fact of life so a successful Lib-Con coalition in the absence of a deal on PR would be worth having. I must admit the green shoots of coalition are promising. Rather than a pointless exercise in taking chunks out of each other we are starting to see politicians talk about what they agree on.

In a way we have nothing to lose, what's the worst that can happen? Things fall apart and an election is called where we lose some percentage share of the vote leading to a reduction in seats - unpredictably fewer due to first-past-the-post system. We'd still be an opposition party with little power in parliament, so in a place broadly similar to the one we found ourselves in before this election campaign. What's different now is that there is a broader movement for electoral reform, that may be the thing we won at this election.

In posting this now (5:30pm on Monday 10th May) I am very aware that I may be overtaken by events!

Footnotes

• The Jenkins Commission Report
• Ministry of Justice: Review of Voting Systems: the experience of new voting systems in the United Kingdom since 1997
• Vernon Bogdanor's report on referenda

I was up for Evan Harris

This is a graph that shows you the number of seats (actual seats) each of the three main parties will get*, and the number of seats (proportional seats) they would get under a pure proportional system. You notice for the Labour and Conservative parties the number of seats they actually get is more than the number of seats in proportion to their votes, for the Liberal Democrats the opposite is true and by a very substantial margin.

When Liberal Democrats went into the polling stations yesterday they were given a single polling card, their Labour and Conservative comrades had three. Look them in the eye, ask them:

What is it about you that makes your vote three times more powerful, three times heavier, three times more important than mine?

What is special about you but not about me?

Explain to me how this is fair.

Explain to me how this is democracy.

To put it another way, every Labour or Conservative seat requires about 33,000 votes to win, a Liberal Democrat one requires 100,000 votes. We are the Great Ignored.

We have come to accept this inequity, it's happened in every election since the early 80's. As a country we just accept it as part of the way things are. It's the defining feeling of being a Liberal Democrat, seeing the overall share of our vote creep up election by election and receiving the same feeble, disproportionate harvest in seats. The sinking feeling in the middle of the night that, no, of course there has been no breakthrough. It's not because we perform poorly, it is because we have one polling card each, the others have three.

In 1997 the defining moment was Michael Portillo losing his seat to Stephen Twigg. My defining moment for this election was seeing Evan Harris lose his Oxford West and Abingdon seat. "I was up for Evan Harris", I had a tear in my eye.

Footnote
*This is based on the exit poll (see entry at 23:11), which looks consistent with the results of the actual election as of 10:30am May 7th which are Conservative 291, Labour 247, Liberal Democrat 51 616 of 650 seats declared. Under pure proportionality UKIP would receive 20 seats, the BNP 12.

No sleep 'til Batley!

I'm planning on staying up late tonight, watching the results of the general election come in, this is an occasion for a graph. The chaps at tweetminster have uploaded a list of predicted declaration times here. I've rearranged the data a bit to plot it, the height of each bar tells you the number of constituencies declaring during the hour starting at the time indicated at the bottom of the bar. As you can see, things don't really get going until about 2am. Key times for me are the declaration in my own constituency, City of Chester, at 3am and Dr Evan Harris' Oxford West and Abingdon at 2:30am. Batley & Spen declares at around 5am hence the title of this blog post.

This has been the most exciting election campaign, and election night, in quite sometime. I spent the 1997 election at a friends house in Darlington, I remember stumbling out into the early morning with "Things can only get better" ringing in my ears. For a few years that seemed to be the case. 1992 was interesting in that we all thought John Major was going to lose, and then he won to the surprise of everyone (including John Major). 2001 and 2005 were rather dull.

As a seasoned Liberal Democrat I'm used to my party getting pretty good percentage poll scores overall and winning pitiably few seats, so to the newcomers out there - welcome to my world! I can only hope that this time things will be different.

Economics: The physics of money?

Today I'm off visiting the economists, this is a bit of a different sort of visit since I haven't found that many to follow on twitter, instead I must rely on their writings.

I've been reading Tim Harford's "The Undercover Economist" which is the main topic of this post, in the past I've also read "Freakonomics" by Levitt and Dubner. Harford's book is more about classical economics whilst "Freakonomics" is more about the application of quantitative methods to the analysis of social data. This is happy territory for a physicist such as myself: there are numbers, there are graphs and there are mathematical models.

David Ricardo pops up a few times, it would seem fair to compare him to the Newton of economics, he lived 1772-1823.

I learnt a whole bunch of things from Tim Harford's book, including what shops are up to: working out how to persuade everyone to pay as much as they are willing to pay, by means such as "Value" and "Finest" ranges whose price differences don't reflect their cost differences, similar pricing regimes are found in fancy coffee. In a way income tax bypasses this, it replaces willingness to pay with ability to pay - I'm sure shops would love to be able to do this! Scarcity power allows a company to change more for its goods or services, and a company's profits are indication that this might be happening.

Another important concept is market "efficiency": perfect efficiency is achieved when no-one can be made better off without someone else losing out, this is not the same as fairness. In theory a properly operating market should be efficient but not necessarily fair. Externalities are the things outside the market to which a monetary value needs to be attached in order for them to be included in the efficiency calculation, this includes things like pollution and congestion in the case of traffic. This sounds rather open-ended since I imagine externality costing can be extremely disputed.

There's an interesting section on inside / asymmetric information, and how this prevents markets from operating properly. The two examples cited are second-hand car sales and health insurance, in the first case the seller knows the quality of the car he his selling whilst the buyer struggles to get this information. Under these circumstances the market struggles to operate efficiently because the buyer doesn't know whether he is buying a 'peach' (a good car) or a 'lemon' (a bad car), this reduces the amount he is willing to pay - the seller struggles to find a mechanism to transmit trusted quality information to the buyer. Work on information asymmetry won a Nobel Prize for Economics for George Akerlof, Michael Spence, and Joseph Stiglitz in 2001.

In the second case, health insurance, the buyer purportedly knows the risk they present whilst the seller doesn't, this doesn't quite ring true to me, it seems the observed behaviour in the US private healthcare system matches this model though. In a private insurance system the people who are well (and are likely to remain well) will not buy insurance, whilst those that believe themselves to be ill, or at serious risk of being ill will be offered expensive insurance because there is not a large population of healthy buyers to support them. Harford recommends the Singapore model for health care, which has compulsory saving for health care costs, price controls and universal insurance for very high payouts. This gives the consumer some interest in making most efficient use of the money they have available for health care.

You might recall the recent auctions of radio spectrum for mobile phone and other applications, this turns out to be a fraught process for the organiser - in the US and New Zealand this process went poorly with the government receiving few bids and less cash then they expected. In the UK the process went very well for the government, essentially through a well designed auction system. The theoretical basis for such auctions is in game theory, with John von Neumann and John Nash important players in the field (both recognised as outstanding mathematicians).

Tim Harford did wind me up a bit in this book, repeatedly referring to the market as "the world of truth", and taxes as "lies". This is a straightforward bit of framing: that's to say the language used means anyone arguing against him is automatically in the "arguing against the truth" camp irrespective of the validity of the arguments. The formulation that taxes represent information loss is rather more interesting and he seems to stick with this more often than not. In this instance I feel the "world of truth" is ever so slightly tongue in cheek, but in the real world free-markets are treated very much as a holy "world of truth" by some political factions with little regard to the downsides: such as a complete ignorance of fairness, the problems of inside information and the correct costing of externalities.

A not inconsiderable number of physicists end up doing something in finance or economics: As Tom Lehrer says in the preamble to "In old Mexico": "He soon became a specialist, specializing in diseases of the rich". It turns out you get paid more if the numbers you're fiddling with represent money, rather than the momentum of an atom. Looking at these descriptions of economic models, I can't help thinking of toy physics models which assume no friction, and are at equilibrium. These things are very useful when building understanding, but for practical applications they are inadequate. Presumably more sophisticated economic models take this things into account. From a more physical point of view, it doesn't seem unreasonable to model economics through concepts such as conservation (of cash) and equilibrium, but physics doesn't have to concern itself with self-awareness - i.e. physical systems can't act wilfully once given knowledge of a model of their behaviour. I guess this is where game theory comes in.

The interesting question is whether I should see economics as a science, like physics, which is used by politicians for their own ends or whether I should see them as being rather more on the inside. Economics as a whole seems to be tied up with political philosophy. Observing economists in the media there seem to be much wider range of what is considered possibly correct than you observe in scientific discussion.