Book Review: The Visual Display of Quantitative Information

 

**I’m now blogging at http://www.ianhopkinson.org.uk/, please adjust your links – this post can be found here**

“The Visual Display of Quantitative Information” by Edward R. Tufte is a classic in the field of data graphics which I’ve been meaning to read for a while, largely because the useful presentation of data in graphic form is a core requirement for a scientist who works with experimental data. This is both for ones own edification, helping to explore data, and also to communicate with an audience.

There’s been something of a resurgence in quantitative data graphics recently with the Gapminder project led by Hans Gosling, and the work of David McCandless and Nathan Yau at FlowingData.

 

The book itself is quite short but beautifully produced. It starts with a little history on the “data graphic”, by “data graphic” Tufte specifically means a drawing that is intended to transmit data about quantitative information in contrast to a diagram which might be used to illustrate a method or facilitate a calculation. On this definition data graphics developed surprisingly late, during the 18th century. Tufte cites in particular work by William Playfair, who was an engineer and political economist who is credited with the invention of line chart, bar chart and pie chart which he used to illustrate economic data. There appears to have been a fitful appearance of what might have been a data graphic in the 10th century but to be honest it more has the air of a schematic diagram.

Also referenced are the data maps of Charles Joseph Minard, the example below shows the losses suffered by Napoleon’s army in it’s 1812 Russian campaign. The tan line shows the army’s advance on Moscow, it’s width proportional to the number of men remaining. The black line shows their retreat from Moscow. Along the bottom is a graph showing the temperature of the cold Russian winter at dates along their return.

Interestingly adding data to maps happened before the advent of the more conventional x-y plot, for example in Edmund Halley’s map of 1686 showing trade winds and monsoons.

Next up is “graphic integrity”: how graphics can be deceptive, this effect is measured using a Lie Factor: the size of the effect shown in graphic divided by the size of the effect in data. Particularly heroic diagrams achieve Lie Factors as large as 59.4. Tufte attributes much of this not to malice but to the division of labour in a news office where graphic designers rather than the owners and explainers of the data are responsible for the design of graphics and tend to go for the aesthetically pleasing designs rather than quantitatively accurate design.

 

Tufte then introduces his core rules, based around the idea of data-ink – that proportion of the ink on a page which is concerned directly with showing quantitative data:

• Above all else show the data
• Maximize the data-ink ratio
• Erase non-data-ink
• Erase redundant date-ink
• Revise and edit.

A result of this is that some of the elements of graph which you might consider essential, such as the plot axes, are cast aside and replaced by alternatives. For example the dash-dot plot where instead of solid axes dashes are used which show a 1-D projection of the data:

Or the range-frame plot where the axes are truncated at the limits of the data, actually to be fully Tufte the axes labels would be made at the ends of the data range, not to some rounded figure:

Both of these are examples are from Adam Hupp’s etframe library for Python. Another route to making Tufte-approved data graphics is by using the Protovis library which was designed very specifically with Tufte’s ideas in mind.

Tufte describes non-data-ink as “chartjunk”, several things attract his ire – in particular the moiré effect achieved by patterns of closely spaced lines used for filling areas, neither is he fond of gridlines except of the lightest sort. He doesn’t hold with colour or patterning in graphics, preferring shades of grey throughout. His argument against colour is that there is no “natural” sequence of colours which link to quantitative values.

What’s striking is that the styles recommended by Tufte are difficult to achieve with standard Office software, and even for the more advanced graphing software I use the results he seeks are not the out-of-the-box defaults and take a fair bit of arcane fiddling to reach.  Not only this, some of his advice contradicts the instructions of learned journals on the production of graphics.

Two further introductions I liked were Chernoff faces which use the human ability to discriminate faces to load a graph with meaning, and sparklines - tiny inline graphics showing how a variable varies in time without any of the usual graphing accoutrements: - in this case one I borrowed from Joe Gregorio’s BitWorking.

In the end Tufte has given me some interesting ideas on how to present data, in practice I fear his style is a little too austere for my taste.There’s a quote attributed to Blaise Pascal:

I would have written a shorter letter, but I did not have the time.

I suspect the same is true of data graphics.

Footnote

Mrs SomeBeans has been referring to Tufte as Tufty, who UK readers of a certain age will remember well.

The House of Lords by numbers

Reform is in the air for the House of Lords, to be fair reform has been in the air for large parts of the last hundred years. Currently reform comes in the form of a proposal put forward by Nick Clegg and backed by David Cameron – you can see the details here. It comes in the context of all three main Westminster parties supporting a largely elected House of Lords in their 2010 General Election manifestos.

The purpose of this post is not to go through the proposals in detail but simply to provide some charts on appointments to the House of Lords over the years. The current composition of the House is shown in the pie-chart below:

The membership of the House of Lords currently numbers 789, I have excluded the handful of members from UKIP, DUP, UUP, the Greens and Plaid Cymru since they are too few to show up in such a chart.

The website www.theyworkforyou.com provides a handy list of peers in an easily readable format, this list includes data such as when they were appointed, what party they belong to, what name they have chosen and when they left and whether they used to be an MP. We can plot the number of appointments each year:

I’ve highlighted election years in red, as you can see election years are popular for the appointment of new members, and it would seem many of those appointed in such years are former MPs, as shown in the graph below:

But to which parties do these appointees belong? This question is answered below:

I hope this provides a useful backdrop to subsequent discussions on reform.

Deficit reduction through growth

This blog post seeks to answer the question: what economic growth rate does the UK need to sustain in order to reduce the deficit to zero?

This seems like a relevant question at the moment, and I’ve not seen a straightforward calculation of the answer – so I thought I’d give it a go myself. The idea being that even if the end result is not particularly informative the thinking behind getting the end result is useful.

The key parameter of interest here is the gross domestic product (GDP): the amount of goods and services produced in a year in the UK; it’s a measure of how wealthy we are as a nation, how it increases with time is a measure of economic growth. Also important are the deficit (how much the government’s annual spending exceeds its income) and debt (how much the government is borrowing).

Inflation means that the GDP can appear to grow each year with no increase in real economic activity, therefore I decided to use “inflation adjusted” GDP figures. I also preferred to use annual GDP figures rather than quarterly ones.

To model this I took a starting point of a known GDP, debt, deficit and government spend which I then propagated forwards in time: I made the GDP grow by a fixed percentage each year, and assumed that government spending would be flat (I’m using GDP adjusted for inflation so I think this is reasonable). Assuming that the total tax take is a fixed proportion of GDP I can calculate the deficit and hence increasing debt in each year, I add the debt servicing cost to the government spending in each. Since I’m doing everything else in the absence of inflation I’ve used a debt servicing rate of 2% rather than the 5% implied by a £43bn debt interest cost in 2010 – this makes my numbers a bit inconsistent.

I’ve put the calculation in a spreadsheet here.

Given this model my estimate is that the UK would need to sustain GDP growth of 4.8% per year until 2020 in order to reduce the deficit to 0%. This 4.8% GDP growth brings in approximately an additional £30bn in taxes for each year for which the growth is 4.8%. During this time the debt would rise to nearly 80% of GDP and so the cost of servicing the debt will double. These numbers seem plausible and fit with other numbers I’ve heard knocking around.

To get a feel for how GDP has varied in the past, this is the data for inflation adjusted annual GDP growth in the UK since 1950:

The red line shows the “target” 4.8% GDP growth, and the blue bars the actual growth in the economic, adjusted for inflation. The data comes from here. What’s notable is that GDP growth has rarely hit our target and what’s worse, over the last 40 years there have been four recessions (where GDP growth is negative), so the likelihood must be that another recession before or around 2020 is to be expected.

In real-life we are actually using a combination of GDP, government spending cuts and tax increases to bring down the deficit. These calculations indicate 0.5% GDP growth is approximately £7bn per year which is equivalent to a couple of pence on basic rate (see here) or about 1% of government spending (see here).

Doing this calculation is revealing because it highlights why there is an emphasis on cuts in government spending as a means of reducing the deficit. This had been a bit of a mystery to me with the figure of 80:20 cuts to taxes ratio being widely quoted as some sort of optimum, although there is some indication of other countries working with a ratio closer to 50:50. The thing is that when you cut your spending, you are in control. You can set a target for reduction and have a fair degree of confidence you can hit that target and show you have hit that target relatively quickly and easily. On the contrary relying on growth in GDP, or taxes, is a rather more unpredictable exercise: taxes because the amount of tax raised depends on the GDP.

The Office for Budget Responsibility (OBR) published uncertainty bounds for it’s future predictions of GDP in their pre-budget report last year (see p10 and Annex A in this report), their central forecast is for growth of 2.5% but by 2014 (i.e. in only 4 years) they estimated only a 30% chance that it lay between 1.5% and 3.5% actually they only claim a 40% chance of being in that range for this year (2011).

At the risk of being nearly topical, GDP is reported to have shrunk by 0.5% in the last quarter of last year, 2010. This is largely irrelevant to this post, although forecasts for GDP were growth of ~0.5% which supports the idea that GDP is not readily predictable. It’s worth noting that the ONS will revise this figure at monthly intervals until they get all the data in – the current estimate is based on 40% of the data being available.

Given this abysmal ability to predict GDP I suspect that there is little governments can do to influence the growth in GDP. It would be interesting to estimate the influence government policy has relative to prevailing global economic conditions, and what timelags there might be between policy changes and growth.

I think these calculations are illustrative rather than definitive, and what I’d really like is for someone to point to some better calculations!

Which country is the UK like economically?

This blog post attempts to answer the economic question: What country is the United Kingdom like economically?

The question arises from discussions of deficit (how much the government’s annual spending exceeds its income) and debt (how much the government is borrowing). To use a digging analogy: debt is a hole, deficit is how fast you are deepening the hole. We can get a feel for how countries compare in this sense by plotting them as a function of their deficits and debts on a graph. I’ve done this for the countries of the OECD (data here). The horizontal axis tells you  the deficit, whilst the vertical axis is the debt. These values are plotted as a percentage of the gross domestic product (GDP) so we can compare big countries and small countries, rich countries and poor countries on an equal footing.

As we go to the top left area of this graph we find countries which are in the deepest hole, digging fastest. Out of the extreme left we find the UK – it is digging its hole fastest at the moment, but it is not in the biggest hole – that honour currently goes to Japan. On this graph our nearest neighbours are Ireland, the United States and Iceland, with Greece and Japan having higher debt but lower deficit. France and Spain have similar debts but rather lower deficits.

Norway is actually bringing in more money than it spends and building a surplus, they have large oil revenues and are saving against the day when it runs out. Korea is doing this too but to a far smaller extent.

Economically it is often said that the PIGS or PIIGS countries are in most trouble in Europe, these are Portugal, Ireland, (Italy), Greece and Spain.

But is it true to say we are like Ireland, the US and Iceland? Ireland and Greece have much higher levels of unemployment than us, whilst the US has slightly higher levels and Iceland’s levels are comparable. Iceland has very high inflation (12%) whilst most other countries have moderate inflation including the UK, with a few countries having moderate deflation including the US and Ireland. We have moderate levels of unemployment, with Ireland, Greece, the US having much higher levels. It’s also worth pointing out that the UK has a population of 60,000,000 whilst Ireland only has 4,000,000 and Iceland a mere 300,000.

So whilst in deficit/debt terms we may resemble other countries in other, economically relevant ways, we are quite different.

From a mathematical point of view there are methods for measuring the closeness of things based on large numbers of variables, called clustering algorithms. These algorithms amount to our eyeballing of the debt/deficit data – they are a measure of distance. However, in economic problems things aren’t so simple. The economy is described by many variables and I don’t know their relative importance in determining economic similarity. The problem that the numbers involved may vary tremendously in size is trivially solved. My suspicion is that economists have probably spent a great deal of time arguing about economic similarity and haven’t come to a definitive answer.

So in answer to the question: What country is the United Kingdom like economically? Although the UK may be most like Ireland, the US and Iceland in debt/deficit terms. In terms of many other economic factors such as inflation, unemployment and so forth it is quite different. I suspect the real answer to this question is that the UK is most like France, Germany and Italy economically: these countries are of similar size, have similar unemployment rates, have inflation of the same sign, usually run public sector/ private sector ratios of similar size furthermore given their common membership of the EU their economic behaviour is probably similar. Of this group of countries Italy has the largest debt and we have a largest deficit.

Poor attendance record in the House of Lords?

I know my readers love a chart, and today I found some data I thought was begging for a good graphing. It’s the attendance figures for the House of Lords found in a report entitled “Members Leaving the House” – found at the bottom of this article. The motivation for the report is to explore the idea of retirement for peers, something some peers are seeking regardless of any other changes taking place. A secondary motivation is that there is wider reform of the House of Lords proposed, and one of the issues is that the new House is envisaged, ultimately to have substantially fewer members – this type of discussion informs how that transition might be achieved.

The report contains a set of tables for the last five years indicating the fraction of sessions which peers attended broken down into groups:

• Attended 75% or more sessions
• 50% to 74%
• 25% to 49%
• 10% to 24%
• Attended at least once but less than 10%
• Zero attendance

This is what the data looks like:

To give some idea of scale: across the period shown here the total number of peers decreased from 777 to 741, the average number of sessions in a year was 140, this latter figure means that a peer attending “less than 10% of sessions” was attending less than twice. It compares with the number of working days in the year of approximately 240 (48*5 day weeks). Nearly 20% of peers attend a session in the House of Lords only once or twice a year.

Being a member of the House of Lords isn’t a proper job, it does not attract a salary, although peers may claim a subsistence and office allowance of up to £26,000 per year. In this sense we should not anticipate the levels of attendance achieved by those working “normally”. Some of the peers will be paid as government or opposition working peers. However, peers do have a direct effect on the laws the country makes and turning up twice a year (which is all 20% of them achieve) does suggest a fairly low degree of interest – if I did something twice a year I wouldn’t even consider it a hobby, I go to the dentist more often!

Yields from income tax

This post is a tour of income tax and personal national insurance yields, it’s motivated by an interest in seeing how one might pay for a part of the reduction in the deficit through taxation. The reason for focusing particularly on income tax is that it yields a fairly large fraction of the total tax income (28.7% in income tax and 46.6% income tax and national insurance combined), as discussed in a previous post; this means that relatively large amounts of money are raised by relatively small changes when compared to other taxes. Furthermore it’s relatively easy to calculate: I can work out how much income tax I pay in a year but would struggle to tell you how much VAT I pay per year, the impact of a tax on insurance premiums the effect of a change on duty and so forth. Thirdly, it is the tax that is most transparently progressive, in the technical sense that the more you earn the greater the fraction of your income you pay in tax.
This calculation is based on a calculation of personal tax rates from wikipedia, this figure generates the tax rates programmatically and I simply translated the code to my computer language of choice – I thought about doing it in a spreadsheet but that turned out to be a bit brainbending. The second component of the calculation is the number of people in each income bracket: this information along with further information on incomes can also be found on wikipedia. Ultimately the data come from the HMRC. I’ve put these two bits of data together into a program which enables me to fiddle with tax rates, tax thresholds and so forth. It appears to be approximately correct since it matches roughly HMRC’s own figures on the effects of small perturbations to the tax system (pdf). This also tells you it’s possible to look this stuff up – but I find it more fun to calculate it myself! It’s also a good illustration of the general process of how to go about repeating someones calculations from literature sources: try to reproduce their graphs; try to match the summary numbers they produce.
This first figure shows the income and national insurance payable as a fraction of gross (total) pay as a function of pay. The thing I hadn’t appreciated intuitively is that the tax banding system gives quite a smooth increase in percentage tax take, this is because you only pay raised rates on the fraction of your income that lies above the threshold:

Extending the horizontal scale out towards incomes of £1,000,000 and the rate tends to 50%. The next figure shows the distribution of incomes, in the UK:

You can see the same information in text form here. The area under this curve between points on the horizontal axis tells you the number of people in an income band. The median income in the UK is £26k per annum – half the population earn more than this, half less. About 1% of the population earns more than about £100k per annum. This final figure shows the amount that each income band pays according to the latest tax rates.

To summarise this final figure in tax bands, the 20% band accounts for about 57% of tax paid, the 40% band for 26% and the 50% band for 17%. These bands contain respectively 90%, 9% and 1% of the income tax paying population.
In case you’re curious my salary puts me close to the top of the basic rate tax band.
To apply the knowledge embedded in these graphs to some recent problems:
As a rule of thumb: 1p on basic gives about £4bn, 1p on upper rate gives £0.75bn, 1p on the new 50% band gives £0.31bn. The reason for this sharpish dropoff is that relatively few people are effected by the upper rate tax changes so to yield a large tax income the rates have to be changed by a relatively large amount.
Reducing the threshold of the 40% tax band to £40k from £43k yields about £3bn.
The £20billion cut in welfare benefits is equivalent to approximately 5p on the basic rate of income tax, taking it to 24.5% from 20%. This would cost me about £1700 per year.
Tuition fees cost about £7.5billion (based on 1.5 million students each requiring an average £5k tuition fees per year), this is about 2p on basic rate. This would cost me about £800 per year.
The £2.5billion income gained from cutting child benefit from those in the upper tax band could be paid for with an increase in the upper rate to ~43% from 40%. Although it seems the £2.5billion figure is dubious. I can’t help thinking simply increasing the upper rate by this amount, rather than a convoluted attempt at clawback would be simpler. This isn’t to say I support the idea of paying child benefit to all regardless of income, just that implementing withdrawal in this way is technically complicated. This tax rise wouldn’t cost me anything!
I’ve not seen anybody volunteering for these tax increases to support their favoured causes, rather they prefer a range of schemes of dubious value impacting other people to avoid the problem falling upon themselves – a subject for my next post.
Note
This modelling was done using Visual C# running under Windows 7, if you’re interested either in the code or in just the application then let me know in the comments below (or on twitter). There are a couple of minor bits of tidying I’d like to do before release. Please note that the application is “good enough for blogging work” and should not be considered an accurate tool for tax calculations – it’s a toy to help me understand things!

Children and numbers

One of this mornings news items is on government plans to limit benefit to a family to the average wage, apparently regardless of the size of the family. This seems to be built around the idea that there are families out there with vast numbers of children who are milking the system to the cost of the rest of us. We can check this idea with numbers. The graph below shows the number of claimants broken down by number of children in the household, the final category is for families containing 8 or more children.

The heights of the columns are a lower bound on the fraction of benefit going to each group, an upper bound would be to multiply each column by the number of children but this would be an over-estimate since benefits don’t increase linearly with number of children. There are a little under 1000 families with 8 children or more. 90% of claimant families have less than four children.
These data tell us nothing about the circumstances of each of the families represented which will include the loss of parents, illness, job loss and all the other small disasters which can befall a family.
The data shown here are from Department of Work and Pensions via The Spectator (here).

I was up for Evan Harris

This is a graph that shows you the number of seats (actual seats) each of the three main parties will get*, and the number of seats (proportional seats) they would get under a pure proportional system. You notice for the Labour and Conservative parties the number of seats they actually get is more than the number of seats in proportion to their votes, for the Liberal Democrats the opposite is true and by a very substantial margin.

When Liberal Democrats went into the polling stations yesterday they were given a single polling card, their Labour and Conservative comrades had three. Look them in the eye, ask them:

What is it about you that makes your vote three times more powerful, three times heavier, three times more important than mine?

What is special about you but not about me?

Explain to me how this is fair.

Explain to me how this is democracy.

To put it another way, every Labour or Conservative seat requires about 33,000 votes to win, a Liberal Democrat one requires 100,000 votes. We are the Great Ignored.

We have come to accept this inequity, it's happened in every election since the early 80's. As a country we just accept it as part of the way things are. It's the defining feeling of being a Liberal Democrat, seeing the overall share of our vote creep up election by election and receiving the same feeble, disproportionate harvest in seats. The sinking feeling in the middle of the night that, no, of course there has been no breakthrough. It's not because we perform poorly, it is because we have one polling card each, the others have three.

In 1997 the defining moment was Michael Portillo losing his seat to Stephen Twigg. My defining moment for this election was seeing Evan Harris lose his Oxford West and Abingdon seat. "I was up for Evan Harris", I had a tear in my eye.

Footnote
*This is based on the exit poll (see entry at 23:11), which looks consistent with the results of the actual election as of 10:30am May 7th which are Conservative 291, Labour 247, Liberal Democrat 51 616 of 650 seats declared. Under pure proportionality UKIP would receive 20 seats, the BNP 12.

No sleep 'til Batley!

I'm planning on staying up late tonight, watching the results of the general election come in, this is an occasion for a graph. The chaps at tweetminster have uploaded a list of predicted declaration times here. I've rearranged the data a bit to plot it, the height of each bar tells you the number of constituencies declaring during the hour starting at the time indicated at the bottom of the bar. As you can see, things don't really get going until about 2am. Key times for me are the declaration in my own constituency, City of Chester, at 3am and Dr Evan Harris' Oxford West and Abingdon at 2:30am. Batley & Spen declares at around 5am hence the title of this blog post.

This has been the most exciting election campaign, and election night, in quite sometime. I spent the 1997 election at a friends house in Darlington, I remember stumbling out into the early morning with "Things can only get better" ringing in my ears. For a few years that seemed to be the case. 1992 was interesting in that we all thought John Major was going to lose, and then he won to the surprise of everyone (including John Major). 2001 and 2005 were rather dull.

As a seasoned Liberal Democrat I'm used to my party getting pretty good percentage poll scores overall and winning pitiably few seats, so to the newcomers out there - welcome to my world! I can only hope that this time things will be different.

Opinion polls and experimental errors

I thought I might make a short post about opinion polls, since there's a lot of them about at the moment, but also because they provide an opportunity to explain experimental errors - of interest to most scientists.

I can't claim great expertise in this area, physicists tend not to do a great deal of statistics unless you count statistical mechanics which is a different kettle of fish to opinion polling. Really you need a biologist or a consumer studies person. Physicists are all very familiar with experimental error, in a statistical sense rather than the "oh bollocks I just plugged my 110 volt device into a 240 volt power supply" or "I've dropped the delicate critical component of my experiment onto the unyielding floor of my lab" sense. 

There are two sorts of error in the statistical sense: "random error" and "systematic error". Let's imagine I'm measuring the height of a group of people, to make my measurement easier I've made them all stand in a small trench, whose depth I believe I know. I take measurements of the height of each person as best I can but some of them have poor posture and some of them have bouffant hair so getting a true measure of their height is a bit difficult: if I were to measure the same person ten times I'd come out with ten slightly different answers. This bit is the random error.

To find out everybody's true height I also need to add the depth of the trench to each measurement, I may have made an error here though - perhaps a boiled sweet was stuck to the end of my ruler when I measured the depth of the trench. In this case my mistake is added to all of my other results and is called a systematic error. 

This leads to a technical usage of the words "precision" and "accuracy". Reducing random error leads to better precision, reducing systematic error leads to better accuracy.

This relates to opinion polling: I want to know the result of the election in advance, one way to do this would be to get everyone who was going to vote to tell me in advance what their voting intentions. This would be fairly accurate, but utterly impractical. So I must resort to "sampling": asking a subset of the total voting population how they are going to vote and then by a cunning system of extrapolation working out how everybody's going to vote. The size of the electorate is about  45million, the size of a typical sampling poll is around 1000. That's to say one person in a poll represents 45,000 people in a real election.

To get this to work you need to know about the "demographics" of your sample and the group you're trying to measure. Demographics is stuff like age, sex, occupation, newspaper readership and so forth - all things that might influence the voting intentions of a group. Ideally you want the demographics of your sample to be the same as the demographics of the whole voting population, if they're not the same you will apply "weightings" to the results of your poll to adjust for the different demographics. You will, of course, try to get the right demographics in the sample, but people may not answer the phone or you might struggle to find the right sort of person in the short time you have available.The problem is you don't know for certain what demographic variables are important in determining the voting intentions of a person. This is a source of systematic error, and some embarrassment for pollsters. 

Although the voting intentions of the whole population may be very definite (and even that's not likely to be the case), my sampling of that population is subject to random error. You can improve your random error by increasing the number of people you sample but the statistics are against you because the improvement in error goes as one over the square root of the sample size. That's to say a sample which is 100 times bigger only gives you 10 times better precision. The systematic error arises from the weightings, problems with systematic errors are difficult to track down in polling as in science.

So after this lengthy preamble I come to the decoration in my post, a graph: This is a representation of a recent opinion poll result shown in the form of probability density distributions, the area under each curve (or part of each curves) indicates the probability that the voting intention lies in that range. The data shown is from the YouGov poll published on 27th April. The full report on the poll is here, you can find the weighting they applied on the back page of the report. The "margin of error" of which you very occasionally hear talk gives you a measure of the width of these distributions (I assumed 3% in this case, since I couldn't find it in the report), the horizontal location of the middle of each peak tells you the most likely result for that party.

For the Conservatives I have indicated the position of the margin of error, the polling organisation believe that the result lies in the range indicated by the double headed arrow with 95% probability. However there is a 5% chance (1 in 20) that it lies outside this range. This poll shows that the Labour and Liberal Democrat votes are effectively too close to call and the overlap with with the Conservative peak indicates some chance that they do not truly lead the other two parties. And this is without considering any systematic error. For an example of systematic error causing problems for pollsters see these wikipedia article on The Shy Tory Factor.

Actually for these data it isn't quite as simple as I have presented since a reduction in the percentage polled of one party must appear as an increase in the percentages polled of other parties.

On top of all this the first-past-the-post electoral systems means that the overall result in terms of seats in parliament is not simply related to the percentage of votes cast. 

Occupations of MPs

Ever alert to the possibility of finding some data to play with I was interested in an article in the Times regarding the number of MP's with scientific backgrounds in parliament. First I found data on occupations in the population as a whole here (Office of National Statistics) and data on MP's here, published by parliament. I thought it would be interesting to compare the two sets of figures, this turns out to be rather difficult because they define occupations very differently so I had to do a bit of playing about to get them into roughly comparable form.

This is what I came up with in the end:

It's a "representation factor", that's to say I take the fraction of MP's in parliament having a particular occupation and I divide it by the fraction of that occupation in the general population. If that occupation is over-represented in parliament then the number is bigger than one and if they are under-represented then it's smaller than one. It would seem barristers, journalists and career politicians are massively over-represented. Lecturers, civil servants and teachers are a little over-represented. Business people are about as expected and doctors are under-represented (along with manual workers and white collar workers).

I think from all of this the figure on doctors is the most surprising. It does make you wonder about how useful the outside interests of MP's are in guiding their deliberations since most occupations are grossly under-represented. You shouldn't really expect to see the House of Commons faithfully representing the overall working population, but I expected the balance amongst professionals to be a bit more uniform.

The House of Commons library document on "Social background of MPs" from which I got the MP occupation data is rather interesting, in particular the age profile (table 1) appears to be shifting upwards despite the greater youth of the party leaders. The educational background (table 7) is quite striking too.

One of the glories of the internet is that data monkeys like me can find tasty fruit to pick and consume.