A brief return to politics - the Budget

Following on from my pre-Budget "Sceptical look at the economy", I thought I'd return to politics and the Budget.

The financial position seems to be largely what was expected before the election and the size of the proposed cuts seems consistent with the scale of cuts in Spain, Greece and Ireland.

What I would have done? I suppose I prevaricated in my last post on what I would have done in the recent Budget. To be a bit more explicit: I would have put probably something like 3p on basic rate tax, lifted the lower threshold of basic rate and brought down the threshold to the higher rate. And looked to cutting something like 15% across government spending with no ring-fencing. I may have put up capital gains tax a bit more at the higher rate and not reduced corporation tax - but to be honest these measures don't bring in much cash anyway. As for benefit cuts, I'd probably have gone for means-testing things such as child benefit, winter fuel allowance and so forth. My impression is this would approximately fill the appropriate gaps (but I haven't done any calculation).

But then nobody voted for me, and the Liberal Democrat experience is if you offer the voting public an increase in income tax they say how great this is, and how they'd really love you to spend the money where you've said you'll spend it, and then vote for someone else who has promised not to raise income tax. Of the national parties the Green Party manifesto was the only one to imply they would not make any cuts, but increase overall taxation to cover the structural deficit; electorally the Green Party didn't do that well in the General Election with about 1.0% of the vote.

As it stands the Budget was somewhat different from my preferred option. There are a few mitigating factors but I'm not convinced that VAT rises are a good way to raise tax (it has been suggested that they are better than income tax rises because they do not fall on essentials and they are "voluntary" to a degree, which income tax rises most definitely are not). It seems rather notable that there was much symbolic "dipping of the hands in the blood" by Nick Clegg, Danny Alexander and Vince Cable, you'd have though the Tories would have been a bit more forthcoming about defending a budget in which they were the majority partner and which largely matched their electoral commitments.

The Labour Party has started pointing out that this is a very political Budget, that's true, and so was their idea of defering cuts into next year. For the Opposition this has the positive political benefit of not needing to be clear about what you would do until well after the General Election (and not even then) and allows you free-reign to criticise cuts by the incoming government without proferring an alternative because obviously you'd be doing this next year when things would have become magically better.

I've come to the conclusion that macroeconomics is almost entirely about politics, and the vehemence with which economic opinions are presented leads me to believe that everyone realises they don't actually know what they're talking about and that by shouting loudly they can get away with it. Presumably MP's and ministers feel they have learnt to run the economy through the odd lecture course on the infamous Politics, Philosophy and Economics undergraduate degree course at Oxford. It seems notable that prior to the election the global consensus appears to have been for "economic stimulus" and after it is for "deficit reduction" (with the exception of the US). I'm not clear how this has happened, because I can't believe it's entirely driven by the UK election.

Inferring what the voting public want from elections and opinion polls is always a tricky business but the evidence seems to be they're happy with the Budget and it's pretty much what they expected. I suspect the reason for this is that the majority of them will be in the private sector and over the past few years the companies they work in would have laid people off, been on pay freezes and, over a longer period, treated employees less generously in pension terms but this largely hasn't happened in the public sector. The same opinion poll shows fairly good support for maintaining the state pension whilst "cutting benefits for those of working age". 

The Office of Budget Responsibility is pretty upfront in saying it's estimates for GDP growth are subject to large uncertainty (see p10 of this report, and also Annex A on how figures are derived - hat-tip to Christopher Cook for that). The biggest problem seems to be that recession are utterly unpredictable. I'd be interested to see similar analysis for unemployment figures - can't help thinking they're not going to be good.

My useful pieces of contextual information for the day: UK employed population is about 30million, of which about 5million are in the public sector.

Some notes on SQL: 3 - changing a table

This is the third in a series of blog posts on SQL, the first covered creating a database, the second selecting information from a database. This post covers commands to modify the structure and contents of an existing database. No claim of authority is made for these posts, they are mainly intended as my notes on the topic.

UPDATE and DELETE allow the rows in a table to be either updated or deleted according to a select-like WHERE clause. This is UPDATE, acting on multiple columns:

UPDATE your_table
SET    first_column = 'newvalue',
       second_column = 'another_value'
WHERE  some_column = 'a test';

And this is the DELETE command:

DELETE FROM your_table
WHERE  some_column = 'a test';

In combination with the ALTER keyword, the following operations can be performed:
The CHANGE keyword allows the name and data type of an existing column to be changed.

ALTER TABLE project_table 
CHANGE COLUMN a_silly_column_name a_better_column_name VARCHAR(100), 
CHANGE COLUMN another_poorly_named_column a_better_name VARCHAR(30);

It's necessary to be cautious here because data loss can occur depending on the source and destination types, for example going from VARCHAR(100) to VARCHAR(30) could potentially lose 70 characters.
The MODIFY keyword allows the data type or position of an existing column to be changed.

ALTER TABLE my_table 
MODIFY COLUMN target_column VARCHAR(120), 
MODIFY COLUMN another_column AFTER target_column;

The ADD keyword allows new columns to be added to a table:

ALTER TABLE my_table 
ADD COLUMN new_column INT NOT NULL AUTO_INCREMENT FIRST;

The ADD and MODIFY keywords take position identifiers: FIRST, LAST and BEFORE, AFTER - which require a second column identifier as indicated in the MODIFY example.

In addition RENAME TO allows the table to be renamed:

ALTER TABLE poor_name RENAME TO good_name;

And DROP deletes a column:

ALTER TABLE my_table DROP COLUMN unwanted_column; 

Obviously you should use DROP COLUMN cautiously!

Keywords: ALTER, UPDATE, CHANGE, MODIFY, ADD, DELETE, AFTER

Book review: Botany of Desire

This post can be read in full here: http://www.ianhopkinson.org.uk/2010/06/book-review-botany-of-desire/

Yet another in my erratic series of book reviews cum notes. This time I'm reading "The Botany of Desire:A Plant's-eye View of the World" by Michael Pollan.

The introduction lays out the land of the book, sections on apples, tulips, marijuana and potatoes and the central thesis: that it's a useful idea to consider that not only do we domesticate plants but that in a sense plants naturalise us. As stated in the introduction this thesis felt a bit hardline, grating a little for my taste but once into the reading this feeling receded since the illustrative stories are enticing and nicely written.

First up, are is the story of apples in American and the folk hero, Johnny Appleseed, who travelled the mid-West, setting up ad hoc orchards from seeds, a little way in front of the settler-wave, and sold them trees as they moved into the area.

The point about apples is that they don't grow true from seed, take a fine apple and plant its seed and what you get is a lucky dip. This is a recurring theme, plants amenable to domestication appear quite often to be those amenable to quickly producing a wide variety. To grow "true" from an apple you need to graft from the parent onto a root stock. It's always struck me as something of a miracle that grafting works and that people managed to discover it.

Apples were significant to the early settlers since they offered sweetness (sugar would not have been very available), a sense of order when planted in neat orchards and cider. It seems cider played a big part in the popularity of Johnny Appleseed during his life, since the apples grown from seed were most often best suited to cider-making rather than eating. After he died the temperance movement gained strength in the US, and this aspect of apple cultivation was pushed into the background.

Despite the focus on Johnny Appleseed (and comparisons to Dionysus) the thing that will remain with me from this section is the descriptions of the wild apple forests around Alma-ata in Kazakhstan. You can get a flavour of the place from the fabulous images here, in an article in Orion Magazine and here, on the BBC website. These wild trees are important because they represent massive genetic diversity. The drawback of grafted plants is that they are genetically identical to their parents, so over time they become more and more susceptible to pests and diseases which evolve freely to take advantage of their stasis.

After the apples come the tulips, and Tulip Mania amongst the unlikeliest of enthusiasts: the Dutch. Tulips are a relatively recent addition to the pantheon of flowers, unlike the rose and the lily which appear in the Bible, tulips appear to have been introduced to Europe from Turkey in around 1550.

Interesting thought from this section: flowers became beautiful before there were ever humans to appreciate them - in a sense flowers are the result of the aesthetic decisions of bees (and other pollinating insects).

Tulip Mania was a speculative bubble in the Netherlands slightly before the middle of the 17th century wherein the prices paid for tulip bulbs skyrocketed, a single bulb fetching the equivalent of a acres of land or a fine townhouse, only to crash thereafter.

The flower in the picture to the left is Semper Augustus, emblematic of the most valued of the tulips during tulip mania. The interesting thing is that the most prized of these flowers - those that had "broken"  - were actually suffering the effects of a virus from which their line would eventually weaken and die. "Broken" refers to the variegated appearance with a dark colour, appearing in streaks on a lighter background. The modern Rembrandt tulips are similar in colouring but, according to Pollan, less impressive than the best of the virus "broken".

A common theme through all these stories is the large variability of the species from which the domesticated cultivars are drawn and the vulnerability of the much more uniform varieties once domesticated.

The third section is devoted to marijuana, clearly a plant for which the author has some fondness. Marijuana has long been cultivated for two reasons: one for fibre as hemp, and one for drugs. Since the early 80's and the American "War on drugs" marijuana production has been pushed underground, or rather indoors. Pollan recounts the story of the recent cultivation of marijuana by Dutch and American growers. The plant has undergone fairly rapid change in the last few years with the crossing of the large, traditional cannabis sativa and the more compact, frost resistant cannabis indicas. A substantial amount of work and horticultural ingenuity has gone into this process, leading to plants that can produce high yields of the active material in small, indoor spaces. The prize being the $13,000 that a hundred plants grown on a 6 foot square table can yield in a couple of months.

For Pollan there is an element of horticultural challenge in this process, he clearly grows a wide range of plants  in his own gardens (from each of the sections of this book) valuing the challenge and the diversity. The garden at SomeBeans Towers is similar: more a plantswoman's garden than a designer's garden.

He digresses at length on purpose of intoxication and whether drug taking really does open the doors of perception, or just lead to inane blithering, falling eventually for the former. There's an interesting section on the neuroscience of cannabis.

The book finishes with a chapter on potatoes, in particular on a genetically modified potato called NewLeaf which was developed by Monsanto to express the pesticide from the Bacillus thuringiensis bacteria (Bt). Organic certification schemes allow the limited 'manual' application of the Bt pesticide. In this chapter he visits various potato growers, spanning the ultra-technological to the organic. He highlights the dilemma that he finds GM potatoes more palatable than the non-organic equivalent when presented with the choice, in large part because the level of inputs, in particular fungicides and insecticides, to conventionally grown potatoes is very high.  His visit to an organic highlights something from the organic movement in which I'm in favour: which is a willingness to explore different methods of cultivation (and a wider range of cultivars), where I part company is where they say "There must be no X" where X is a somewhat arbitrarily drawn list, enforced with religious fervour.

The section also covers the history of the cultivation of the potato, from the wide variety in the mountain gardens of its native Peru, to its introduction into Europe as a favoured staple crop. Prior to the introduction of the potato bread was the staple food in Europe; wheat is somewhat fussy in its growing conditions particularly in Northern Europe and getting bread from wheat is quite an involved process. Potatoes, on the other hand, are less fussy on growing conditions and exceedingly simple to prepare for eating (stick in fire and wait, or if feeling extravagant: boil in water).

Overall I enjoyed this book, each section seemed to divide into two unlabelled parts one largely factual and one rather more philosophical - I preferred the more factual sections but appreciated the philosophical too.

Some notes on SQL: 2 - Basic SELECT

Part 1 of this sequence of blog posts provided a preamble and showed how to create databases. This post introduces the basic SELECT command, which shows you what lies within your database and as it's name implies allows you to select only parts of the data contained within.

The basic form of SELECT is:

SELECT * FROM my_contacts
WHERE first_name = 'Anne';

* indicates that all fields should be returned from the table 'my_contacts', where the first_name field = 'Anne'. We don't have to take all the fields from a table:

SELECT first_name, last_name, email FROM my_contacts
WHERE first_name = 'Anne';

As well as the equivalence operator =, we can also use comparison operators <> (not equal), <, >, <=, >= these work not only with numerical values, but also with text values. WHERE clauses can also be combined with AND and OR operators.

SELECT drink_name FROM drink_info
WHERE calories >= 30
AND calories <= 60;


SELECT drink_name FROM drink_info
WHERE calories BETWEEN 30 AND 60;

The second select using the BETWEEN keyword is equivalent to the first.
In addition there are wildcards, % meaning 'any number of characters' and _ meaning 'one character' which are accessed via the LIKE keyword:

SELECT first_name FROM my_contacts
WHERE first_name LIKE '%im';

This first search will return 'Tim', 'Slim', and 'Ephraim'.

SELECT first_name FROM my_contacts
WHERE first_name LIKE '_im';

This second search will only return 'Tim'. NULL is special, nothing equals NULL but you can check if something is NULL:

SELECT first_name FROM my_contacts WHERE flag IS NULL;

Comparisons can be made to a list with the IN keyword:

SELECT drink_name FROM drink_info
WHERE rating IN ( 'good', 'excellent', 'average' );

Finally, the NOT operator can be used to find the inverse of the selection made, the NOT keyword goes directly after IN but otherwise goes after WHERE:

SELECT drink_name FROM drink_info
WHERE rating NOT IN ( 'good', 'excellent', 'average' );


SELECT first_name FROM my_contacts
WHERE NOT first_name LIKE '_im';

Keywords: AND, OR, BETWEEN, IS NULL, NOT, LIKE, IN

Some notes on SQL: 1 - creation

These are some notes on SQL a language for creating and querying databases, I'm learning it because a couple of programming projects I have in mind for work and home will need it. The source for these notes is  Head First SQL book. I've used a previous book in this series and I quite like the presentational style. I'm using MySQL via it's command line client to do the exercises, because it's about as straightforward as you can get. The code is formatted for display using this SQL to HTML formatter. Notepad++ recognises SQL as a language and will do syntax highlighting, so I type my commands into it and copy them into the MySQL command line client.

SQL is quite an old language and the convention is to write keywords in block capitals (reminds me of FORTRAN!). Command sequences are terminated by a semi-colon.

To start, this sequence creates a database, sets it as active and then adds a table containing a range of fields of different types, the command DESC shows the layout of a table:


CREATE DATABASE my_database;
USE my_database
CREATE TABLE contacts
(
contact_id INT NOT NULL,
first_name VARCHAR(20),
last_name VARCHAR(20),
birthday DATE,
life_story BLOB,
weight DEC(3, 2) NOT NULL DEFAULT 80.00,
state_code CHAR(2),
appointment DATETIME
);
Desc contacts;



NOT NULL keywords are used if a field must be specified on INSERT. Once created data can be added to the table using the INSERT command:

INSERT INTO contacts (contact_id,first_name,last_name,birthday,life_story,weight,appointment)
VALUES
(1,'Ian','Hopkinson','1970-24-04','A very long text string',80.0,
'10:30am 2010-21-06');


INSERT can be used with no specified fields (in which case values for all supplied fields must be provided for all fields), or with a subset of fields. In order to add the ' character we can either use '' or \' (where '' is two single quote characters, rather than a double quote).

To delete a table:

DROP TABLE contacts;

This command should be used with care since it deletes the table whether or not it contains data. The next post should be on the SELECT command. 


Of course you can find SQL cheatsheets elsewhere.

Keywords: CREATE, DATABASE, TABLE, USE, DESC, DROP, INSERT INTO, VALUES, INT, VARCHAR, BLOB, CHAR, DATETIME, DATE, DEC

A sceptical look at the economy

This blog post was written partly because I'd got fed up with hearing about how all cuts were evil, without hearing an alternative plan. It has ended up more a collection of interesting data sources, and some mild ranting.

First to define some terms: The gross domestic product (GDP) is a measure of the total economic output of the country. It's handy because we can use it to compare any other number we come up with to see how big they are. £1billion may sound like a lot, but the GDP is £1.5trillion, so £1billion is a less than 0.1% of GDP.

The debt is the total amount of money that the state owes; the deficit is the annual gap between what the state takes in taxes and what it spends. A debt is sustainable in the long term but running an annual deficit above a certain size, for a period, is not sustainable. The deficit can be divided into two components: a cyclical component which goes up and down with current economic conditions and a structural component which is on top of this. The structural component is the bad bit. There is some dispute over the validity of this division since economic cycles are not easy to define.

As Mr Micawber says in David Copperfield: "Annual income twenty pounds, annual expenditure nineteen nineteen six, result happiness. Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery."

I'm a big fan of the Guardian's government spending chart (see the image at the top of page), it's a rather pretty way of seeing where government spending goes. For the year 2008/9 the total spending is £621bn, in this year the Treasury gets an enormous, anomalous amount in financial stabilisation (18%) - this will not recur in future years. Beyond this there's the Department of Health, spending about 18%, Department of Work and Pensions (22%) with the largest fraction of this going on the state pension, Department for Children, Schools and Families (10%). The key point here is that a very large chunk of the money spent is spent on things that people very vocally want (schools, health care, state pensions).

The figures for where tax comes from are perhaps a little surprising, extracting the data from table 1 in this report by the Institute for Fiscal Studies. The major chunks are shown in the piechart below:

The "other" category is made up of various minor indirect taxes (tobacco, alcohol etc) and capital taxes (3%). The surprising thing to me was the relatively low level of corporation tax. There seems to be evidence of tax avoidance by corporations amounting to something in the region of £10bn, but this would only be roughly 1.6% of the tax take.

By the way, as a physicist, I look down on piecharts!

The total tax take of around 36% of GDP puts the UK roughly in the middle of the OECD table of taxes, with countries like New Zealand and Germany taking very similar levels of tax, France, Italy and Finland taking rather more (at around 44%) and countries like Ireland, Switzerland and the United States taking rather less at ~30%. The full OECD data is here, and wikipedia has a sortable list for all the countries of the world. An interesting exercise is to consider this list, and think in which countries you might want to live.

Of income tax the top 1% of earners pay nearly 25% of all income tax, you can see the full breakdown in this article on the BBC website. Put another way, the 40% tax band covering perhaps 15% of tax payers provides nearly 40% of income (here). This does exclude National Insurance contributions though. I was surprised by these figures, I'd assumed that the relatively small number of higher tax rate payers would result in a much lower total take.

The net result of these incomings and outgoings is that we had a national debt of 68.1% of GDP at the end of 2009, and a total deficit of 11.4% of GDP. (According to the Office of National Statistics). According to the newly formed Office of Budget Responsibility  (table 4.5) the national debt (which they call net debt) is 53.5% this year and the deficit (net borrowing) is 11.1% of which 8.8% is structural. 

Numbers without context are meaningless: a priori I have no idea how these numbers for debt and deficit compare to the past. However, help is at hand: this report shows how they have varied over the past 100 or so years. I've copied the key figure for national debt below:

At the end of the second world war the UK had a national debt of around 250% of GDP, much larger than our current debt (and even our predicted debt over the next few years). Interestingly we see in the same report (figure 1) that the deficit is rarely negative (i.e income greater than expenditure), hovering around 2% (i.e. still a deficit) debt is still paid off via growth in the economy and inflation.

I suppose the purpose of all of this preamble is a discussion of cuts, or if you prefer tax increases. Prior to the election everyone seemed to agree on the size of the gap to be filled but none of the parties managed to fill more than 25% of the gap, as evidenced in this report by the Institute of Fiscal Studies, the Guardian's data blog had a nice breakdown of the measures proposed by the three main parties. The major political point of departure was when cuts should start (not if cuts would start), and my view prior to the election was that whoever won broadly similar levels of cuts would be made although there was some evidence that the balance between taxation and cutting would be different depending on party but since none of them revealed (or had) much of their plan it's rather difficult to say.  It is a very minority view that no cuts are required, although I see the unions are trying that one out today along with threats if there are any cuts.

YouGov,  for the Sunday Times, helpfully asked the public:

"The government has asked for public advice on where it should cut public spending. Which, if any, of the following areas do you think should be targeted for cuts? Please tick up to three". 

And the public demonstrated that if you ask a stupid question, you get a stupid answer (or, being generous to the public, if you ask a question without providing contextual information you get a stupid answer). I considered trying to find a fancy way of presenting this information, but in a nutshell: by far the most popular area for cuts (61%) is in international aid whose total budget is 0.8% of total spending (i.e. pretty much the smallest bit of the budget you can find).

To be fair to the public, many of them will be working in the private sector and will have variously experienced pay cuts (or at least freezes), reduced working, redundancies, budget reductions and frozen recruitment and they may well be feeling it's someone else's turn.

You can experiment with cuts yourself with this handy tool from the FT, have a play and think about how you'd stand up and justify the cuts you've made. To paraphrase Polly Toynbee: "Don't be young, old, vulnerable, one of our brave boys, sick etc".

The alternative to cuts are tax increases, but nobody seems keen to talk about them. Prior to the election there was a report stating that the deficit was equivalent to about 6p on the basic rate of income tax. Proposals to raise tax were normally described as a "Tax on jobs" or "Death tax", which is unhelpful to say the least. Another popular idea is to tax the bankers, one option here is the Tobin or Robin Hood Tax which puts a small tax (typically fractions of a percent) on every financial transaction, because there are very many of these transactions potentially the amount raised could be large this would seem to require international coordination and it isn't clear where the money raised would be spent (climate change, international aid, fund for future bank collapses have all been suggested). The banking sector contributes approximately £70bn to GDP, or 6.8%. The structural deficit isn't about any money spent rescuing banks though, it's about an ongoing gap between spending and taxation.

My personal view is that we should be talking about taxation, and where the balance between cuts and increased taxation should lie (currently it looks like 80/20 cuts to taxes). There should be some discussion of where tax rises are best levied : "on someone else" isn't really a proper answer. Income tax seems like the best place to me, probably at basic rate with uplift of the lower threshold to protect some of the least well off, but possibly lowering the threshold to the upper tax bracket. In the longer term making the public sector more flexible to economic hardship would be nice, this time there seems to have been much more flexibility in how companies have approached recession - not necessarily painless, but better than losing your job. One element of this could be variable pay in the public sector (or bonuses as we colloquially call it) this provides two things to an employer: the ability to vary pay when income to the company is poor and some decoupling of current salaries from pension entitlements (since bonuses are typically not counted towards pension payments).

So to end on a happy note: I propose bonuses for the public sector!

* Update: hat-tip to AlexConner who pointed out that it is Mr Micawber not Uriah Heep who is responsible for the quote from David Copperfield. 

Compare and contrast

I thought I might try describing my job as an academic in a physics department, and comparing that to my current work as an industrial scientist.

Some scene setting: in the UK undergraduates are students who study taught degree courses lasting 3-4 years, typically they start at age 18 or 19. Postgraduates are studying for PhD's, research courses lasting 3-4 years (after which research councils start getting nasty). After PhD. level there are postdoctoral workers who typically do contract research lasting 2-3 years per contract - they may do multiple contracts at an institution but it's a rather unstable existence. Permanent academic staff are lecturers, senior lecturers, readers and professors in increasing order of seniority/pay.

As a lecturer-level academic, the shape of the year revolves around teaching, if not the effort involved. Undergraduate students start their year in September, with breaks over Christmas and Easter followed by exams in May/June. The teaching year amounts to about 30 weeks. Should you be lecturing the students, you will spend time preparing and giving lectures; how long this takes depends on your conscientiousness, the number of times you have lectured the course and the number of other things you have to do. In addition you will probably give tutorials, small groups of students working through questions set by other lecturers, practical classes and manage final year undergraduate projects and literature surveys. Compared to a school teacher or further education college lecturer your "contact" time with students will be relatively low - maybe 10 hours a week.

Final year projects are of particular interest to you as a researcher since there's always vigorous competition amongst academics to attract the best undergraduates to do PhD.'s as postgraduates. A final year project done by a good student can be an excellent way to try an idea out. To be fair to students though, their performance in a final project and talking about that final year project can be the strongest part of a CV - since it demonstrates the ability to work individually in an unknown area.

In between undergraduate teaching there's grant application writing, doing research of your own, writing papers, and then, come the end of term, the possibility of conferences.

In the end it was the apparently endless futility of writing grant applications which did for me as an academic. My success rate was zero, furthermore I had this terrible feeling that even after successfully winning a grant I would struggle to recruit postdocs or PhD students to do the work and there was little chance that having started a fruitful line of research there would be a good chance of continuing it with further successful grants.

I was recruited to my current company by a recruitment agency, who found my webpage still hanging around at Cambridge University a couple of years after I had left. I didn't actually end up doing the job they nominally recruited me for but what I do is relevant to my research background and can be rather interesting.

I turned up to my new workplace on the Friday before I started and was shown my desk - in a shared office. I did wonder at that point whether I had done the right thing, back in academia I had an office roughly the size of a squash court and could go days without seeing anyone. As it turns out sharing an office isn't too bad, you get to find out what's going on, but it can be a pain when your neighbour decides to have a long, detailed meeting next to you.

Another novel aspect to working in industry is that someone seems interested in my career within the company. In getting on for 15 years as an academic I can remember rarely ever talking about my career with anyone who might have influence on its direction whilst in a company it's at least an annual occasion. It's true that the company's enthusiasm for management-speak can be excessive (and changeable) as new human resources fads come and go.

I get to go to lots of meetings.

Relevant to current discussions on the public sector we have regular restructuring, and in the past year or so: pay freezes, arbitrary cuts in travel budget mid-year, a change to pensions for new recruits, and redundancies - the latest round equivalent to losing about 15% of the people on the site I work at. It's fair to say that we are not necessarily models of efficiency internally: I heard on the news that it takes 5 signatures for someone in the NHS to buy a new bed costing about £1000 - sounds about par for the course.

One noticeable difference is that largely I feel much more wanted, inasmuch that if I'm put on a project then the project leader will be keen to get some sort of intellectual exertion on my part and will even appear quite pleased when this is achieved. Even better, people for whom I do "a bit on the side" are even more grateful. This is a big difference from being an academic, where the odd student (undergraduate or postgraduate) may appreciate your efforts but largely nobody shows much sign of caring about your research.

Looking back on my time as an academic: I think I would have benefited from some sort of master plan and career direction. I'd quite liked to have carried on as a postdoc, i.e. actually doing research work rather than trying to manage other people doing research. However, this isn't a career option and is a rather unstable existence.

That's nice, dear

This blog post is about programming, for people that don't program - at least that's the effect I'm aiming for. The title is in recognition of my tolerant wife, The Inelegant Gardener, who has learnt the appropriate response to my enthusiastic displays of the results of my programming: "That's nice, dear"!.

I started programming a long time ago - in around 1980, at the school computer club, when I was 10. Since then I've been taught odd bits of programming by scientists, and done quite a lot of programming as part of my scientific job. I've started to get more interested in proper software engineering in the last few years. This is a roundabout way of saying I am an enthusiastic amateur.

People associate programming with the mathematically minded, but this isn't necessarily the case: the codebreakers at Bletchley Park, who were amongst the first users of electronic computers, had a range of skills - amongst them were linguists and crossword wizards. I was talking to a Fellow in linguistics, who'd helped write his college's library software - as he pointed out: a very logical view of language is a great benefit for a programmer. Programming is about giving an idiot very exact instructions, if the instructions concern maths then you need to know maths - otherwise you don't.

The core of programming is still what I learned years ago, data (numbers or letters) is stored in "variables" that have names. There are conditional statements: "If [something is true] Then [do this] or else [do the other]". There are looping statements: "Do this 100 times". And there are functions: "add 2 to this number, square it, add the number you first thought of and tell me the answer" or "how many times does the letter a occur in this sentence".

These simple statements are being buried under an increasing depth of additional ideas. Since the 80's the big thing in programming has been "object-orientation". In object-orientated programming you package up data of a particular sort with functions that relate to that data. So if you had data modelling an octopus you would include functions such as "wave-tentacles" and "change colour", such functions would be useless for data describing a horse. The real benefit to this is comprehending larger software systems, because a sea of functions and data is grouped together into logical islands. Beyond this there are design patterns - reoccurring systems of objects which I haven't entirely go the hang of.

In addition to the changes in language, there are changes in the tools used to program: syntax highlighting is nice, it amounts to colouring the verbs, nouns and proper names in programming in different colours - makes it easier to spot mistakes. Auto-completion is another handy tool, in a well-designed language there are only a limited number of next possible statements when you are programming - auto-completion presents you with them as you type. Sites like Stackoverflow are great for asking programming questions, and there no end of function libraries available on the web to help you out.

I have a number of little software projects on the go, you can see them in much the same way as woodworking projects, suduko or crosswords: they keep me out the way, muttering quietly to myself and exercising my brain. It doesn't matter that what I'm doing isn't groundbreaking and new.

Programming does lead to some odd habits; when I started programming it was useful to know binary and hexadecimal number systems, as a consequence I believe that numbers such as 1024 and 128 are nice and round. I've come to appreciate a wide range of bracket styles [] (){} since they are all used for different things and the semi-colon is one of the most important pieces of punctuation in my life. If I program for too long in a stretch I start to forget how to speak to people.

And just to show off the results of my latest fiddlings: maps of the UK election results. I got interested in doing this just after the General Election. The Guardian has published a lovely spreadsheet of election results, including data on every single candidate. You see lots of maps of data of this sort, I wanted to know how it was done. (Technical details beyond the maps.)

First of all the gender of MP's by constituency: constituencies represented by ladies are marked pink, those by men marked blue:

The black constituency in northern England is Thirsk and Malton, which held its election on 27th May, following the death of one of the candidates during the general election campaign.

The population of each constituency is also interesting, here I have coloured the constituencies with 9 different shades of green, the palest shade corresponds to a voting population of between 20,000 and 30,000, the darkest shade corresponds to a population of between 100,000 and 110,000:

The Western Isles (now known by it's Gaelic name: Na h-Eileanan an Iar) has the smallest population at about 22,000 and the Isle of Wight has the largest population with just under 110,000 potential voters. I used ColorBrewer to find a nice set of colours.

Finally here's a map of which party came second in each constituency in the 2010 General Election:

Red for Labour, blue for the Conservatives, orange for Liberal Democrats, yellow for Scottish Nationalists, pale green for Plaid Cymru, dark green for Sinn Fein, blue for Ulster Conservatives and Unionists, and there are a few independents and minor Northern Island parties which are all coloured white. 

Footnotes

So the task is to get the spreadsheet data into a map: To get started I did a bit of memory trawling and googling, a couple of people have written about colouring in maps: this one uses shapefile format map data and the R programming language, whilst this one uses SVG format map data and Python (another programming language). It turns out the shapefile format data for constituencies is a little difficult to get - you have to fill in forms! However enterprising people on Wikipedia have made SVG format constituency maps available. SVG stands for Scaleable Vector Graphics, it's an XML format which means it's plaintext and there are standard means to extract data from it and manipulate it. The only real problem is that the constituency names in the spreadsheet don't exactly match the names inside the SVG format map - I had to resort to some horrible constituency by constituency coding for a load of them. To do this I used the C# programming language, largely because Visual Studio Express C# is a very nice, free development environment which I've used before. To view the SVG maps inside my application I used the Webkit .NET library to provide a webbrowser control (which wraps up the rendering engine used in the Safari and Google Chrome browsers) - the native C# webbrowser control is based on Internet Explorer - which doesn't render SVG. Output to bitmaps is a bit clumsy, Inkscape (a free SVG editor) wasn't keen on displaying the original constituency map, so I resorted to viewing the map in Google Chrome and taking a screen shot (a terrible bodge).

Book review: The World of Gerard Mercator

Once again I have been reading, this time "The World of Gerard Mercator" by Andrew Taylor. As before this blog post could be viewed as a review or, alternatively, as some notes to remind of what I have read. Overall I enjoyed the book, it provides the right amount of background information and doesn't bang on interminably about minutiae. I would have liked to have seen some better illustrations, but I suspect good illustrations of maps of this period are hard to come by and a full description of Mercator's projection was probably not appropriate.

The book starts off with some scene setting: at the beginning of the 16th century the Catholic church were still keen on Ptolemy's interpretation of world geography in fact to defy this interpretation was a heresy and could be severely punished. Ptolemy had put down his thoughts in Geographia produced around 150AD, which combined a discussion of the methods of cartography with a map of the known world. As a precedent Ptolemy's work was excellent, however by the time of the 16th century it was beginning to show it's antiquity. Geographical data, in Ptolemy's time, from beyond the Roman Empire was a little fanciful, and since the known world was a relatively small fraction of the surface of the globe the problems associated with showing the surface of a 3D object on a 2D map were not pressing. Ptolemy was well aware of the spherical nature of the world, Eratothenes had calculated the size of the earth in around 240BC, he stated that a globe would be the best way of displaying a map of the world. However, a globe large enough to display the whole world at sufficient detail would have to be very large, and thus difficult to construct and transport.

Truly global expeditions were starting to occur in the years before Mercator's birth: Columbus had "discovered"  the West Indies in 1492, John Cabot made landfall on the North American landmass in 1497. Bartolomeu Dias had sailed around the Southern tip of Africa in 1488, Vasco da Gama had continued on to India in 1497, around the Cape of Good Hope. The state of the art in geography could be found in Waldseemüller's map of 1507, showing a recognisable view of most of our world. Magellan's expedition would make the first circumnavigation of the globe in the early years of Mercator's life (1519-1522).

Mercator was born in Rupelmonde in Flanders on 5 March 1512, he died 2 December 1594 in Duisburg in what is now Germany at the age of 82. This was a pretty turbulent time in the Netherlands, the country was ruled by Charles V (of Spain) and there appears to have been significant repression of the somewhat rebellious and potentially Protestant population. Mercator was imprisoned for heresy in Rupelmonde in February 1543, remaining in custody until September, many in similar circumstances were executed, however Mercator seems to have avoided this by a combination of moderately powerful friends and a lack of any evidence of heresy.

Mercator's skill was in the collation and interpretation of geographical data from a wide range of sources including his own surveys. In addition he was clearly a very skilled craftsman in the preparation of copperplate engravings. He was commercially successful, manufacturing his globe throughout his life, as well as many maps and scientific instruments for cartographers. He also had a clear insight into the power of patronage.

His early work was in the preparation of maps of the Holy Land (in 1537) and Europe (in 1554), along with a globe produced in 1541. The globe seems to be popular amongst reproducers of antiquities, you can see details of it on the Harvard Map Collection Website.

Mercator is best known for his "projection", in this context a projection is a way of converting the world - which is found on the surface of a 3D sphere into a flat, 2D map. Mercator introduced his eponymous projection for his 1569 map of the world, illustrated at the top of this post. The particular feature of this projection is that if you follow a fixed compass bearing you find yourself following a straight line on the Mercator projected map. This is good news for navigators! The price you pay for this property is that, although all regions are in the correct places relative to each other, their areas are distorted so those regions near the poles appear much larger than those near the equator. Mercator seems to have made little of this discovery, nor described the method by which the projection is constructed - this was done some time later, in 1599, by Edward Wright. Prior to maps following Mercator's projection navigation was a bit hit and miss, basically you headed up to a convenient latitude and then followed it back to your destination - an inefficient way to plan your course. If you're interested in the maths behind the projection see here.

In terms of it's content the 1569 map shows Europe, Africa and a large fraction of Asia much as we would see it today, certainly in terms of outline. The Eastern coast of North and South America is fairly recognisable. The map fails in it's representation of the West coast of America - although to give credit where it is due, it at least has a west coast. The landmasses indicated at the northern and southern poles are close to pure fantasy. The Southern continent had been proposed by Ptolemy as a counterbalance to the known Northern continents - with no supporting evidence. Exploration of the far North was starting to occur during Mercator's life, with expedition such as that of Frobisher.

Mercator is also responsible for the word "atlas" to describe a book containing a set of maps, in this instance he coined the term to describe the volumes of maps he was preparing towards the end of his life, the last of which was published published posthumously by his son, Rumold, in 1595.

Following my efforts on Joseph Banks, I thought I'd make a map of significant locations in Mercator's life. You can find them here in Google Maps, zoom out and you will see the world in Mercator projection - a legacy from a man that lived nearly 500 years ago.

Understanding mayonnaise

Some time ago I wrote a post on confocal microscopy - a way of probing 3D structure at high spatial resolution. This post is about using confocal microscope to understand mayonnaise (and a bunch of other things)

As young scientists we are introduced to the ideas of solids, liquids and gases very early on. We make these distinctions, amongst other things, to understand their mechanical properties, to answer questions such as: How thick do I have to make the legs of my chair to support my weight? How fast will liquid run out of a bucket? How high will my balloon fly?

But what is mayonnaise? It's very soft, and can be made to flow but it's not a proper liquid - you can make a pile of mayonnaise. How do we describe grain in a silo, or an avalanche? In some senses they have properties similar to a liquid: they flow - yet they form heaps which is something a solid does. What about foams -  a pile of shaving foam looks pretty similar to mayonnaise? Starch paste is an even weirder example, it acts like a liquid if you treat it gently but a solid if you try anything quick. (This is known as shear thickening). These mixed systems are known as colloids.

The programme for understanding solids, liquids, gases and these odd systems is to understand the interactions between the "fundamental" particles in the system. For our early courses in solids, liquids, and gases this means understanding what the atoms (or molecules) are doing - how many of them are there in a unit volume, how are they ordered, how do they move and how they interact. Typically there are many "fundamental" particles in whatever you're looking at so rather than trying to work out in detail what all of them are up to you resort to "statistical mechanics": finding the right statistical properties of your collection of particles to inform you of their large scale behaviour.

The distinguishing feature of all of our new systems (mayonnaise, grain piles, avalanches, foams, starch paste) is that they are made from lumps of one phase (gas, liquid, solid) in another. Avalanches and grain piles are solid particles in a gas; mayonnaise is an emulsion: liquid droplets (oil) inside another liquid (water); foams are air inside a liquid and starch paste is a solid inside a liquid. These systems are more difficult to analyse than our traditional gases, solids and liquids: firstly their component parts aren't all simple and aren't all the same. Particles most likely have different sizes and shapes. Atoms and molecules are all the same size and all the same shape. Secondly, they're athermal - ambient temperatures don't jiggle all their bits around to make nice averages.

Confocal microscopy looked like an interesting way to answer some of these important questions about the structures to be found in these complex systems. Mayonnaise turns out not to be a good model system to work with - you can't see through it. However, you can make an emulsion of different combinations of oil and water, and if you're cunning you can make an emulsion with over 50% of droplets by volume which is still transparent. Using even more cunning you can make the distribution of droplet sizes relatively small.

Having spent a fair bit of time getting the emulsions transparent with reasonable droplet size distributions, my student, Jasna, came in with some pictures of an emulsion from the confocal microscope: where the oil droplets touched each other the image was brighter, you can see this in the image at the top of this post. This was rather unexpected, and useful. The thing about squishy balls, is that the amount by which they are squished tells you something about how hard they are being squeezed. The size of the little patches tells you how much force each droplet is feeling. So all we have to do to find the force network in an emulsions is measure the size of the bright patches between them.

In the end our work measured the forces between droplets in a compressed emulsion and we found that these measurements agreed with a theory and some computer simulations. Criticisms of the work were that the relationship between luminous patch size and force was more complicated than we had assumed, and that the force distribution was all very well but the interesting thing was the arrangement of those forces. These criticisms are fair enough. Must have been pretty good though, because someone wrote a paper for Science claiming to have done it first, whilst citing our paper (they had to publish a correction)!

Footnotes
This work can be found in this paper:

Brujic, J., S. F. Edwards, D. V. Grinev, I. Hopkinson, D. Brujic, and H. A. Makse. “3D bulk measurements of the force distribution in a compressed emulsion system.” Faraday  Discussions 123, (2003), 207-220.  (pdf file on Scribd)

Jasna Brujic was the PhD student who did the experimental work, Sir Sam Edwards is a theoretician who works on granular materials, Dmitri Grinev worked with Sir Sam on the theory, I supervised Jasna, Djordje Brujic is Jasna's dad and wrote the image analysis code and Hernan Makse is a computer simulator of granular materials.

Go back to your constituencies and prepare for government!

I think this will be my final political blog post for a while. I've written quite a few over the past month: I started with an explanation as to why I was a member of the Liberal Democrats here. I hunted out some data on the occupations of MP's and made a graph here (what a lot barristers!). Then I wrote a blog post about the statistical errors in opinion polls, which I think was a little prescient (latest reports suggest that the voters that didn't turn out were predominantly LibDem - hence biasing the pre-election polls). Approaching the finishing line, I wrote a post on my plans to stay up late on election night, followed by a post on my crashing gloom of the night itself. Then I made a post on what it might mean as the negotiations for coalition continued.

This post is about coalition, and a look at the debris of the election. My friends at work from Labour and Conservative tendencies called me a "turncoat weasel" today (in a friendly manner) so I have illustrated this post with a picture I took of a weasel.

First up, my take on the agreement between the Liberal Democrat and Conservative Party: remind me not to negotiate with Nick Clegg for anything, I think he's played a blinder on this one. The agreement contains a mix of LibDem and Tory policies, and five positions in cabinet. It's quite clear that the LibDems are a junior partner, but then that's what you'd expect. I'm also pleased Nick Clegg did what he said he would do before the election: go to the party with the largest electoral mandate in the first instance.

Despite the massive apparent differences on Europe, I suspect it won't be a big problem for the coalition. Liberal Democrats have a principled view that power should dissociate to the appropriate level - some things are best handled at a local level, some at a national level, and some to the European level. My guess is that over the next five years there will not be another major treaty moving power towards Europe, and joining the Euro in the next five years, as we all crawl out of recession, would be unwise as far as anyone is concerned. Perhaps David Cameron will value a coalition partner who has not isolated itself in Europe.

I'm wondering whether the Labour Party are trying to take the mantle of "the nasty party", in the background to David Cameron's speech in Downing Street could be heard a chant of "Tory scum". David Blunkett described Nick Clegg as a harlot, amusing coming from an adulterer. And this morning Nick Cohen, left-wing columnist for the Observer and various other places, tweeted:

Here's my writing sorted out, Will crucify every fucker who voted Lib Dem, one by one, in the national press. Better you had voted Tory.

I suppose I should proviso this last one, on the whole I don't believe a single tweet means national news, perhaps Cohen was joking, perhaps he was tired and emotional. But it picks up the tenor of quite a few of the tweets I've seen from Labour supporters today.

Maybe this is why I'm a natural Liberal Democrat, I feel nothing like a visceral hatred for either of the two other main parties. I believe that people are largely in politics for the right reasons, I may think they are wrong but I don't question their faith in what they believe is for the best. Getting a glimpse of how elections work via twitter, I see just how much work is involved in getting elected as an MP (at least in a contestable seat).

It's tickled my fancy that much of the discussion of the last few days has centred around the percentage share of the vote that a different alliances would get, that's lovely and I agree entirely with that line of argument but your past record: opposing proportional representation, tells me that you're a hypocrite if you raise it just now.

I'm still glad I voted Liberal Democrat, I didn't vote Tory because of the social conservatives, Margaret Thatchers "no such thing as society", Michael Howard's blind faith in market solutions, isolationism in Europe, Peter Lilley and his little list, John Major's denial of support to the Iraq rebels after the first Gulf War and the moves of senior Tories into directorships of companies benefiting from their privatisations. The Tories claim to have changed, perhaps they have, perhaps the LibDems will reinforce that strand.

I hope the move for fair votes takes off, in a sense this is one of the key things I'm after long term in politics. Democracy is important, when I go to the ballot box I want my vote to count just as much as yours. Sorting out the deficit is ephemeral, fair votes will be a lasting legacy.

Perhaps the coalition will fall apart before the end of its term, perhaps the LibDems will suffer for this coalition in the next election (there's some suggestion that this happens to smaller coalition parties). But for the first time in 70 years people who call themselves "liberal" are in government.