The Scientist’s Toolkit: Know your trend

10 02 2010

“Let me introduce you to a radical and highly complex, story-wrecking mathematical insight. Ready? Numbers go up and down.”

Another very educational piece about why stats can go wonky, from the BBC’s Go Figure series. Michael Blastland looks at the fluctuations of teen pregnancy on the Scottish island of Orkney, which, like the Hawthorne effect, shows some of the dangers of making a story out of what we know.

Looking at the annual figures for teenage pregnancy in Orkney, we can see one of the problems with our tendency to make stories from our data: The long-term annual view show the data fluctuating constantly, but there’s not much of an overall trend one way or the other. But if you only look at the figures from, say, 2002 onwards, then you see a peak followed by a clear decline. Obviously, this is due to the heroic actions of health workers on Orkney, taking action to halve the teen pregnancy rate overall between 1994 and 2006.

All this is great, of course, until you review the figures again at the end of 2007, and discover that they’ve cycled right back to their 1994 peak. (Incidentally, the first graph Blastland shows is one of the most beautifully misleading pieces I’ve ever seen. An excellent example of how you can torture your data until it confesses to anything).

Here’s the thing: data is always “noisy”. There are hundreds, if not thousands, of factors you simply can’t control or account for at any given time, and they will make the data randomly fluctuate up and down. Teenage pregnancy, for instance, shows a seasonal variation: teenagers are most likely to get pregnant at the end of the school year, probably because they’re having sex more on account of the warm weather and lack of schoolwork. If you only look at a short period of time, it’s easy to be convinced that the data show an overall upward or downward trend… but you’ve really got to take the long view to make sure that this isn’t simply random variation, or “noise”. The more data you have, the less vulnerable your data are to random fluctuations – take a look at the line representing Scotland, for instance, which shows some minor variations but is much more flat overall. (We call this the law of large numbers.)

If you really think your data (teenage pregnancies, sales, salaries) are showing an overall trend… make sure you’re taking a long view. Are there seasonal fluctuations you haven’t taken into account? Anomalous weather? What was happening in the economy at the time – are you comparing it to the right things? These things matter.

Is it worth it (financially) to go to uni?

6 02 2010

As someone who employs new graduates, I wonder about this question a lot. It’s a popular one for blog posts and media stories; Penelope Trunk is firmly on the “con” side of education for education’s sake. The BBC offers some rather more unconventional reasons (you might meet a spouse with good middle-class earning potential!), but, with costs and graduate unemployment going up both in the US and UK, and places available likely to face a crunch, I think it’s worth considering soberly just what you’re likely to get out of it.

I was prompted to make this post by this Times Online article, which argues that so-called “Mickey Mouse degrees” like golf course management and brewing are in fact a smart bet, leading to profitable careers because they directly prepare you for a particular sector. The article also trots out the “a degree is worth X thousand pounds over a lifetime” figures. As usual, the top-earning degrees are quite comfortably medicine and law, with engineering and modern languages tailing behind by some way.

But I can’t help wondering whether this actually proves anything, really. The single greatest predictive factor of success – both in university and in a career – is intelligence. Medicine and law courses are highly sought-after. They’re immensely challenging careers, intellectually. Universities have enough competition for places to set the entry bar extremely high, and pick and choose who they accept, which makes the degree, in some respects, just a “surrogate marker” for intelligence. If these courses were effectively open entry, I wonder if the graduate earning potential would be somewhat diluted. It certainly helps, though, that vocational courses directly give you the skills you’ll need on graduation, rather than instilling “soft skills”. While I want to hire, and work with, people who can think intelligently and critically, I’m yet to come across a uni that does much, in its courses, to instil the kinds of soft skills that are crucial when first starting work, and one of my key jobs as a graduate manager is to give them a crash course in these skills, and quickly. The social and interpersonal skills tested by any job are very different from the skills imparted by your average undergraduate course.

I suspect that, on top of the vocational nature of some of these “Mickey Mouse courses” (and I don’t agree with that name) also comes from the fact that the graduates on them are determined and focused enough to dedicate themselves to a course in something very specific. I further suspect that this determination and focus starts before the graduate begins the degree. In the absence of a study that controls for intelligence, it’s hard to tell how much is really added by the degree itself. But my advice for seventeen- and eighteen-year-olds, based on my experience at work, would be this: If you aren’t sure what you want to do after university, and you aren’t passionate about any particular subject, and you aren’t sure you’ll get into a top university – strongly consider working for a year or two, and thinking about it.