Digging deeper into the UCAS applications data – there’s more to it than meets the eye

Following this week’s coverage of university applications, we could all be forgiven for thinking that the government’s proverbial chickens had come home to roost. The 9 per cent drop in applications compared to this time last year was reported as the first real indicator that the new student finance regime was taking its toll.

But as always with statistics, there’s more to the story than that.

So, what does that 9 per cent represent and what does it actually tell us?

Who are these applicants?
They fall into two distinct categories:

1. People who had a real deadline for application i.e. those who applied for Oxford or Cambridge or for medicine, dentistry, veterinary science courses . These courses and universities have a final deadline of 15 October.
2. People who chose to apply early.

It’s also important to underline at this point that the applications cycle is a still only a matter of weeks old. For everyone out there who doesn’t fall into the first category above, you have until 15 January 2012 to get your application in. Context is everything here.

Looking at this graphic, we can see that the applicants who had a real deadline of 15 October make up just seven per cent of this reduction in numbers. The second category of people, the ones who, for whatever reason, were organised enough to apply early in the cycle, make up 93 per cent of the fall.
 
There was plenty of speculation about why this fall in early applicants occurred, with most of the blame laid at the Government’s door. Higher tuition fees were frightening the early birds away. While much of the reporting of the data left a lot to be desired, perhaps there’s some element of truth to this line of thought. Universities are undergoing a big upheaval, so would it be so surprising if prospective students were waiting longer to decide what to do?

Are we able to forecast final acceptance numbers from this data?
We’ve also seen the tendency to use the October application data to predict a general trend: “applications are down now, so of course they’ll be down in the New Year after the final deadline”. But can we really use the data to forecast acceptance numbers (the numbers of people who actually get a place at university)? Remember, at this stage this is just about levels of people who, by applying, are expressing a desire to go to university , not about who is turning up to lectures in autumn 2012.

*All data final end of year except 2011 - end of clearing applicants & acceptances

In 2009, changes in final acceptances did seem to follow changes in the earlier October data. However, between the 2009 and 2010 cycles, October applications were up 12 per cent, while acceptances increased by just 1 per cent. There’s no second dark blue arrow in 2012 because we don’t yet know how many applications there will be in total and how many universities will accept.

So what can we make of that? Well, one thing we can say is that the October data is not necessarily a predictor of the applicants and acceptances overall. And there’s more complexity than you might think looking at the headline figures: for a university to accept a candidate, they need to have a place to give them. There’s also the small issue that these numbers include applicants who have deferred to the following year.

What can we deduce from the October data release?
We’ve established that there’s a lot we can’t predict, mainly because it hasn’t happened yet and the figures we do have haven’t proved themselves to be a reliable indicator of things to come.

However, focusing on courses with time-critical deadlines, the fall was 0.8 per cent. The level of reduction varied by course subject and there was a 3.1 per cent fall in medicine and dentistry applications (choices). These results are concrete and require investigation; has demand fallen from particular groups?

Age profiles of October 2011 applicants can give us some insights. The demography of applicants has changed; the typical applicant this year, compared to October 2010, was an estimated three and a half months younger. The 18 year-old age band has grown as a proportion of all applications by 7.6 per cent, and ‘17 year olds and younger’ by 13.4 per cent. This is an example of how data can throw up leads, asking us to follow the trail. Could this subtle change in age reflect a wider change in applicant profile and could be having a pronounced effect on medicine and dentistry? Or could the reduction be down to something we can’t know yet?

“It’s the context, stupid”
A misquote, but I think Bill would forgive us because it captures the essence of the argument here. A question to always ask yourself when it comes to sensationalist, but fact-filled headlines is – what does this data actually cover? In universities there are many different types of students – part-time and postgraduate, as well as full-time undergraduates who apply through UCAS. Of all students in their first year of a higher education course in 2009-10, only two in five applied through UCAS. The October applicant data last year represented just one in ten of all UCAS applicants for the 2011 cycle. It’s apparent then, that useful as it is, the UCAS data isn’t the whole picture.

Getting this kind of stuff right matters, because selective or incorrect interpretation of data has effects in the real world. Saying it’s simply too early to read too much into these figures may sound like a cop-out but it’s the exact opposite in this case. We need to monitor the impact of these policy changes very carefully and it’s vitally important we do so objectively, with the information we have to hand. And we must be very clear about what conclusions can and can’t be drawn from any given set of data. If new data appear to match your presumptions, alarm bells should ring. The best thing to do then is to stop and think. And then start digging a little deeper.