X Factor 2012 Review Part 2: The Running Order

The voting system was tweaked on the X Factor in 2012: for the first time, lines were open from the beginning of the show rather than only after all the acts had performed. After the first two live shows saw Carolynne Poole eliminated from singing 12th of 13 and Melanie Masson from singing 12th of 12, the Sofabet community scrambled to come to terms with the extent to which our previous working assumptions about the effect of the running order on the public vote had been upended.

Traditionally, it always was considered an advantage to sing late in the early weeks. Singoff appearances from this late in the show, this early in the series were entirely unprecedented. The debate raged: was late the new early? And, if so, was early the new late? After week 4, we wrote an article summarising the consensus that had been developing the the Sofabet comments section about the effect of the running order under the new voting system.

Now the percentages are in, it all looks rather less dramatic than we’d guessed at the time.

When we performed similar analyses of the running order after the 2011 series and the 2010 series, we argued that the most meaningful way to get a sense of its effect was to look act-by-act. Our reasoning goes like this: there isn’t too much we can conclude from single data points such as, for example, Carolynne Poole hitting the singoff from 12th of 13. Was the problem singing 12th of 13, or being Carolynne Poole? Short of re-running the show in a parallel universe with the running order shuffled, we can’t be sure.

We can, of course, draw some general conclusions simply by looking at position in the show and performance in the vote. This is how the consensus had developed, over the years, that it’s better to sing at the end of the show than near the start. But we can’t push this line of analysis too far, for fear of conflating cause and effect. It may well help to sing last; but equally, programme-makers want to keep people watching to the end of the show, and have an interest in putting popular acts on last.

So we can observe that in all but two of the live shows in 2012 (the exceptions being Melanie Masson in week 2 and Ella Henderson in week 3), the pimp slot – that is, the last act to sing – was shared between just three acts: the eventual finalists, James Arthur (weeks 4, 6, 9, 10), Jahmene Douglas (weeks 1, 7) and Christopher Maloney (weeks 5, 8). Did they get to the final because they monopolised the pimp slot, or did they monopolise the pimp slot because producers decided they were the acts people were tuning in to see, a variable which also explains why they got to the final? We of course can’t tell.

These difficulties explain why we reckon it makes most sense to try to tease out the effect of the running order on the vote by looking act-by-act at their trajectory from week to week. So, let’s start with James’s:

(A quick explanation of how to read these graphs: the line shows the performance in the vote, with 100 being the average given the number of acts performing that week. Where the line goes orange, that shows a sympathy bounce – an act coming off a singoff survival. The grey bar shows how far through the show they sang: no grey bar at all means they opened the show, a bar that goes right to the top of the graph means they closed the show).

Clearly, James’s trajectory is defined by the extraordinary bounce that followed his survival in the week 7 singoff against Ella Henderson. But before that – well, it doesn’t look like much changed with the new voting system, does it? His pimp slots in weeks 4 and 6 coincided with peaks in his voting performance, just as we would have expected from the old voting system.

Here’s second-placed Jahmene:

Jahmene did noticeably better singing last in week 1 than singing first in week 2. Producers, seeing this, must have been a lot more sanguine than we all were after week 2 about the effects of the voting change. Jahmene’s week 3 peak, by the way, is likely to have much to do with the extraordinarily powerful VT focusing on his survival of domestic abuse. You can argue that week 7’s pimp slot didn’t boost him much, but did at least arrest the decline he’d been on since week 3.

Christopher’s chart reads like a slow puncture:

It probably doesn’t tell us a great deal about the effect of the running order. Performing last in week 5 was the steadiest his vote held, but the pimp slot in week 8 didn’t help him; having said that, though, his performance against the mean in week 8 was probably depressed primarily by the large number of James fans emerging from the woodwork.

Union J:

This is much more about the sympathy bounce than the running order. You could argue that singing 6th of 7 in week 6 helped to lessen the comedown from their first bounce, though not by enough to prevent their second singoff appearance. But, really, there doesn’t appear to be a great deal for students of the running order to take from this trajectory.

Fifth-placed Rylan Clark:

Again, not much to learn about the running order here. This trajectory is all about a gradual comedown from the sympathy bounce after week 1’s singoff, then another sympathy bounce after week 5’s singoff, with the week 7 banter with Gary likely delaying his comedown till week 8.

Ella Henderson:

The week 3 pimp slot didn’t give her a spike, though you could argue it temporarily arrested a declining trend. And it’s clear that she did better from 11th of 13 in week 1 than from 4th of 12 in week 2, as you would have expected from the previous voting system.

Seventh-placed District 3:

Their vote performance tracks their running order position fairly closely, boosted by the sympathy bounce after their week 2 singoff survival and harsh judges’ comments that motivated supporters in week 5. Nothing here to suggest the running order worked very differently in 2012 than in previous years.

Here’s eighth-placed Kye Sones:

On the face of it, you could claim that singing first in week 4 helped him. But surely that has more to do with the sympathy bounce after his week 3 singoff survival, given that we don’t see a similar spike for other acts who opened the show (District 3 in week 1, Jahmene in week 2, Christopher in week 3, Rylan in week 5, etc).

Ninth-placed Jade:

On the evidence of previous seasons we would ostensibly have found it surprising that she did proportionally no better from 8th of 9 in week 4 as compared to from 4th of 11 in week 3 – but then, by common consent producers’ knives were out for her in week 4. So, not much evidence here that the running order is operating much differently than in previous years.

Lucy Spraggan:

At last, some evidence for an inverted running order effect, with Lucy’s three progressively later slots in the show being reflected in progressively lower vote totals.

But here’s an immediate counterpoint in MK1:

Melanie Masson’s departure from the pimp slot in week 2 sent shockwaves through the X Factor punting community, in part because a singoff appearance from the pimp slot this early in the series was unprecedented. But Melanie’s chart shows that she actually did better from 12th of 12 in week 2 than from 3rd of 13 in week 1:

Our conclusion, then, is that the opening of phone lines from the start of the show had much less effect on the impact of the running order than we had all surmised in the early weeks.

Clearly, something important has changed – the departures of Carolynne Poole and Melanie Masson show us that singing last or second last is no longer the near-guarantee of survival in the early weeks that once it was. But perhaps we need put more weight on the idea that these were, in the end, two unusually weak contenders, scuppered by a questionable song choice and a 10.35pm finish respectively. Because, taken as a whole, it doesn’t look to us like the relationship between running order and vote in 2012 differed tremendously from previous series.

Or are we missing something in the data? As ever, do let us know below.