This is the fifth article in our review of the 2011 series. You can see our favourite ten posts of the 2011 series here.
We punters, of course, don’t get to see how the X Factor votes are stacking up until the show releases them after the final. We have to look for clues, and the YouGov poll is the gold standard.
When we got the 2010 results, for example, we saw that YouGov’s mid-October poll had captured the early sentiment accurately, showing Matt Cardle carving out a lead over Mary Byrne. The final poll rightly had Matt comfortably beating Rebecca, with One Direction and Cher a distant third and fourth.
In 2011, we saw when the results came out that the late October poll was right to show Janet Devlin ahead of the pack, and the mid-November poll (conducted after Amelia’s week 6 return had deposed Janet from the top spot for the first time, and before Little Mix’s week 7 breakout) was correct to indicate an extremely tight contest.
The 2011 final poll showed a much closer contest than in 2010, with Marcus on 39% once “don’t know” and “will decide on the night” voters had been proportionally reallocated, Little Mix close behind on 35%, and Amelia off the pace with 26%. The result, at the first voting freeze, showed Amelia’s vote was spot on at 26%, with Marcus (35%) and Little Mix’s (39%) shares reversed.
YouGov’s Director of Political and Social Research, Joe Twyman, kindly agreed to talk to Sofabet about the firm’s X Factor polling in 2011 and what punters can expect in 2012.
Sofabet: Joe, in your article on the YouGov website explaining the findings of the final 2011 poll, you said “If I was absolutely forced to pick a winner I would go for Marcus, but it really is too close to call”. You said that performance on the night – which you very wisely stressed meant not just singing, but how the acts were packaged and presented – would be decisive. What was your reaction to the result?
Joe Twyman: This was the first year when the polling before the final was so close, we weren’t confident enough to make a prediction and instead we said it was too close to call. You have to remember that the X Factor is trickier to predict from polling than political elections, in the sense that on a political election day itself nothing typically happens to change voters’ minds. In contrast, what happens during the X Factor final programme itself can obviously have a big effect on the vote.
For example, when it was first announced that the acts would be duetting with mentors, I expected it to be a big plus for Marcus – and I still say it would have been, had they done a Take That song. As it was, after watching the Saturday show my gut instinct was the same as yours, that Little Mix had pulled ahead. I would say it was, indeed, performance on the night that decided it.
Sofabet: There was some discussion in our comments box before the final about whether the polling might underestimate support for Little Mix because of multiple voting. That is, the suspicion is that acts like Little Mix, One Direction and Cher seem more likely to have obsessive young teenybopper fans who might multiple-vote, whereas the likes of Marcus would appeal to the more mature housewife type of voter, who might be more likely to vote once and consider their democratic duty to be done. Does the YouGov polling account for multiple votes?
Joe Twyman: Yes, it does. We handle it in a way that builds on the question about likelihood of turnout that we ask in political polls. Obviously this is a key piece of information when you’re trying to predict an election result. Any political pollster will ask something like, “on a scale of one to ten, how likely are you to vote?” And the higher the number you tell us in response to that question, the more weight we will then give to your response to the question about who you’re going to vote for.
With the X Factor, we extend this idea. If you say you’re likely to vote, then we ask how many times you’re likely to vote. In this way we can give more weight to the responses of the people who say they’ll multiple vote, and build it into our figures.
Sofabet: Is it possible that multiple voting might be something people are too embarrassed to admit to?
Joe Twyman: I’m confident that’s not the case. We craft all of our polling questions extremely carefully so that people aren’t going to be ashamed to give a true response.
What we have observed over the years, however, is that people kid themselves about their own behaviour. They may genuinely believe that they’re going to vote only once, but then something happens on the night – they feel moved by Jesy crying, say – and they vote multiple times.
Sofabet: Do you think multiple voting might be a significant boost for some acts?
Joe Twyman: All our evidence in previous years was that it hasn’t been a statistically significant factor. The real obsessives among the “teenyboppers” might want to vote a hundred times but they just don’t have the disposable income to do it. So in 2010, for example, when analysis of social media was showing One Direction way ahead, we were seeing that it didn’t translate into votes for them.
Interestingly, though, this year we did see a greater difference than in previous years in respondents who said they intended to multiple vote. There is a suspicion that this year some voters as well as viewers might have dropped out, and that broadly speaking the “teenybopper enthusiast vote” held up better than the “housewife vote”. As a result, we may have seen the balance of power shifting.
Which, incidentally, if you’re Simon Cowell, you might not feel too unhappy about. The profile of the X Factor voting public has generally differed a lot from the profile of the record buying public, which is why acts like Matt Cardle and Joe McElderry can stroll to victory and then not have stellar recording careers. It may be that the profiles are converging a bit more.
Sofabet: On this subject of housewives and teenyboppers, can you tell us more about how you do the demographic weightings in your polls? For example, when we compare the unweighted figures of respondents per age group with the weighted figures, we’re a bit surprised that more weight isn’t given to the younger age brackets – we would have guessed that voters might skew younger than the weighing suggests. How do you make your judgement calls on what proportion of voters are likely to be accounted for by each age group?
Joe Twyman: The polls that get published aren’t the only ones we do on X Factor. We poll throughout the X Factor period, usually from the week before it starts all the way through to the final, as part of wider surveys. This daily polling means that we can establish the profile of people who are watching and people who are voting. From this we can decide what weight to give the responses from different demographics.
We did take account in our 2011 polling of the seeming shift in profile of voters, as we became aware of it. There is a limit in how far we can account for it, in that we know we are short of particularly very young people among our polling respondents, and we obviously don’t want to unbalance the results by weighting too aggressively towards the relatively small number of people we have in that age group.
We’ll look some more at whether anything needs to be tweaked further in terms of the weightings for 2012. Having said that, in previous years we were very confident with the weighting and profile we applied to younger people, and we’re pretty sure that generally speaking we’re fit for purpose. Any changes we make to the model for 2012 will be an evolution, not a revolution.
Sofabet: We are nervous about what 2012 holds, as we felt the show ventured dangerously close to shark-jumping this year with the twist of Amelia’s return. This sort of unpredictability makes life more complicated for punters, and it must make it more complicated for pollsters, too?
Joe Twyman: We noticed this year that the narratives and sob stories and manipulation seemed to be more influential than ever. I would bet that if Janet had sung a slow song and burst into tears in the week her grandfather died, she would have become as uncatchable as Matt Cardle became in 2010 – she was miles ahead when we first started polling.
The problem for us is that the more the voting rests on sob stories and crying and manipulation, the harder it is to make any predictions because the vote can be more influenced by what happens on the night. You have to assume that ITV and Simon Cowell won’t be happy about the viewing figures this year, so we may see a Big Brother style evolution with even more extreme events to try to hook the viewers’ attention. If it goes down that route, that won’t make our lives as pollsters any easier.
Leave a Reply