HTV has published an opinion poll on the forthcoming assembly elections. I shan't comment on its contents because the only thing one can say about Welsh opinion polls is that they are consistently wrong.
For example HTV commissioned a poll in February 1999, just before the May Assembly elections of that year. The poll predicted:
Lab 58%
PC 12%
Con 20%
LD 9%
The election result was
Labour 36.5% (wrong by -21.5%)
PC 29.5% (wrong by + 17.5%)
Con 16.1% (wrong by - 3.9%)
LD 13% (wrong by + 4%)
A much more inclusive poll was commissioned before the 2003 election. Because it was more inclusive it was more expensive and the cost of commissioning it was shared between HTV the BBC and S4C. Being a more inclusive poll it was better than the 1999 effort by HTV alone, but it still failed the + or - 3% test of a scientific poll.
Poll result:
Lab 39%
PC 31%
Con 15%
LD 15%
Election result
Lab 37% (wrong by - 2%)
Plaid 20% (wrong by -11%)
Con 19% (wrong by + 4%)
LD 11% (wrong by - 3%)
The current offering from HTV is similar to the 1999 poll, so one can expect it to be as inaccurate as that poll was.
UK wide opinion polls are fairly accurate; they are conducted according to a scientific discipline called psephology. Psephologists look at the make up of an electoral community and poll people by selecting respondents who reflect that balance. They try to create a gender balance, age balance, education balance, social class balance, earnings balance etc. that is the same as the balance of the electorate.
Because of a lack of regular opinion polling in Wales there is no such thing as Welsh psephology. Wales is not a microcosm of the UK it is a completely different place, one needs different polling methods in Wales than those used in the UK to get an accurate picture of voting intentions.
In comparing the two opinion polls the most outstanding factor is how wrong the Plaid result is in both. Grossly underestimated in 1999, badly over estimated in 2003. This is because UK based psephology just can't account for the Plaid vote (even if Plaid won every single vote in Wales it would be a minuscule 4% of the UK wide vote). The way UK psephology deals with Plaid is to ignore the party. If Plaid has 4 seats after a general election - UK polls will show it having the same four seats through out the period leading up to the next election - no matter what the political climate in Wales. Which is fine on an UK basis, a few Plaid gains or loses are unlikely to impact much on the overall result of a Westminster election.
Because UK psephological standards are used to conduct Welsh polls, Plaid always stands still in Welsh polls too!!!
HTV's poll today, like the 1999 poll (where Plaid's result reflects its 1997 UK election vote) and the 2003 poll (where Plaid's vote is similar to its 1999 Assembly vote) puts Plaid on no change by % or seats won - which doesn't make sense. If Labour loses 4% of its vote (as the poll suggests) and Plaid stands still Plaid must gain at least 2 more seats.
For Wales see England doesn't work in psephology any more than it works in any other discipline. An opinion poll based on a for Wales see England attitude to polling is a waste of the poll commissioner's money.
My advice to all parties in response to HTV's poll is - ignore it, it's bollocks!
Other Bloggs on HTV's poll include:
Glyn Davies Predictable Poll
Blamerbell Briefs: Poll says Labour set for assembly election losses
Chanticleer: The first poll
e-clectig: Pol Piniwn
Ordovicius: Fudged Poll to Frighten Faithful?
Iain Dale's Diary: Tories Set to Become Largest Opposition Party in Wales
Live from the Socialist Fortress: Straw Poll: hands up if you learnt something new today
Peter Black AM: Ebbs and tides
Alun Cairns
Diolch, Alwyn. This has to be the most informative post on the poll that I've read.
ReplyDeleteAgreed. The failure of the Welsh media to provide regular polling points to a real democratic deficit. Having a one-off poll is not enough, especially as there are doubts about the scientific robustness of the sampling.
ReplyDeleteIt'd almost be better to do without and rely on election night...
Labour's share in 2003 wasn't 30%, it was 40%.
ReplyDeleteThe 1999 poll didn't put PC's ahare at it's 1997 level, it put it 20% higher (12% compared to 9.9%)
If Labour's share falls by 4% and PC's stands still, PC do not necessarily gain two seats. It depends on distribution.
Typical Labor tactics Normal Mouth - if the sums don't suit change the sums. The difference between predicted and actual levels in my post is percentage points not a percentage of a percentage.
ReplyDeleteThe difference between Plaid's 10% actual vote in 1997 and its 12% poll prediction in 1999 is flat-lining within the + /- 3% rules of British psephology.
Of course if you really want to change the sums - NOP underestimated Plaids result by 150% in 1999. If the latest offering by NOP is as inefficient that puts Plaid on 50% !!!
I've changed nothing, Mr Old Fart. You claimed the 1999 poll put PC at the same level as its '97 share. It didn't. It put them 2% higher. That this difference was within the margin of error doesn't change the fact that it was still 2% higher.
ReplyDeleteThat's not me changing the sums. That's you getting your facts wrong. Like you got Labour's 2003 share wrong. By 10%.
The HTV poll shows % voting for seats and regions as virtually identical because people are not generally aware of the effect one has on the other. The only decent poll would be to ask about the 40 seats, calculate the results, then go back to the same people and explain what would have happened in the 40 seats before asking them how they would then vote for the 20 regionals. It's not going to happen so while we have the current system any poll is at best a wild stab in the dark.
ReplyDeleteSome good points but I'm not sure where you got your figures from.
ReplyDeleteYou've got the 1999 results about right, but the 2003 election results (regional ballot) were:
Labour 36.6%
Plaid Cymru 19.7%
Conservatives 19.2%
Liberal Democrat 12.7%
The HTV/NOP poll from April 2003 estimated:
Labour 39%
Plaid Cymru 31%
Conservatives 11%
Liberal Democrats 15%
So the poll seriously underestimated the Conservative vote as well as overestimating the Plaid vote.
You're right to say that opinion polls have a poor record at forecasting the level of Plaid's support. The NOP poll in 1999 was no worse than any of the other opinion polls published in the run-up to (or the period after) the 1999 election. The problem seems to have been that the polls didn't identify the extent to which Plaid supporters were much more likely to vote in the first Assembly elections or the number of voters who would vote Plaid in those elections despite not normally voting Plaid. The reverse of that was that Conservatives were much less likely to turn out to vote in the Assembly election just 18 months after losing the 1997 referendum. Neither of these factors seemed to apply to the same degree in 2003 and I wouldn't expect them to feature at all this year.
This month's ITV Wales poll is based on a much larger sample than the 2003 opinion polls. Of course that doesn't necessarily mean that it will be any better at forecasting the result of the election but it is more likely to represent the level of support in Wales at the moment.
Most people agree that this election will be decided by differential turn-out as much as changes in support. That's why NOP decided to base their headline figure on those who are certain to vote. MORI do the same for general election polls. Other companies weight support by how likely people say they are to vote.
NOP should release the data table for this poll so that people can get a better idea of how reliable it is. ITV Wales will be releasing another poll a week before the election. It's quite possible that it will be very different, but that doesn't mean that either was unfairly skewed against any one party.
For what it's worth, my guess would be that Labour will poll in the mid 30s and both Plaid and the Conservatives will be very close together on 20%+ (ie much closer to the ITV Wales poll than to Plaid's much advertised private polling by Beaufort). Either party could take second place.
The figures came from the BBC the 30% was a typo so I have changed that. I assume that the BBC rounded up the results to the nearest whole numbers, which is accurate enough to show how wrong the polls were.
ReplyDeleteI don't think that there is any intentional party political bias in any of the polls, its just that we are without a tradition of polling in Wales, similar to the UK wide tradition or the Scottish tradition. Without such a tradition there is no way of knowing how to interpret results.
I understand, for example that the raw data for UK wide polls always underestimates the Tory vote by about 3% so an adjustment to account for this discrepancy is worked into the published results. Is the same true of the Tory vote in Wales? We just don't know so we don't know if a similar adjustment is needed or not.
As for Plaid's Beufort research, Beaufort isn't even a "proper" opinion polling company. And it must be remembered that Plaid used Beaufort in 2005 but failed to realise that the were about to lose Ceredigion and fail to recapture Ynys Mon - another lesson in not putting too much faith in Welsh opinion polls.