MODERNISING MAUDE'S STATISTICAL FLAWS
THE KEY OPINION POLLS
The conviction that the Conservative Party was inherently unpopular as a brand and was contaminated had been gaining ground among some Conservatives at least since the leadership of Ian Duncan Smith.
In 2005, two opinion polls appeared influential in fostering this conviction among leading Tories. Francis Maude waved what he called ‘killer polls’ at a Conservative Party Conference on which he had already commented on in August 16th 2005 in an interview with Epolitix.com: ‘It wasn’t that people particularly disliked our policies, lots of research showed that people approved of the policies by and large until you told them they were Conservative Party policies and that made them feel differently about it’. This is clearly a widespread view since Adam Rickitt, not a noted commentator on polls, told the Daily Telegraph on 6th June 2006: ‘I recall with horror the findings of a Populus focus group before the last election where a Conservative policy was presented to people without the Conservative label attached, 82 per cent of respondents agreed with it. When people were told it was a Conservative policy, that support fell to 65 per cent. The problem was the brand’. (Actually Rickitt is referring to an opinion poll conducted by ICM, not a focus group by Populus.)
What actually happened and were Francis Maude’s killer polls actually as murderous as he made out?
As Maude said to ePolitix, “We need to understand what’s happening”.
As the contention that the polls showed that voters approved Conservative policies until they were told they were the policies of the Conservative Party is the very core and justification for the direction with which Maude and Cameron are leading the Tories, it is astonishing that examination of these polls does not bear out the conclusion that Maude draws from them.
We are in danger of a Franz Ferdinand moment here. That is, the joke that a 1920’s spoof news headline read: ‘Franz Ferdinand discovered alive: First World War a mistake’.
Certainly, the polls impressed the pundits. A Populus poll in March 2005 reported in the Times was commented on by Dennis Kavanagh in The British General Election of 2005 ‘A Populus poll found that once a relatively popular policy was identified as a Conservative one, support for it immediately fell away. Labour polling picked up a mood for change but this never translated into support for the Conservatives.’
Robert Worcester, in Labour’s Landslip (his book on the 2005 election), reported on an ICM poll in January [actually February] 2005 for Newsnight, ‘It was clear enough that an obstacle to the Tories’ exploitation of the immigration issue was that it was the Tories who were doing it – another example of image being more powerful than substance. An ICM survey in January found what, while 82 per cent of the public said they supported ‘the idea that immigration should be controlled more strictly’, only 65 per cent said they supported ‘Conservative policy to control immigration more strictly’.
But were these polls as definitive in their conclusions as Kavanagh & Worcester believed and Maude & Rickitt also believed.
It is the conclusion of this analysis that the polls did not show at all what Maude claimed. Also that there were errors in the polls and biases in the database; many of those expressing preference for minor parties were excluded altogether, etc.
THE ICM POLL
The ICM poll need not detain us long. Two questions were asked to two different but similarly weighted panels (although apparently the panels were not weighted by political support).
Yes No
Q1 Do you support the idea that immigration should be controlled more strictly? 82% 18%
Q2 Do you support Conservative policy to control immigration more strictly? 65% 35%
One would expect that committed Labour, LibDem, and other voters who supported the idea of immigration being controlled more strictly, would not support this as strongly once it was defined as a Conservative policy. (One leaves aside the more sophisticated analysis that some respondents might have concluded that they supported control of immigration but they did not believe the Conservative policy would actually do this.)
Although the ICM Survey did not give any breakdown by Party allegiance, it did give a breakdown by regions. In the Labour heartlands, such as Scotland and Northern England, there was a marked difference in response. In Scotland, 87 per cent supported Q1 and only 42% Q2. In Northern England, 81% supported Q1 and only 58% Q2. In the South East, the difference was small. 73 per cent supported Q1 and 69% Q2. Clearly, in Scotland and the North, where in Q1 substantial higher numbers supported a stricter immigration policy, Labour, LibDem and Nationalist voters did not support a Conservative policy.
However, the Populus poll which I refer to below and which did break down voters into Labour, Conservatives, LibDems and Swing, showed clearly that, for example, Labour voters supported unattributed Labour policy on immigration at 63 per cent but this jumped to 80 per cent when it was attributed to Labour. Similarly, 75 per cent of Conservatives supported Labour policy when it was unattributed but this slumped to 56 per cent approval when the policy was attributed to Labour.
So, the ICM poll certainly showed that, once a popular policy was badged as a Conservative policy, support dropped. But that is of no interest unless an opposite effect or a substantially lesser can be shown to have occurred when a policy was badged as Labour or LibDem. In other words, to see if badging a policy as Conservative was contaminating and led to a drop in support while badging it as Labour or LibDem was not and did not lead to a significant drop in support. The ICM poll did not carry out this basic comparison but analysis of the regional results shows that the fall in support for a Conservative–badged policy occurred mainly in Labour areas, as one would expect.
Nothing of significance can be drawn, therefore, from the ICM poll. If the poll had replaced the word ‘Conservative’ with ‘Labour’, there would have been a substantial drop in support between Q1 and Q2.
THE POPULUS POLL
The Populus poll of 4th March 2005 reported in The Times was, however, much more interesting. It was a split poll, with half of the respondents in split ‘A’ and half in split ‘B’.
An extract from the Populus report on the poll is as follows:
"AGREEMENT WITH POLICIES ON IMMIGRATION (ATTRIBUTED & UNATTRIBUTED)
Half the respondents were read a summary of Labour’s immigration policy, attributed to Labour and a summary of Conservative immigration policy without attribution (B). The other half of the sample was read a summary of Conservative immigration policy attributed to the Conservatives and a summary of Labour’s immigration policy without attribution (A)."
The summaries are as follows:
Conservative They will set an annual limit on the number of immigrants able to enter Britain, and give priority to those who would make a positive contribution, as they do in Australia. They will change the work permit system so that people coming to Britain on a temporary permit are no longer able to settle here permanently. They will set up a 24 hour surveillance at ports and airports, stop people who are not genuine refugees applying for asylum, and introduce health checks for immigrants to detect things like HIV, hepatitis and TB.
Labour They will introduce a points system to ensure that those who enter Britain will benefit the country and end the right for immigrants to bring in their relatives. They will only allow skilled workers to settle long-term in the UK. They will introduce English language tests for those who want to stay permanently, and expand detention for failed asylum seekers.
The results were as follows:
Split Poll Results % agreeing with immigration policies All Labour Con LibDem Swing
B Labour policy – attributed 63% 80% 56% 53% 65%
A Labour policy – unattributed 70% 63% 92% 60% 70%
A Conservative policy – attributed 70% 63% 92% 60% 70%
B Conservative policy – unattributed 73% 71% 81% 60% 74%
Split Poll Results Net agreement with immigration policies All Labour Con LibDem Swing
B Labour policy – attributed 34% 63% 22% 11% 35%
A Labour policy – unattributed 32% 34% 54% 12% 30%
A Conservative policy – attributed 43% 29% 87% 20% 41%
B Conservative policy – unattributed 55% 50% 67% 30% 57%
3% margin of error
The net figure was the total in agreement minus the total in disagreement.
PROBLEMS WITH THE POPULUS POLL
There are three problems with this poll.
First, there were incorrect figures published in the line.
“Those agreeing with Labour policy – unattributed” ‘A’
All Labour Con LibDem Swing
The published figures (above) were 70% 63% 92% 60% 70%
While the Poll details revealed the correct figures should be 63% 65% 75% 53% 61%
The net agreement figures contained the correct figures throughout.
As the whole poll exercise turned on the difference in support for attributed and unattributed policies, this was unfortunate. While the polling company says the important matter is the net figure, publication of incorrect headline figures may well have had influence on Francis Maude.
Second, there were far more Labour voters in the survey as a percentage of all voters in the survey than those who actually voted Labour in the general election three months later (May 2005). In the split poll ‘A’ 21.8 per cent are Conservative voters and 30.2 per cent Labour so any question where Party identification was an issue would inevitably show a greater bias in the overall sample against a Conservative policy than a Labour policy and out of line with the actual voting figures three months later. This bias means the ‘all’ column is biased towards Labour.
Third, who were the Swing voters? According to Populus, these are an amalgamation of:
52 per cent of the ‘don’t knows’ or ‘refusers’ who define themselves as ‘floating voters’.
a) 25 per cent of those who gave their voting intention as Conservative, Labour or LibDem but said they ‘may well’ change their mind. Of the total Swing voters, one-third came from group (a) and two-thirds from group
b) (One should note in passing that ten per cent of the Populus voters who wanted to vote for other than for one of the three major parties are not catered for at all in the Populus poll. This was another flaw in the whole exercise. This ten per cent cannot by definition be ‘Swing’ voters – indeed, they are the opposite of this but they simply disappear from the results.)
In examining the ‘may well change their mind’ vote, there are no Populus figures for the original voting intention of these ‘Swing’ voters but it is reasonable to assume that they were in proportion to the firm voters.
To take the (A) split poll the details of the Populus poll (Table 18 of the Fieldwork) show the voting intention was:
Labour Con LibDem Swing
Populus Poll Percentage 30.2% 21.8% 14.84% 33.05%
Add (25% 'Swing' voters subtracted by Populus) 10.0% 7.2% 4.90% -
40.2% 29.0% 19.74% -
It is at this point that one can see how the Populus poll has eliminated the ten per cent of the electorate who intended to vote for other than the three parties.
The Conservatives are clearly under-represented, even in terms of the voting intentions recorded by Populus, in detailing the voting question in the same poll, which shows the following figures:
Labour 39%
Conservative 32%
LibDems 20%
Others 9%
More important, in the election three months later, Labour only got 36.2 per cent of the poll and the Tories 33.2 per cent, with LibDems on 22.6 per cent (GB only).
With all this technical background, what apparently was murderous for Mr. Maude about this poll? The pollsters at Populus indicate that it is apparently in the Section ‘Net Agreement with Immigration Policies’ and, in particular, the difference in support for the ‘Conservative policy – attributed’ and ‘Conservative policy – unattributed’. On the face of it, this looks quite a big difference, 43 per cent of the total polled supported Conservative policy when it was attributed and 55 per cent when it was unattributed. The pollsters draw particular attention to the position among ‘Swing’ voters where the difference was between 41 per cent support for ‘Conservative policy – attributed’ and 57 per cent support for ‘Conservative policy – unattributed’.
As one would expect, there was a swing to support ‘Labour policies – attributed’ from ‘Labour policies – unattributed’ by Labour voters and a swing to support ‘Conservative policies – attributed’ from ‘Conservative policies – unattributed’ among Conservative voters.
But there is a problem among the ‘Swing’ voters. There was actually a swing to ‘Labour policy – attributed’ from ‘Labour policy – unattributed’. But why should this happen? One would expect ‘Swing’ Conservative voters who ‘may well change their minds’ to swing against a policy when it was attributed to Labour. For the swing among ‘Swing’ voters to be in favour of a Labour-badged policy can only mean that there were considerably more Labour voters in the ‘Swing’ voters than Conservative voters and the whole poll to be flawed – and we have shown this to be the case. Surely badging any policy as Labour, should not attract a net favourable swing from a popular ‘unbadged’ policy among supposedly ‘Swing’ voters? It implies that the Labour badge is overall an attraction to voters, slapped on any policy. It certainly was not three months later. It appears from the above that the Labour and LibDem ‘Swing’ voters ‘who may well change their mind’ were over double the number of Conservative ‘Swing’ voters ‘who may well change their mind’.
Of course, we are talking about small numbers here. (The figures are extracted from Tables 18-21 of the Fieldwork)
Split A Split B
Labour unattributed Labour attributed Difference
‘Swing’ voters FOR 143 150 +7
AGAINST 74 68 -6
NET 69 82 13
Split A Split B
Cons unattributed Cons attributed Difference
‘Swing’ voters FOR 170 165 -5
AGAINST 39 68 +29
NET 131 97 34
Populus dismisses the movement of 13 voters in the Labour data of the survey as within the margin of error but the movement of 34 voters, almost certainly caused by an over-high representation of Labour- and LibDems-leaning voters among ‘Swing’ voters, has led to upheaval in the Conservative Party.
FUTURUS/31 March 2010
The conviction that the Conservative Party was inherently unpopular as a brand and was contaminated had been gaining ground among some Conservatives at least since the leadership of Ian Duncan Smith.
In 2005, two opinion polls appeared influential in fostering this conviction among leading Tories. Francis Maude waved what he called ‘killer polls’ at a Conservative Party Conference on which he had already commented on in August 16th 2005 in an interview with Epolitix.com: ‘It wasn’t that people particularly disliked our policies, lots of research showed that people approved of the policies by and large until you told them they were Conservative Party policies and that made them feel differently about it’. This is clearly a widespread view since Adam Rickitt, not a noted commentator on polls, told the Daily Telegraph on 6th June 2006: ‘I recall with horror the findings of a Populus focus group before the last election where a Conservative policy was presented to people without the Conservative label attached, 82 per cent of respondents agreed with it. When people were told it was a Conservative policy, that support fell to 65 per cent. The problem was the brand’. (Actually Rickitt is referring to an opinion poll conducted by ICM, not a focus group by Populus.)
What actually happened and were Francis Maude’s killer polls actually as murderous as he made out?
As Maude said to ePolitix, “We need to understand what’s happening”.
As the contention that the polls showed that voters approved Conservative policies until they were told they were the policies of the Conservative Party is the very core and justification for the direction with which Maude and Cameron are leading the Tories, it is astonishing that examination of these polls does not bear out the conclusion that Maude draws from them.
We are in danger of a Franz Ferdinand moment here. That is, the joke that a 1920’s spoof news headline read: ‘Franz Ferdinand discovered alive: First World War a mistake’.
Certainly, the polls impressed the pundits. A Populus poll in March 2005 reported in the Times was commented on by Dennis Kavanagh in The British General Election of 2005 ‘A Populus poll found that once a relatively popular policy was identified as a Conservative one, support for it immediately fell away. Labour polling picked up a mood for change but this never translated into support for the Conservatives.’
Robert Worcester, in Labour’s Landslip (his book on the 2005 election), reported on an ICM poll in January [actually February] 2005 for Newsnight, ‘It was clear enough that an obstacle to the Tories’ exploitation of the immigration issue was that it was the Tories who were doing it – another example of image being more powerful than substance. An ICM survey in January found what, while 82 per cent of the public said they supported ‘the idea that immigration should be controlled more strictly’, only 65 per cent said they supported ‘Conservative policy to control immigration more strictly’.
But were these polls as definitive in their conclusions as Kavanagh & Worcester believed and Maude & Rickitt also believed.
It is the conclusion of this analysis that the polls did not show at all what Maude claimed. Also that there were errors in the polls and biases in the database; many of those expressing preference for minor parties were excluded altogether, etc.
THE ICM POLL
The ICM poll need not detain us long. Two questions were asked to two different but similarly weighted panels (although apparently the panels were not weighted by political support).
Yes No
Q1 Do you support the idea that immigration should be controlled more strictly? 82% 18%
Q2 Do you support Conservative policy to control immigration more strictly? 65% 35%
One would expect that committed Labour, LibDem, and other voters who supported the idea of immigration being controlled more strictly, would not support this as strongly once it was defined as a Conservative policy. (One leaves aside the more sophisticated analysis that some respondents might have concluded that they supported control of immigration but they did not believe the Conservative policy would actually do this.)
Although the ICM Survey did not give any breakdown by Party allegiance, it did give a breakdown by regions. In the Labour heartlands, such as Scotland and Northern England, there was a marked difference in response. In Scotland, 87 per cent supported Q1 and only 42% Q2. In Northern England, 81% supported Q1 and only 58% Q2. In the South East, the difference was small. 73 per cent supported Q1 and 69% Q2. Clearly, in Scotland and the North, where in Q1 substantial higher numbers supported a stricter immigration policy, Labour, LibDem and Nationalist voters did not support a Conservative policy.
However, the Populus poll which I refer to below and which did break down voters into Labour, Conservatives, LibDems and Swing, showed clearly that, for example, Labour voters supported unattributed Labour policy on immigration at 63 per cent but this jumped to 80 per cent when it was attributed to Labour. Similarly, 75 per cent of Conservatives supported Labour policy when it was unattributed but this slumped to 56 per cent approval when the policy was attributed to Labour.
So, the ICM poll certainly showed that, once a popular policy was badged as a Conservative policy, support dropped. But that is of no interest unless an opposite effect or a substantially lesser can be shown to have occurred when a policy was badged as Labour or LibDem. In other words, to see if badging a policy as Conservative was contaminating and led to a drop in support while badging it as Labour or LibDem was not and did not lead to a significant drop in support. The ICM poll did not carry out this basic comparison but analysis of the regional results shows that the fall in support for a Conservative–badged policy occurred mainly in Labour areas, as one would expect.
Nothing of significance can be drawn, therefore, from the ICM poll. If the poll had replaced the word ‘Conservative’ with ‘Labour’, there would have been a substantial drop in support between Q1 and Q2.
THE POPULUS POLL
The Populus poll of 4th March 2005 reported in The Times was, however, much more interesting. It was a split poll, with half of the respondents in split ‘A’ and half in split ‘B’.
An extract from the Populus report on the poll is as follows:
"AGREEMENT WITH POLICIES ON IMMIGRATION (ATTRIBUTED & UNATTRIBUTED)
Half the respondents were read a summary of Labour’s immigration policy, attributed to Labour and a summary of Conservative immigration policy without attribution (B). The other half of the sample was read a summary of Conservative immigration policy attributed to the Conservatives and a summary of Labour’s immigration policy without attribution (A)."
The summaries are as follows:
Conservative They will set an annual limit on the number of immigrants able to enter Britain, and give priority to those who would make a positive contribution, as they do in Australia. They will change the work permit system so that people coming to Britain on a temporary permit are no longer able to settle here permanently. They will set up a 24 hour surveillance at ports and airports, stop people who are not genuine refugees applying for asylum, and introduce health checks for immigrants to detect things like HIV, hepatitis and TB.
Labour They will introduce a points system to ensure that those who enter Britain will benefit the country and end the right for immigrants to bring in their relatives. They will only allow skilled workers to settle long-term in the UK. They will introduce English language tests for those who want to stay permanently, and expand detention for failed asylum seekers.
The results were as follows:
Split Poll Results % agreeing with immigration policies All Labour Con LibDem Swing
B Labour policy – attributed 63% 80% 56% 53% 65%
A Labour policy – unattributed 70% 63% 92% 60% 70%
A Conservative policy – attributed 70% 63% 92% 60% 70%
B Conservative policy – unattributed 73% 71% 81% 60% 74%
Split Poll Results Net agreement with immigration policies All Labour Con LibDem Swing
B Labour policy – attributed 34% 63% 22% 11% 35%
A Labour policy – unattributed 32% 34% 54% 12% 30%
A Conservative policy – attributed 43% 29% 87% 20% 41%
B Conservative policy – unattributed 55% 50% 67% 30% 57%
3% margin of error
The net figure was the total in agreement minus the total in disagreement.
PROBLEMS WITH THE POPULUS POLL
There are three problems with this poll.
First, there were incorrect figures published in the line.
“Those agreeing with Labour policy – unattributed” ‘A’
All Labour Con LibDem Swing
The published figures (above) were 70% 63% 92% 60% 70%
While the Poll details revealed the correct figures should be 63% 65% 75% 53% 61%
The net agreement figures contained the correct figures throughout.
As the whole poll exercise turned on the difference in support for attributed and unattributed policies, this was unfortunate. While the polling company says the important matter is the net figure, publication of incorrect headline figures may well have had influence on Francis Maude.
Second, there were far more Labour voters in the survey as a percentage of all voters in the survey than those who actually voted Labour in the general election three months later (May 2005). In the split poll ‘A’ 21.8 per cent are Conservative voters and 30.2 per cent Labour so any question where Party identification was an issue would inevitably show a greater bias in the overall sample against a Conservative policy than a Labour policy and out of line with the actual voting figures three months later. This bias means the ‘all’ column is biased towards Labour.
Third, who were the Swing voters? According to Populus, these are an amalgamation of:
52 per cent of the ‘don’t knows’ or ‘refusers’ who define themselves as ‘floating voters’.
a) 25 per cent of those who gave their voting intention as Conservative, Labour or LibDem but said they ‘may well’ change their mind. Of the total Swing voters, one-third came from group (a) and two-thirds from group
b) (One should note in passing that ten per cent of the Populus voters who wanted to vote for other than for one of the three major parties are not catered for at all in the Populus poll. This was another flaw in the whole exercise. This ten per cent cannot by definition be ‘Swing’ voters – indeed, they are the opposite of this but they simply disappear from the results.)
In examining the ‘may well change their mind’ vote, there are no Populus figures for the original voting intention of these ‘Swing’ voters but it is reasonable to assume that they were in proportion to the firm voters.
To take the (A) split poll the details of the Populus poll (Table 18 of the Fieldwork) show the voting intention was:
Labour Con LibDem Swing
Populus Poll Percentage 30.2% 21.8% 14.84% 33.05%
Add (25% 'Swing' voters subtracted by Populus) 10.0% 7.2% 4.90% -
40.2% 29.0% 19.74% -
It is at this point that one can see how the Populus poll has eliminated the ten per cent of the electorate who intended to vote for other than the three parties.
The Conservatives are clearly under-represented, even in terms of the voting intentions recorded by Populus, in detailing the voting question in the same poll, which shows the following figures:
Labour 39%
Conservative 32%
LibDems 20%
Others 9%
More important, in the election three months later, Labour only got 36.2 per cent of the poll and the Tories 33.2 per cent, with LibDems on 22.6 per cent (GB only).
With all this technical background, what apparently was murderous for Mr. Maude about this poll? The pollsters at Populus indicate that it is apparently in the Section ‘Net Agreement with Immigration Policies’ and, in particular, the difference in support for the ‘Conservative policy – attributed’ and ‘Conservative policy – unattributed’. On the face of it, this looks quite a big difference, 43 per cent of the total polled supported Conservative policy when it was attributed and 55 per cent when it was unattributed. The pollsters draw particular attention to the position among ‘Swing’ voters where the difference was between 41 per cent support for ‘Conservative policy – attributed’ and 57 per cent support for ‘Conservative policy – unattributed’.
As one would expect, there was a swing to support ‘Labour policies – attributed’ from ‘Labour policies – unattributed’ by Labour voters and a swing to support ‘Conservative policies – attributed’ from ‘Conservative policies – unattributed’ among Conservative voters.
But there is a problem among the ‘Swing’ voters. There was actually a swing to ‘Labour policy – attributed’ from ‘Labour policy – unattributed’. But why should this happen? One would expect ‘Swing’ Conservative voters who ‘may well change their minds’ to swing against a policy when it was attributed to Labour. For the swing among ‘Swing’ voters to be in favour of a Labour-badged policy can only mean that there were considerably more Labour voters in the ‘Swing’ voters than Conservative voters and the whole poll to be flawed – and we have shown this to be the case. Surely badging any policy as Labour, should not attract a net favourable swing from a popular ‘unbadged’ policy among supposedly ‘Swing’ voters? It implies that the Labour badge is overall an attraction to voters, slapped on any policy. It certainly was not three months later. It appears from the above that the Labour and LibDem ‘Swing’ voters ‘who may well change their mind’ were over double the number of Conservative ‘Swing’ voters ‘who may well change their mind’.
Of course, we are talking about small numbers here. (The figures are extracted from Tables 18-21 of the Fieldwork)
Split A Split B
Labour unattributed Labour attributed Difference
‘Swing’ voters FOR 143 150 +7
AGAINST 74 68 -6
NET 69 82 13
Split A Split B
Cons unattributed Cons attributed Difference
‘Swing’ voters FOR 170 165 -5
AGAINST 39 68 +29
NET 131 97 34
Populus dismisses the movement of 13 voters in the Labour data of the survey as within the margin of error but the movement of 34 voters, almost certainly caused by an over-high representation of Labour- and LibDems-leaning voters among ‘Swing’ voters, has led to upheaval in the Conservative Party.
FUTURUS/31 March 2010