A dinner-time phone call from a Tory pollster sparks some thoughts about the wording of political questions and the biases that appear between the words
I received a curious phone call last night. Actually, I received a Curia phone call last night; David Farrar's polling company wanted to know whether I'd spare five minutes to take part in a political survey. Never one to miss a chance to research the researchers, I naturally said yes.
I was asked the questions you'd expect – who I'd vote for if an election was held tomorrow, whether I could be convinced to change my mind, which issue was most important to me and who should be Prime Minister. I was asked which party I trusted most, had the best team behind the leader, and thought they had the best interests of the country at heart.
And they wanted to know about just two National ministers – John Key and Bill English. I had to find them either favourable or unfavourable (or very), there was no middle 'meh' option.
They were direct questions, requiring one word answers – yes or no, a party name, or a favorability ranking. I can understand the clarity that gives, and I'm sure over the years those yes or no answers to certain questions have become well-known weather vanes for trends or voters' willingness to be won over.
But when I was asked whether I thought New Zealand was going in the right or wrong direction, I had to stop myself replying, 'in what area?'
I was particularly intrigued by the wording used by my pollster.
I can't remember word for word, so I'll refrain from using full quote marks. But I remember my eye-brows going up to a few questions. Such as the one above about the best interests of New Zealand.
As I remember it, I wasn't asked which party I thought was best for New Zealand, but which party thought it was best for New Zealand, as if testing the parties' confidence levels rather than my views.
The pollster took me through a range of portfolio areas, asking which party I thought would do best in each one... Health, education, tax, foreign policy, welfare...
But the question changed subtly, but significantly for a couple, and in a way that seemed somewhat leading. I assume it was on purpose, and therefore am left wondering whether that doesn't undermine the use of the data.
On the environment, I was asked not which party would do best, but which was most balanced. Now, I can't remember the precise wording, but it was a question constructed differently from the others. I remember thinking, 'balanced is a word so heavily used by National that it points in that party's direction'....
What I do remember exactly was being asked which party would be "toughest on crime". Not which party would make us safer, or reduce crime or address the causes of crime. Which would be "toughest". As if being tough on crime equated to being the best party to deal with crime.
I'm someone whose heart sinks when I read that our prison population has topped 200 per 100,000. I see that as a failure of our country, not something to be proud of. I drive past that prison rearing over Auckland's southern motorway and wonder why all the money being spent there isn't going into education or innovation. I think the four prisons Labour built during its three terms represents a craven cop-out.
So I told the pollster which party I thought "toughest", but they learnt nothing about which party I'd support on law and order issues. Which presumably was what they really wanted to know.
The climax of the interview were questions about the two marquee policies announced by the major parties this year. It was Labour's Tax-free zone vs National's partial privatisation – interesting what most concerns DPF and his clients.
Again, the questions had a noticeable lean. Asking about Labour's policy, the pollster told me that critics say it's unaffordable while supporters say it can be funded by increasing taxes on the rich (a simplistic description, but no more than Labour deserves given its vague outline). However National's privatisation plans were proposed to simply 'pay off debt'. There was no talk of critics, such as those who complain at the loss of dividends or foreign encroachment, or indeed at other reasons to support the plan, such as strengthening our stock exchange and access to capital.
Now DPF can ask whatever questions he wants, but by asking elading questions, don't you get less accurate results? Aren't people more likely to tell you what you want to hear, rather than the views that will actually drive them when they're standing in the booth on election day?
David, feel free to respond. Or perhaps our own Rob Salmond might have thoughts - Rob? Anyway, thanks for the call, David!