June 3, 2025 • 13 mins
How is it that two recent polls had such starkly different outcomes?
The latest RNZ Reid Research poll —out this morning— has the right bloc on 46.4, behind the left bloc on 50.3.
But the latest 1 News Verian poll —released last night— has the right bloc on 50-percent, well ahead of the left bloc on 45.
Curia Market Research Owner David Farrar told Kerre Woodham that when you poll 1000 people, they say there’s a 3% margin of error – so if a party is sitting at 50%, in reality they’re somewhere between 47% and 53%.
His advice for conflicting polls is to average them out, as that will generally give you a pretty good idea.
LISTEN ABOVE
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:06):
You're listening to the Kerrywood and Morning's podcast from News Talks,
he'd be.
Speaker 2 (00:11):
We've had two very different polls out in a twelve
hour period from One News last night. Both National and
Labor have slid in the latest One News variant poll,
while New Zealand First have moved to their strongest position
in eight years. If an election were to be held today,
the Right Block of National Act in New Zealand First
would have sixty three seats, enough to form a coalition,
(00:34):
while the Left Bloc would have fifty eight. Then from
Ready in New Zealand after the budget and pay equity changes,
the Left Block would have the support to turf the
coalition out of power. To give us some insight into
how the polls work, I'm joined by bloger, media commentator
and owner of Curier Market Research, David Fara.
Speaker 3 (00:52):
Good morning to you, Good morning.
Speaker 2 (00:56):
You can understand skepticism about these polls, can't you.
Speaker 3 (01:01):
Yeah, Look, it's almost Polster's worst nightmare is when you
have two out within twelve hours and they tell a
different story, and raw they do tell a different story.
I don't think we should overemphasize that they're both saying.
It's very close. One of them has the center right
(01:21):
with a little majority of two seats. The other has
a center left with a local majority of two seats
you need sixty one. One has sixty three center right there.
So one way to look at is they're both saying
this is a very close result and it could go
either way. In terms of though why they're different, some
(01:42):
of it's just random luck. When you poll a thousand people,
they say there's a three percent marge of error, So
that means that if you say a parties on fifty percent,
it's somewhere between forty seven and fifty three percent. So
there has plans. Possibly some of the difference between the
polls just that you can't see them as absolutely precise.
(02:06):
They give you a no oble. Sorry, having said that,
with labor, certainly one's got them on twenty nine, one's
got them on thirty three. There is quite a difference
there in that can come down to what we course
of non sampling there. I'll try not to get through
geeky about this, but it can be as simple as
(02:30):
do you do phone polling as well as online polling
Radio New Zealand Read Research and now fully online panel.
Now I was going to say, call my brunt and
then they'll call Varian do half phone and a half
online panal?
Speaker 2 (02:45):
What is an online panel?
Speaker 3 (02:48):
Online panel is where people sign up to be on
a panel. You generally get some sort of reward, And
some of these panels are really big, like they've got
hundreds of thousands of New Zealanders on there. And what happens.
Did you get an offer to take partner pole? You say, yeah,
I'll take part this one and I'll go up. Until
(03:09):
a few years ago, online panel poles weren't seen as
reliable as phone poles because they are a bit more
self selecting. You you have to go on the online panel,
you're getting paid to take part. Our phone poles have
worked on politeness. Yeah, a nice young person rings you
up at home and says, can you spare five minutes?
But in the US, where they have thousands of poles,
(03:32):
at the last election, they found that actually the online
penal polls are just as reliable overall as poles which
are both phone and online. Now, so it doesn't mean
that the reg research pole, which is online only, is
less reliable, but that might explain some of the difference
(03:53):
because they are online only. Ralverian are also do a
phone combat. The other factors that come in is how
they wait to match the population. The reg Research pole
waits it by gender, agent area. The Varian pole also
waits by ethnicity and I think education, and that might
(04:14):
explain some of those differences. Also in the end, my
always advice is we you conflicting ones, just average them.
Usually will give you a fairly go there. And ironically,
if you average these two out, it's a hung parliament
with sixty seats each the other, which would.
Speaker 2 (04:32):
Probably be about right. But explain waiting to me. And
also when you say that there's hundreds of thousands that
sign up to these online panels to get their twenty
dollars Brisco's voucher, why did they only take a five
hundred and two And how do they only take five
hundred and two?
Speaker 3 (04:48):
What they do is it's the first five hundred people
who respond, who who match the criteria. So if you're
one of the first few hundred, you'll get in once
they've got, for example, once you've got two hundred and
fifty woman, then they'll only take men until they also
have two hundred and fifty men. So it's a mixture
(05:11):
of who's first to respond, plus do for the demographics
of what they need to say. Lo, this is fairly
representative of New Zone. The other thing people do with
the online panels is you often exclude people who did
the same pole within the last six months, so it's
not just the same people you keep going back to.
Speaker 2 (05:33):
So what happens if you don't get enough, say PACIFICA responding, Well.
Speaker 3 (05:39):
That's a great question, for it goes into the waiting.
Let's say Pacifica around ten percent of New Zealand population,
but say you only get fifty Pacifica in the pole, Well,
you know that it's not representative. So you give the
Pacifica respondents a waiting and in this case it would
be of two because they're meant to be ten percent
(06:02):
of the population, but they are only five percent of
your pole. So you wake them up so that overall
you think it still looks like New Zealand. Now waiting
Everyone does it. You have to do it because otherwise,
especially with phone polling, you might be calling five hundred
people to find the one person who's a university educated
(06:28):
person earning fifty thousand dollars a year who lives in
auct to do his age, thirty five, etc. But you
want the weights to be as small as possible. Generally,
you know, if you're waiting more than two that's that's
a bit risky because if your will sample was itself
(06:50):
slightly wonky because you just happen to get you particular
labor or voting national one and you're waiting them by
a large amount, that can actually make the eraror worse.
So one of the things pulses try to do is
get as close as possible to the New Zealand population
and only use waiting to do minor changes. So one
(07:12):
of the things I always look at with my polls
is what would be the result before waiting and what
would be the result after waiting. And you're usually pretty
happy if it doesn't changing of the parties by more
than half a percent or one percent. If the waiting
is changing your results by like three or four percent,
then you sort of think, well, maybe I have to
(07:33):
work harder to get that better sample.
Speaker 2 (07:36):
But Okay, if you're going to give say two points
to the PACIFICA respondents, they that doesn't necessarily mean they're
going to think the same way. But you're giving you
you're allowing them to have waiting surely in that and
(07:56):
what the people who did respond vote for, if you know.
Speaker 3 (08:00):
What I mean. But yeah, the thing is you wait
all the PACIFICA respondents. So some of them might say
they vote national, some might say they vote labor. But
what that will do is if more Pacifica are saying
they vote labor than National, when you wake them upwards,
that will increase the party vote for labor. Probably the
(08:21):
easier one for people to reflect on is gender because
we all know it should be around fifty to fifty
and all the polls tend to show women are more
likely to vote for labor or Greens, men are more
likely to vote for a center right party. If in
your polling you end up with our poll which is
only forty percent woman, well that's not the country. So
(08:47):
you wake them up to fifty percent and that will
increase the vote for labor and Greens. But that is
reflecting that that's the population. So way things what you
call us a necessary evil every pulse that has to
do it because you never get a perfect sample, but
you want to minimize how much you do it.
Speaker 2 (09:09):
And how much do they cost to do.
Speaker 3 (09:12):
Oh, there's a very good question look, they're into the
for a thousand respondents, they're into generally the tens of
thousands of dollars, not lots of tens, et cetera. But
it's definitely a five figure sun because if you're phoning
people up, you have all the costs of a call center,
(09:33):
the office, the outgoing calls, et cetera. And it actually
takes a lot of phone calls often to get hold
of someone. You have all the disconnected numbers, the answer phones,
and then there's the people say, Okay, a bit busy,
can you try me tomorrow, call me back, or the
people just say don't want to take part. So that's
(09:53):
quite a big cost. And for the online panel, because
you're actually paying people to take part, that is also
quite costly. Different panels are charged different amounts by no
One panel charges fifteen dollars per response, so before you
have any of your other costs, if you get a
thousand people from that online panel, that's fifteen thousand dollars
(10:17):
before you even have the analysis, the writing, et cetera.
Some panels are cheaper than that. I use around three
or four different panels because some of them are cheaper,
some of them have more people and hard to get
areas like rural New Zealand, older voters, etc. So yeah, no,
the pole polls aren't cheap. And this is one of
(10:38):
the things with the media company to do them too,
is if you've paid let's say fifteen twenty thousand dollars
for a pole, you want to make sure you get
big stories out of it, even if sometimes the poles
are things haven't changed much from two months ago. Perhaps
the other thing just Worth mentioned curious, I don't know
what the media companies get charged. Some posters will actually
(11:02):
do it almost cost for the media company. So it's
about cheaper because they get the publicity of having their
brand name out there, et cetera.
Speaker 2 (11:11):
And it is like, in a way it has just
been used to generate stories because ultimately nothing much is
I mean, as Winston Peters famously said, the only poll
that matters is on election day.
Speaker 3 (11:23):
It does. But look, we're the useful is I always
Look when I talk about the trend, I don't just
mean the differ between one pole on the next, but
lot if there's a trend over multiple months, and the
poll I do for the Taxpayers Union, you know we're monthly,
so you actually see the trends there more that can
(11:44):
tell a story. One example is Ja Sindra doing it
at the height of COVID popularity, had what we call
a plus eighty percent approval rating. That means like ninety
percent said using a great job, ten percent a poor job.
And I joke that Jesus Christ came back today, he
would not get plus eighty percent approval rating. She left
(12:05):
office two years later with a minus one percent rating.
So there was this massive trend and that did tell
your story. Likewise, you know parties, Labor's party vote and
that time went from fifty percent down to twenty six percent.
So they definitely do tell a story, but not so
much the individual poll. It's more what's the trend here?
(12:28):
And to a degree though the last twelve months, every
poll has been is pretty close. Most of them have
the government and the lead, but never by more than
two or three seats. Some of them have the opposition
and the lead again by two or three seats. Where
it will probably start to get really interesting is closer
to the election, when people start to tune more into
(12:52):
politics and say, yo, okay, this is how I voted
last time. Am I going to change my mind? The
other thing, you get the poles. So as issues, you
know what you think of the budget? What did you
think of pay equity?
Speaker 2 (13:06):
Yeah, we saw differing results in that one as well, David,
Thank you for that explanation.
Speaker 1 (13:11):
For more from Kerry Wood and Mornings, listen live to
News Talks at B from nine am weekdays, or follow
the podcast on iHeartRadio
You're listening to the Kerrywood and Morning's podcast from News Talks,
he'd be.
Speaker 2 (00:11):
We've had two very different polls out in a twelve
hour period from One News last night. Both National and
Labor have slid in the latest One News variant poll,
while New Zealand First have moved to their strongest position
in eight years. If an election were to be held today,
the Right Block of National Act in New Zealand First
would have sixty three seats, enough to form a coalition,
(00:34):
while the Left Bloc would have fifty eight. Then from
Ready in New Zealand after the budget and pay equity changes,
the Left Block would have the support to turf the
coalition out of power. To give us some insight into
how the polls work, I'm joined by bloger, media commentator
and owner of Curier Market Research, David Fara.
Speaker 3 (00:52):
Good morning to you, Good morning.
Speaker 2 (00:56):
You can understand skepticism about these polls, can't you.
Speaker 3 (01:01):
Yeah, Look, it's almost Polster's worst nightmare is when you
have two out within twelve hours and they tell a
different story, and raw they do tell a different story.
I don't think we should overemphasize that they're both saying.
It's very close. One of them has the center right
(01:21):
with a little majority of two seats. The other has
a center left with a local majority of two seats
you need sixty one. One has sixty three center right there.
So one way to look at is they're both saying
this is a very close result and it could go
either way. In terms of though why they're different, some
(01:42):
of it's just random luck. When you poll a thousand people,
they say there's a three percent marge of error, So
that means that if you say a parties on fifty percent,
it's somewhere between forty seven and fifty three percent. So
there has plans. Possibly some of the difference between the
polls just that you can't see them as absolutely precise.
(02:06):
They give you a no oble. Sorry, having said that,
with labor, certainly one's got them on twenty nine, one's
got them on thirty three. There is quite a difference
there in that can come down to what we course
of non sampling there. I'll try not to get through
geeky about this, but it can be as simple as
(02:30):
do you do phone polling as well as online polling
Radio New Zealand Read Research and now fully online panel.
Now I was going to say, call my brunt and
then they'll call Varian do half phone and a half
online panal?
Speaker 2 (02:45):
What is an online panel?
Speaker 3 (02:48):
Online panel is where people sign up to be on
a panel. You generally get some sort of reward, And
some of these panels are really big, like they've got
hundreds of thousands of New Zealanders on there. And what happens.
Did you get an offer to take partner pole? You say, yeah,
I'll take part this one and I'll go up. Until
(03:09):
a few years ago, online panel poles weren't seen as
reliable as phone poles because they are a bit more
self selecting. You you have to go on the online panel,
you're getting paid to take part. Our phone poles have
worked on politeness. Yeah, a nice young person rings you
up at home and says, can you spare five minutes?
But in the US, where they have thousands of poles,
(03:32):
at the last election, they found that actually the online
penal polls are just as reliable overall as poles which
are both phone and online. Now, so it doesn't mean
that the reg research pole, which is online only, is
less reliable, but that might explain some of the difference
(03:53):
because they are online only. Ralverian are also do a
phone combat. The other factors that come in is how
they wait to match the population. The reg Research pole
waits it by gender, agent area. The Varian pole also
waits by ethnicity and I think education, and that might
(04:14):
explain some of those differences. Also in the end, my
always advice is we you conflicting ones, just average them.
Usually will give you a fairly go there. And ironically,
if you average these two out, it's a hung parliament
with sixty seats each the other, which would.
Speaker 2 (04:32):
Probably be about right. But explain waiting to me. And
also when you say that there's hundreds of thousands that
sign up to these online panels to get their twenty
dollars Brisco's voucher, why did they only take a five
hundred and two And how do they only take five
hundred and two?
Speaker 3 (04:48):
What they do is it's the first five hundred people
who respond, who who match the criteria. So if you're
one of the first few hundred, you'll get in once
they've got, for example, once you've got two hundred and
fifty woman, then they'll only take men until they also
have two hundred and fifty men. So it's a mixture
(05:11):
of who's first to respond, plus do for the demographics
of what they need to say. Lo, this is fairly
representative of New Zone. The other thing people do with
the online panels is you often exclude people who did
the same pole within the last six months, so it's
not just the same people you keep going back to.
Speaker 2 (05:33):
So what happens if you don't get enough, say PACIFICA responding, Well.
Speaker 3 (05:39):
That's a great question, for it goes into the waiting.
Let's say Pacifica around ten percent of New Zealand population,
but say you only get fifty Pacifica in the pole, Well,
you know that it's not representative. So you give the
Pacifica respondents a waiting and in this case it would
be of two because they're meant to be ten percent
(06:02):
of the population, but they are only five percent of
your pole. So you wake them up so that overall
you think it still looks like New Zealand. Now waiting
Everyone does it. You have to do it because otherwise,
especially with phone polling, you might be calling five hundred
people to find the one person who's a university educated
(06:28):
person earning fifty thousand dollars a year who lives in
auct to do his age, thirty five, etc. But you
want the weights to be as small as possible. Generally,
you know, if you're waiting more than two that's that's
a bit risky because if your will sample was itself
(06:50):
slightly wonky because you just happen to get you particular
labor or voting national one and you're waiting them by
a large amount, that can actually make the eraror worse.
So one of the things pulses try to do is
get as close as possible to the New Zealand population
and only use waiting to do minor changes. So one
(07:12):
of the things I always look at with my polls
is what would be the result before waiting and what
would be the result after waiting. And you're usually pretty
happy if it doesn't changing of the parties by more
than half a percent or one percent. If the waiting
is changing your results by like three or four percent,
then you sort of think, well, maybe I have to
(07:33):
work harder to get that better sample.
Speaker 2 (07:36):
But Okay, if you're going to give say two points
to the PACIFICA respondents, they that doesn't necessarily mean they're
going to think the same way. But you're giving you
you're allowing them to have waiting surely in that and
(07:56):
what the people who did respond vote for, if you know.
Speaker 3 (08:00):
What I mean. But yeah, the thing is you wait
all the PACIFICA respondents. So some of them might say
they vote national, some might say they vote labor. But
what that will do is if more Pacifica are saying
they vote labor than National, when you wake them upwards,
that will increase the party vote for labor. Probably the
(08:21):
easier one for people to reflect on is gender because
we all know it should be around fifty to fifty
and all the polls tend to show women are more
likely to vote for labor or Greens, men are more
likely to vote for a center right party. If in
your polling you end up with our poll which is
only forty percent woman, well that's not the country. So
(08:47):
you wake them up to fifty percent and that will
increase the vote for labor and Greens. But that is
reflecting that that's the population. So way things what you
call us a necessary evil every pulse that has to
do it because you never get a perfect sample, but
you want to minimize how much you do it.
Speaker 2 (09:09):
And how much do they cost to do.
Speaker 3 (09:12):
Oh, there's a very good question look, they're into the
for a thousand respondents, they're into generally the tens of
thousands of dollars, not lots of tens, et cetera. But
it's definitely a five figure sun because if you're phoning
people up, you have all the costs of a call center,
(09:33):
the office, the outgoing calls, et cetera. And it actually
takes a lot of phone calls often to get hold
of someone. You have all the disconnected numbers, the answer phones,
and then there's the people say, Okay, a bit busy,
can you try me tomorrow, call me back, or the
people just say don't want to take part. So that's
(09:53):
quite a big cost. And for the online panel, because
you're actually paying people to take part, that is also
quite costly. Different panels are charged different amounts by no
One panel charges fifteen dollars per response, so before you
have any of your other costs, if you get a
thousand people from that online panel, that's fifteen thousand dollars
(10:17):
before you even have the analysis, the writing, et cetera.
Some panels are cheaper than that. I use around three
or four different panels because some of them are cheaper,
some of them have more people and hard to get
areas like rural New Zealand, older voters, etc. So yeah, no,
the pole polls aren't cheap. And this is one of
(10:38):
the things with the media company to do them too,
is if you've paid let's say fifteen twenty thousand dollars
for a pole, you want to make sure you get
big stories out of it, even if sometimes the poles
are things haven't changed much from two months ago. Perhaps
the other thing just Worth mentioned curious, I don't know
what the media companies get charged. Some posters will actually
(11:02):
do it almost cost for the media company. So it's
about cheaper because they get the publicity of having their
brand name out there, et cetera.
Speaker 2 (11:11):
And it is like, in a way it has just
been used to generate stories because ultimately nothing much is
I mean, as Winston Peters famously said, the only poll
that matters is on election day.
Speaker 3 (11:23):
It does. But look, we're the useful is I always
Look when I talk about the trend, I don't just
mean the differ between one pole on the next, but
lot if there's a trend over multiple months, and the
poll I do for the Taxpayers Union, you know we're monthly,
so you actually see the trends there more that can
(11:44):
tell a story. One example is Ja Sindra doing it
at the height of COVID popularity, had what we call
a plus eighty percent approval rating. That means like ninety
percent said using a great job, ten percent a poor job.
And I joke that Jesus Christ came back today, he
would not get plus eighty percent approval rating. She left
(12:05):
office two years later with a minus one percent rating.
So there was this massive trend and that did tell
your story. Likewise, you know parties, Labor's party vote and
that time went from fifty percent down to twenty six percent.
So they definitely do tell a story, but not so
much the individual poll. It's more what's the trend here?
(12:28):
And to a degree though the last twelve months, every
poll has been is pretty close. Most of them have
the government and the lead, but never by more than
two or three seats. Some of them have the opposition
and the lead again by two or three seats. Where
it will probably start to get really interesting is closer
to the election, when people start to tune more into
(12:52):
politics and say, yo, okay, this is how I voted
last time. Am I going to change my mind? The
other thing, you get the poles. So as issues, you
know what you think of the budget? What did you
think of pay equity?
Speaker 2 (13:06):
Yeah, we saw differing results in that one as well, David,
Thank you for that explanation.
Speaker 1 (13:11):
For more from Kerry Wood and Mornings, listen live to
News Talks at B from nine am weekdays, or follow
the podcast on iHeartRadio
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com