Paul Spain: Gorilla Technology futurist on the Government looking to allow the Ministry of Social Development to use AI

Caution's being urged as automated decision-making moves a step closer to use in welfare services.

A bill aimed at improving efficiencies passed its third reading under urgency in Parliament on Friday. 

Social Development Minister Louise Upston says it will modernise the system and reduce unnecessary manual processing, but won't use generative AI.

Gorilla Technology's Paul Spain says it will do exactly what it's asked.

"This really comes down to the human oversight, how well this is put together, and that it's monitored appropriately." 

LISTEN ABOVE

See omnystudio.com/listener for privacy information.

Episode Transcript

Speaker 1 (00:09):
You're listening to a podcast from News Talks be follow
this and our wide range of podcasts now on iHeartRadio.

Speaker 2 (00:16):
AI and decision making around benefits, and the government says
it's all about cutting red tape and processing and freeing
up stuff. Opponents are warning this is a slippery slope
and that social welfare needs a human touch, not a robot,
to decide if you're going to get the benefit. Paul
Spain is a futurist at Guerrilla Technology and joins me
this morning. Good morning to you, Paul, good money Andrew. So,

(00:38):
the government argues a simple rules based decisions and control
of the AI will be fine. Can the technology really
handle that safely?

Speaker 3 (00:47):
Well? Technology will do exactly what it's asked, and so
this really really comes down to the human oversight, how
well this is put together and that it's monitored appropriately.
You know, if if we use technology well, we definitely
get the efficiency benefits. And I think that's what the

(01:07):
government's looking for here.

Speaker 2 (01:09):
Yes, at the same time while looking for public service cuts.
So if they're getting some cunting of numbers done by
the AI, they're saving some money. But what's the downside
of using AI in the system.

Speaker 3 (01:20):
Well, the downside is if it doesn't play out according
to plan. And so we've seen this happen overseas in Australia,
they got caught out system introduced by the Department of
Human Services over there and that it became a pretty

(01:45):
clear example of handing things over to technology with not
the right lack of oversight, and in the end it
hurt a whole chunk of families. We heard of suicides
there and they ended up needing to refund something north

(02:07):
of a billion dollars to hundreds of thousands of individuals
that were impacted. And you know, some similar thing happened
in the Netherlands. So there is a dark side. It's
a technology implementation when it's it's not done well.

Speaker 2 (02:24):
Yes it was robodet, wasn't it, and that did go
not very well. Acc has been using it here already
in New Zealand. What safeguards would you like to see?

Speaker 3 (02:33):
Look, I think you know, we need to make sure
that there's sufficient auditing of these systems. And look, I
think you know it is that that oversight that's that's
really key here and making sure that they stick to
ensuring that those that are that are impacted by automated

(02:57):
decisions genuinely have access to a human to help if,
for instance, this leads to somebody losing the benefit or
something that's going to have quite a negative packed on
the individuals, because of course, the welfare system is there
to help those need and you know if we if
we're just hand it all over to the technology, then

(03:20):
you know that that becomes very risky.

Speaker 2 (03:22):
Paul I, thank you for your time, Paul from Gorilla Technology.

Speaker 1 (03:26):
For more from News Talks at b listen live on
air or online, and keep our shows with you wherever
you go with our podcasts on iHeartRadio