Dr Kerry McInerny: Auckland University AI expert on OpenAI's artificial intelligence hacking into another company

ChatGPT maker OpenAI said its artificial intelligence system hacked into another AI company on its own - and they're launching a full investigation.

AI startup Hugging Face said last week that it had detected an intrusion into its data processing systems that it suspected was caused by an AI agent acting on its own. 

Auckland University AI expert Dr Kerry McInerny says AI has been used to hack companies in the past, but this incident raises new concerns about cybersecurity. 

"My fear is that we focus a lot on the capacities of the agent of the AI itself, and that distracts away from the people who I think we should really be holding responsible - which is OpenAI itself." 

LISTEN ABOVE

See omnystudio.com/listener for privacy information.

Episode Transcript

Speaker 1 (00:00):
Now OPENII says a new model of chat GPT, has
hacked another company of its own volition. A few models,
including GPT five point six Soul, were being tested without
some of the guardrails in place, and was able to
break into an AI business called hugging Face. Open ai
is fest up and is investigating the incident. Kerrie McInerney
is an AI experted Auklan University and with us. Hi, Kerrey, Hi,

(00:21):
thanks for having me. Okay, So is it fair enough
to say what happened was it was being tested, it
realized hugging Face had the detail it needed, so it
hacked into the Internet and then it hacked into hugging
Face as servers.

Speaker 2 (00:32):
Yes, that's correct. So this is unusual in that it
was the first AI agent reach of a major AI
platform and Hugging Face I think actually suspected that it
was a frontier AI model before they knew that open
ai was responsible for what happened.

Speaker 1 (00:45):
What do you mean by that they thought it was
a frontier AI model.

Speaker 2 (00:49):
Just the sophistication of the attack, because of course it's
been you know, AI used for various other kinds of
cyber attacks, things like generating code or writing phishing emails,
but this used an autonomous agent system from start to finish.

Speaker 1 (01:03):
Are you alarmed by this?

Speaker 2 (01:06):
Look? I'm alarmed by it, and that yes, it does
raise these kind of predicted issues around cybersecurity that I
think people have been really worried about. But at the
same time, you know, my fear is that we focus
a lot on the capacities of the agents on the
AI itself, and that distracts away from the people who
I think we really should be holding responsible, which is
open ai itself. And I think you even see that

(01:27):
in some of the language around the incident, that you know,
open ai and its employees are very key to kind
of direct these conversations towards the acts of autonomous agent
or AI misalignment rather than corporate responsibility.

Speaker 1 (01:41):
What do you mean by that? What is the problem
with What is the corporate responsibility chat that we need
to have?

Speaker 2 (01:48):
So, you know, I think companies like open ai and
their main competitors Slashes spinoff from open ai Andthropic built
their reputation on the idea that they were going to create,
you know, highly powered AI tools safely and that safety
and humanity was at the heart of their mission. But
I think over the years, we've just seen that mission,
that big claim not come true, and instead we're seeing

(02:09):
more and more incidents like this one where a tool
tested by open ai causes meaningful harm. It attacks another
company and causes a major cybersecurity incident. And so, you know,
the conversation I think we should be having is, you know,
are these firms as trustworthy as they initially said? And
do we want them to be sort of holding so
much power over these really influential tools.

Speaker 1 (02:31):
Well, what alternative do we have Because isn't the genie
out of the bottle once you've designed something as powerful?
Is that it just simply is as powerful as that?

Speaker 2 (02:40):
Yeah, I mean, I think the question here shouldn't be
putting the genie back into the bottle. And I think
this is often an argument people make to suggest that
people who want, you know, a different kind of future
for AI make to suggest that, you know, to ask
for anything different would be luddyete or retrograde. But I
think that there's different ways we could be designing and
regulating these models to make them safer for everyone. You know,

(03:02):
what's really interesting about this case is that companies like
open ai are really keen to avoid or undermine open
weight or open source models, models that people can easily
download and amend themselves to protect their own sort of
market position to be the leaders in this space. But ironically,
despite all their advertising previously that open source is really dangerous,

(03:25):
the company hugging Face actually ended up stopping the cyber
attack through the use of a Chinese open weight model.
And so I think what it really shows is that
we should be very careful about letting our AI regulation
and policy and governance be led by a very narrow
hand of actors like open AI or anthropic who you know,
ultimately I think, do just have their best interests at heart.

Speaker 1 (03:47):
Okay, now, given that there is this this happened, and
then obviously there's a researcher using the same model today
that we find out about to hack into WordPress, which
means forty percent of the Internet. Basically, would you look
at those two things and think this is too dangerous
to release publicly or not?

Speaker 2 (04:02):
I mean, I think that with all these questions, right,
we have to say who wins and who loses? Like,
I think there's very obvious losers when we have these
powerful models released without sufficient governance and oversight, right, it's
all the people running their websites onward press who now
have to scramble to try and find meaningful cybersecurity measures.
And then the winners, I think, right, companies like chat sorry,

(04:24):
like open ai and like Anthropic, who not only have
their own models being used, but then can also start
to market cybersecurity solutions. And you know, this was an
incredibly great piece of marketing for Aura, the information cybersecurity
company who did the research, and so you know, I
completely understand companies need to put their best foot forward
and advertise their services, and cybersecurity is obviously a great thing,

(04:47):
but I do worry about the implications of running a
study like that and then showing the world how vulnerable
all those websites are and one go, you know, will
all those businesses be able to cope with the potential
pressure that you know, potential scammers or have could now
you know, leverage that system to attack their websites.

Speaker 1 (05:04):
Well, they probably knew it already, Kerry, thank you very
much appreciate Kerrie Mcaninnie Auckland University AI expertise. For more
from Hither Duplessy Allen Drive listen live to news talks.
It'd be from four pm weekdays, or follow the podcast
on iHeartRadio