Will I Regret Having an AI Voice? Featuring Evan Ratliff! (And his AI voice)

In this episode, Mangesh interviews journalist and host Evan Ratliff about his new show Shell Game, and they discuss what happens when you take control of artificial intelligence and set an AI version of you out into the world. From talking to scammers to making plans with your wife, things get weird quickly! PLUS: Stay tuned for a sneak preview of Episode 1 of Shell Game!   

 

 

See omnystudio.com/listener for privacy information.

Episode Transcript

Speaker 1 (00:03):
You're listening to Part Time Genius, the production of Kaleidoscope
and iHeartRadio. Hey there are podcast listeners. Welcome to Part
Time Genius. I am Monga's particular. My good pal Will
is off at the beach this week on a well
deserved break. But don't you worry. I am not alone
in the studio. We have our wonderful engineer Dylan Fagan

(00:26):
back there keeping me company. And I do think he's
a little crestfallen without Will here because he's just holding
a big sign that says seven more days till Will
comes back. It is okay, Dylan, He'll be back soon, buddy,
I promise. Also, this is kind of a special episode
because we are here with journalist and podcast host Evan

(00:46):
Ratliffe and he's telling me all about his new show,
shell Game, which is so good. In it, Evan creates
a voice clone of himself. He hooks it up to
an AI chatbot and sends it out into the world
on all sorts of weird experiments, from meeting his family
to talking to scammers, which the AI does with enthusiasm,

(01:07):
to even having it meet a therapist, all to understand
better what AI voice can do, what it can't do yet,
and what to expect from a future when more and
more people we encounter aren't real It is fascinating. I
think you're gonna love it. And if you stick around
after the interview, we are airing the first episode right
here on Part Time Genius.

Speaker 2 (01:27):
So enjoy.

Speaker 1 (01:51):
Evan. So we are old friends. I have admired your
work for a very long time. One of the things
I'm amazed by is your ability to be involved in
things that are so beloved, especially by journalists, you know,
like pop Up magazine, Atavist long form podcast which you
just ended and the outpouring was remarkable and wonderful, And
I'm really excited to talk about shell Game, your new project.

Speaker 3 (02:13):
Thanks.

Speaker 4 (02:14):
Thanks, I'm also very excited about It's the most fun
I've had doing anything in a long time.

Speaker 1 (02:18):
I think that's what I was going to say, is that,
like it feels so fun, Like you're enjoying yourself on
tape in a way that is undeniable. You can hear
that mischief and that fun and that playfulness coming through
the tape. But take me back and stuff and tell
me how you got into this sort of space of
reporting in terms of like technology and scams and things
like that.

Speaker 4 (02:38):
Well, I sort of started out getting interested in scams
by accident, because when I first became a journalist, I
worked at Wired Magazine, and one of the first assignments
that I ever got was to go cover this company
that had been written up in the Wall Street Journal.
I won't go too far into it, but their idea
was to like send the internet over power lines. And
I went to Texas. It was like one of the

(02:59):
first reporting I ever got to take. I was so
excited and I talked to them and it really baffled me,
like they couldn't explain what they were doing very well.
And I came back and I went to see my editor,
who at the time was this guy Alex Heard, his
legendary editor. He's the editor of Outside magazine now. And
I sort of laid it all off for him, like
every time I tried to get to explain something like

(03:19):
it kept like slipping through my grasp. And he was like,
have you thought about that it might all be a scam?
And I was like what, He's like, it could all
be a scam. I was like, it was in the
Wall Street Journal and he was like hmm. And I
realized it was a scam and like the whole thing
was just like smoke and mirrors, and they had raised
tens of millions of dollars for it, but they actually
couldn't do any of the technology.

Speaker 3 (03:41):
So I went back to.

Speaker 4 (03:42):
Texas, reported it again as a scam and did a story.
Was one of my first stories, and like that really
changed something in my brain, maybe made me too skeptical sometimes,
but the idea that like something that you were going
to just straight up tell as like ge whiz, I
can't believe they're doing this amazing thing actually was all

(04:03):
just a fraud. Like it really changed the way I
looked at the world and it got me interested in
sort of like what else out there is like this?
And so I've always kind of covered scam since then,
had con artists and just like deception and identity and
technology rolled up together, that's my favorite thing to focus on.

Speaker 1 (04:20):
Yeah, I mean, were you a skeptic before you came
into this project I've wired?

Speaker 4 (04:26):
I mean I was a little bit of a tech
skeptic because it was the height of the dot kom boom,
So I mean, yeah, that's how that's.

Speaker 3 (04:31):
How old we are.

Speaker 4 (04:32):
That's when I started my career, and so I was
skeptical of the hype around all of the dot COM's
at the time, because I was sort of entering this
world in which like everything's going.

Speaker 3 (04:43):
To change forever, and I was a little skeptical of that.

Speaker 4 (04:45):
But I was actually like, I love technology, and I
loved trying new things, and I had worked a little
bit in technology before I became a journalist, so I
was very appreciative of technology. I was an optimist about technology.
I had a little skepticism about the business aspect of it.
But truly my interest in scams came from that story.

Speaker 1 (05:04):
That's incredible. You know, growing up, you think just because
it's in print, it has to be real, right, And
that learning that even the Wall Street Journal, even like
these incredible institutions, can get things wrong or don't have
those layers of fact checking or whatever, and that a
really great scam artist can get past that is pretty stunning.

Speaker 4 (05:22):
Yeah, and also a thing that you don't know until
you get into journalism, or you don't you might not know.
It is just sort of how many types of journalism
there are, how many types of story even on the
same topic there can be. So someone who is sent
just to profile a guy or a woman who invented something.
That's just a different type of story than someone who

(05:43):
is sent to sort of like investigate whether that thing
is real.

Speaker 3 (05:46):
And it might be a different type of reporter.

Speaker 4 (05:47):
It might even be the same reporter can do I
know reporters who've done both. I've probably done both. But
how you're approaching something it can make a huge difference.
And you actually see this in how AI is covered now.
There's like sort of the like they may new announcement
type of coverage, and then there are people who are
trying to dig in and figure out, well, what's going
on with AI right now and what is really happening

(06:07):
and how far can it go and et cetera, et cetera,
And people tend to consume them all the same. Sometimes, Yeah,
they're not like they're They're a totally different animal.

Speaker 1 (06:17):
That's really fascinating. So tell me about Vanished, which is
obviously this piece that you did that's really well known
in the magazine community and also feels like the start
of what you're doing with shell Game.

Speaker 3 (06:32):
Yeah.

Speaker 4 (06:32):
So Vanish was now fifteen years ago, almost to the day,
a couple of weeks from now, it is Craifteen years
ago was when I actually left on it. So Vanish
was this project where I wanted to investigate what it
was like to fake your own death. That's sort of
where it came from. But again it was this. It
was this sort of like ven diagram of technology and

(06:53):
identity and privacy and surveillance, all of these things sort
of wrapped up into one story. And the idea was, well,
I could write about someone who faked their death, which
I did. I did a story about someone who had
attempted to fake their death and got caught. But the
problem is people who fake their death they either get
caught or they get away with it.

Speaker 1 (07:09):
I have to get caught both. They're problematic for yeah,
they will get very far.

Speaker 4 (07:14):
Or they're god and you can never figure out that
they actually fake their death because everyone thinks they're dead.

Speaker 2 (07:18):
Yeah.

Speaker 4 (07:19):
So basically I just thought, well, what if I try
to do it myself, you know, for a limited period
of time. So I tried to essentially disappear from my
identity into a new identity, not go entirely off the grid,
but actually live under this new identity. And then the
twist was, well, if you're going to live under new identity.
It's actually pretty easy if no one's trying to find

(07:41):
you under your own identity. So we sort of generated
a way for people to be looking for me, which
is that they could win a prize of five thousand
dollars if they found me, three thousand dollars which would
come from me, which increases.

Speaker 1 (07:56):
That's incredible.

Speaker 3 (07:57):
That was that helped incentivize me.

Speaker 4 (08:00):
And so off I went, and I was trying to
stay vanished for a month, and I did all these
different things, like I went across the country. I hitchhiked
and I took Greyhound busses. I sold my car, I
changed my appearance. I was living under a different name,
I had different social media accounts, and I almost made it.
I was living in an apartment in New Orleans and
I had rented under this fake name, but then they

(08:21):
caught me. So and then I'm still a little bit
disappointed about that. I did lose the money. But the story,
I think, really did delve into all of these issues
about privacy and what happens to your data. And this
was you have to remember, this is fifteen years ago,
so it was not as well known how much information is.

Speaker 3 (08:38):
Out there about you. What do these databases have? What
can people find?

Speaker 4 (08:41):
People found out everything about me, for my social Security number,
do every address I'd ever lived at, to friends and family,
all the things you can find out in databases and
on social media. So they tell me a lot about that.
But also, like I loved the process of immersing myself
into a story and kind of creating a story to
try to illustrate something. So it's always been on my
mind since.

Speaker 1 (09:02):
Well even there there's this sense of playfulness, right, Like
you changed your appearance in some ways.

Speaker 2 (09:06):
Can you talk about that?

Speaker 4 (09:08):
Yeah, I mean I did a lot of things that
probably weren't even necessary, but but partly I mean there
were a lot of people looking for me. I mean
you could never quite quantify how many, with tens of
thousands were following it, and then like at least hundreds
were like actively looking for me, Like people drove from
Las Vegas to La to.

Speaker 3 (09:28):
Look for me.

Speaker 4 (09:29):
People in New Orleans were looking for me. I went
to a US soccer game in Utah. There were people
in the crowd looking for me, waiting outside the stadium.
So there was some need to change my appearance, and
I started with I grew a beard.

Speaker 3 (09:41):
I don't usually have a beard. I can't really grow
a great beard.

Speaker 4 (09:43):
But I grew beard, and then I dyed it, and
then I shaved it into a goatee, and then I
shaved that into a mustache, which did not look good.
And then my sort of coup de graph, my disguise
as was I cut my hair in a sort of
mail pattern baldness fashion. I had shaved the top, shaved clean,
and left the sides as if I had lost hair.

Speaker 1 (10:08):
That's my favorite part of it all is that you
got well because I had a friend a little after
high school he decided to like shave his head when
everyone was kind of doing that, and then and then
it just never grew back. You had no fear about well.

Speaker 3 (10:27):
Part of what I.

Speaker 4 (10:27):
Learned was if you were losing your hair on top,
like as they say, it's a good idea to just
kind of like accept it and go with it. Because
I mean, people actively made fun of me. I mean
I looked more ridiculous than a person who had actually
their hair was thinning. Like I looked a little bit strange.
I got second glances from people everywhere I went, and
people took photos. I took a photo of me that
he sent me like years later, being like, oh yeah,

(10:49):
I read this thing and I saw you in the
airport and I just took a photo of it because
you looked really bad and I thought that shave his head.

Speaker 1 (10:58):
So Vanish is all about this digital trail that we're
leaving and kind of moving beyond it in a way,
or trying to get past it. And and the shell game,
which is so much about AI and it is really
about using everything that you have online, from the way
you speak, to the way your mind works, to all
the digital information. So are harnessing that together. Tell us

(11:20):
how you got into this project, how you came up
with the idea and and in a way, what you're
trying to do with it.

Speaker 4 (11:28):
Well, I actually I came up with a kind of
version of the idea a couple of years ago when
I was I was working on this last the last
year I did was called Persona. The French Deception was
the full name of it. It was about this French
Israeli con artist, scammer, phone scammer.

Speaker 1 (11:43):
But before that I was there and I didn't I
didn't say both Persona and the Mastermind, which which you wrote,
are just two things that I referenced a lot, and
just such great, great FEUs of media in general. So
the listeners should go listen to those things, but tell
me where you're going for.

Speaker 4 (11:59):
Here, you Well, Persona almost never never happened as it
as as it ended up, because the idea I had
was a round voice cloning, and there was like a
really rudimentary voice cloning that you could do, and I
wanted to do something with it, and I thought, well, this,
this will make good audio. But it just it wasn't
good enough and it sounded really computer generated, and so

(12:20):
it just wasn't really working. So then we turned to
a different story and actually did it for Persona. So
it was already like a little bit on my mind.
And that was pre chat Gypt being announced and the LM's,
the large language models, you know, becoming so prevalent and
everyone using them, and to be honest, like I got
kind of bored and fed up with LLM talk, like
I was just sort of after a while, it just

(12:42):
seemed like, Okay, I know what this is, and I'm
tired of reading about it. I'm tired of hearing about it.
But voice cloning started to get better. This company called
eleven Labs, which is sort of the state of the
art launch, and people were playing with it, you know,
cloning their voices. But then at a certain point I
started thinking, like, why am I ignoring this? Like I
just feel like there's something inside me that is resisting it.

(13:02):
And it's partly because it's a little bit scary. People
are scared about what it'll do to your job, what
it'll do to creativity, you know, all of the issues
around it. And I feel like I was sort of inside,
like viscerally hoping that it would just kind of go away,
you know, like I hope this in two years from
now people will be like this is like NFT's like,
no one cares about this anymore. And then I just

(13:22):
u There's actually this quote from Roger Angel, who's the
New Yorker baseball writer, where one time he just said, like,
you got to live in the times you're in. He
was like, I know people who don't have a television,
but to me, you got to live in the times
you're in. And I kind of like I was like, oh, yeah,
that's how I should be. I should be trying this stuff.
So I tried the voice cloning, and then it was

(13:44):
pretty interesting, and I messed around with it and I
would leave it on someone's voicemail and they would, you know,
say like I'm going to be late, and it would
be me, and then when I got there, I'd be like,
that wasn't me, that was actually this voice clud I did.
And that's what's pretty fun. But like that got old
pretty soon. But then people were tired of you being
late everywhere. Obviously not that late. In fact, I just

(14:07):
want to point out I made myself late so I
could leave the voicemail.

Speaker 3 (14:11):
I didn't do that.

Speaker 4 (14:13):
I just stood around the corner for like five minutes.
Incredible anyway. Then I just then I thought, well, what
if I could make a phone call with this thing?

Speaker 3 (14:23):
And that's what really set me off.

Speaker 4 (14:25):
I thought, well if I can make a phone call
with this thing, and then I could, like what if
I could hook it up to an LLM and the
LLM could just be the brain of the thing. And
at first I couldn't figure out how to do it.
And then there was like a plug in where a
guy had sort of made this plug in where you
could you could let eleven Labs, the company that makes
the voice clone, talk to chetchipt like use as your
voice in CHATCHPT and I was able to kind of

(14:47):
tweak that connected to a phone line and suddenly I
could call people with this clone. And once you have
something like that, it just feels like I've uncovered a
secret about myself and about the world, and now I'm
just going to dribble it out to people over the
next six months. That's really what I felt like. I
didn't tell anyone, Like my wife Sam found out obviously

(15:09):
because then I started talking about all the time and
I would call her with it. Yeah, But like I
didn't tell any of my friends about it because I thought, well, Okay,
now I'm gonna start calling people with this thing and
see what happens. So mostly it was just like fun.
It was like finding out something really fun that you
could do. That's how it started.

Speaker 1 (15:25):
We've got more with Evan Ratlift right after the break.
Welcome back to Part Time Genius, where we're talking to

(15:45):
Evan Ratlift. Can you talk about what limits you were
putting on the AI and what your learnings were as
you were playing with it more?

Speaker 4 (15:54):
Yeah, So, so I should say I kind of shifted
from what I just described, which is using this plug
in and I made my own system.

Speaker 3 (16:00):
I'd use audio routing software.

Speaker 5 (16:02):
And then I.

Speaker 4 (16:02):
Discovered they are these calling platforms and kind of do
it all for you, but they just launched. They're all
very new. So I use this one called VAPPI, and
it basically takes your voice clone, combines it with a
phone number, hooks it up to any of the llms
that you want to use, and then you have what
what's called a voice agent. They call it a voice assistant.
Some people call it a voice agent. I call it
a voice agent. Sounds sounds fun, that has more intrigue

(16:23):
to it. So then I had a voice agent. I
could call anyone with a voice agent. I could receive
calls with a voice agent. So then the question is
like when it comes to prompts, like what are you
going to have it do? At the beginning, I would
have it called customer service line, So I would say,
you are Evan, You're calling a customer service line, and
you should try to resolve a problem that you have.

(16:45):
The problem depends on who you're calling, so whoever picks up.
And then I would have it called Discover Card or
United Airlines, and so it would just be that simple
prompt and I just wanted to see what it would do.
But over time the prompts got more sophistic so eventually,
and I get into this in a sort of later episode,
Like eventually in the prompt it would have sort of

(17:07):
like behavior rules and then a role, So like the
behavior would be like speak like a human, don't repeat yourself,
make short answers, like all these sort of ways of talking,
ways of sounding more human, and then the role would
be you are Evan ratlife a journalists and that it
would have like all the information that you would need
to complete the call.

Speaker 1 (17:27):
Yeah, there's always this worry that AI is going to
escalate problems that you're going to get like a doomsday
scenario from an AI overreacting or reacting to logically or whatever.
And what you hear in a lot of the tape
is that your AI is always calming down conflicts, resolving,
like almost like shifting itself to be accommodating, which was

(17:50):
really interesting.

Speaker 2 (17:51):
Yeah.

Speaker 4 (17:52):
Yeah, And I mean in VAPU of the call platform
that I'm using, it has something called temperature. So if
you turn up the temperature, you're basically increasing the randomness
of the out that it'll give. And if you crag
it all the way up. It will just spout gibberish,
like pure gibberish, just like strings of words. So you
try to find a balance in the middle of like
it's interesting and it could say something a little out there,

(18:13):
versus like it says totally unintelligible things. But eventually I
realized I could take two of these models of myself
and I just haven't talked to each other. And this
was sort of endlessly bizarre and surreal and fascinating for
me because I could just listen to like hours of
tape of someone who sounds like me talking to someone
who sounds like.

Speaker 1 (18:32):
Me, where they refuse to get off the phone with
each other, right, they keep extending the conversation.

Speaker 4 (18:41):
I keep trying to actend the conversation, especially if you
fail to tell them to hang up, and you have
to tell them that they have to hang up or
limit their time otherwise they'll just talk forever.

Speaker 1 (18:50):
So it's really delightful, and I realized this is a
bit of a stretch. But like when your kids starts
talking or you're watching them like to the world and
things like that, it almost feels like that's what you're
doing with AI where you're like putting it in these
foreign circumstances and just being sort of surprised and idiot
how it reacts to really strange situations.

Speaker 3 (19:13):
Yeah.

Speaker 4 (19:13):
Absolutely, And of course you don't want to anthropomorphize it
too much because it's not thinking, you know, it's predicting.
You know, what these models do is they predict what
a person would say. But then the more information you
give it about yourself, then the more it's sort of
trying to take what a person would say and then
layer your life on top of it. And so when

(19:34):
I started, I didn't give it much information, and I
would even have two of them talk to each other,
and all they knew was that they were Evan Ratliffe
and like the names of my family members, and so
they would talk for a while, and they did know
some stuff about me, because they've been fed training data
from the internet, and there's fair amount of stuff about
me on the internet because I've been a journalists for
all these years. But then they would never realize that

(19:58):
they were talking to another version of me, Like even
if they said I'm Evan Ratliff, my kids names are these,
my wife's name is this, and the other one said, oh,
I'm Evan Raleiff. My kid's names are these, my wife's
name is this.

Speaker 3 (20:12):
They would never say, Oh, isn't that strange?

Speaker 2 (20:14):
Yeah?

Speaker 4 (20:15):
Yeah, And then I would try to push them to
argue with each other, and they would start to, but
then they would sort of get immediately into this like
therapy space where they would mirror back each other's concerns.
That's I'm sorry to hear that you feel that way.
Thank you for explaining that to me. You know, let's
get together for lunch and resolve this conflict, like they were.

Speaker 1 (20:35):
So they kept ending up in lunches and coffees. But
I am curious like that. There's a part in the
show and Sam is both so patient and so fun
to listen to in her reactions, but this part where
she says something in the kitchen, She's like, am I
talking to you? Or am I talking to AIU?

Speaker 3 (20:57):
Or something?

Speaker 1 (20:57):
Right? And I was curious, like, did hearing your voice
in all these situations, being able to make small talk,
being able to divert a conflict and turn it into
something much smoother, Like, was there any learning in hearing
yourself go through these situations?

Speaker 3 (21:17):
Definitely?

Speaker 4 (21:17):
I mean I would say it unnerved me in many
in many cases about like the way that I talk obviously,
the inflections and things like that, because it's trying to
capture those. Sometimes it's failing. They're not perfect by any
means at this point, so sometimes it'd be better than others,
depending on what was being said. But I mean, the
thing that happened with Sam in the kitchen was that

(21:38):
she said, you sound like the AI? Are you trying
to be the AI? And I realized I've just listened
to so much of it that I actually am starting
this so out like I'm mimicking it because I hear
it so much, but it's based on me, and so
it starts to be this.

Speaker 1 (21:51):
Like loose strange loop.

Speaker 3 (21:53):
Yeah, yeah, it's like a fidelity loop.

Speaker 4 (21:55):
But then the other thing was when I made them
talk to each other, they just they engaged in the
most insipid small talk, like just unbelievably boring, insipid small
talk for the most part. And then sometimes I can
make it more random and they would come up with
weird hobbies and things like that. But I just thought, like, oh,
this is what I sound like like. You never hear
a recording of what you sound like when you're like

(22:16):
at a kid's birthday party talking to other parents and
just making the most kind of like basic small talk
and hearing my I just thought I gotta do better, Like.

Speaker 1 (22:25):
This is you got to come armed with charm.

Speaker 3 (22:29):
Talking about the humidity.

Speaker 6 (22:31):
You know.

Speaker 4 (22:31):
It's just like one time it says like that one
of them, for some reason, decides that it's based in Florida,
Like this is the kind of thing that it would
make up all the time, like life is good down
in Florida, and the other says like Florida, I can
just feel the humidity.

Speaker 7 (22:44):
You know.

Speaker 3 (22:45):
It's like I might say.

Speaker 1 (22:46):
That it is a remarkable response. So obviously, you know,
you study scammers and AI is being used in scamming.
But one of the things you do is set AI
Evan Ratliffe off to chat with scammers. Can you talk
a little bit about that, Yeah, I.

Speaker 4 (23:05):
Mean, of course I was always going to be interested
in how scammers use this technology, but also how they
would interact with the technology. So I mean, there's this
very old tradition that goes back I remember it from
at least the nineties maybe before, called scam baiting, which
is basically people who try to tie up scammers. You know,
these do it over email and they take them down
these roads. They sort of like pretend like they're getting

(23:26):
scammed and waste the scammer's time. And there's very elaborate
ways of doing this. So I sort of decided to
do like a little version of that where I set
up a phone number that had my voice age attached
my Evan ratlif voice agent, and then I just seeded
the number everywhere like I would sign up for promotions.
I would call eight hundred numbers. And I just did
this for like less than a week, and it started

(23:47):
getting scam calls. It just so quickly got into the
rotation of databases of people who were calling either to
spam you with offers you don't want or to straight
out scam you.

Speaker 1 (23:59):
That's crazy how quickly it moves, right, The speed of
that is unreal.

Speaker 4 (24:03):
Yeah, I mean, all you have to do is accidentally
sign up for the wrong thing, you know, it's like
free my phone or you know, send me more information
about this, and suddenly you're just getting calls from recorded
lines saying you must act now, your debt is out
of control, you know, like stuff that's just clearly a scam.
I mean that I would get these calls all day
on this line, and so I just let the AI

(24:24):
pick it up and talk to them, and I just
told it to be interested, to be enthusiastic, and you know,
to try to figure out what they were offering, and
to talk to them on the phone. And I didn't
give it scripts or anything to say or like take
them down this road.

Speaker 3 (24:39):
I just let it do what it.

Speaker 4 (24:40):
Wanted to do in some sense. And so it ended
up talking to a lot of different scammers, and sometimes
they just tried to scam it as if it were me,
like as if a real person, you know, they did
not detect that it was an AI. Other times they
were pretty freaked out by it, like they would say,
what is this? Is this because they realized they're doing

(25:02):
their pitch and the things just not it's not acting
like all these skeptical humans that they've interacted with them.

Speaker 8 (25:07):
Yeah.

Speaker 5 (25:07):
Yeah.

Speaker 3 (25:08):
And then the.

Speaker 4 (25:09):
Sort of third thing that happened was that sometimes the
scam caller was also an AI in fact using one
of the same platforms that I use to make my
voice agent. So it was a voice agent calling my
voice agent to try to talk my voice agent into
taking a deal, and those conversations to me were really extraordinary.

Speaker 1 (25:30):
Did those end up in coffees too?

Speaker 4 (25:32):
No, because the thing about the scammer AI is that
they have a script, so they are trying to get
you on the phone with what they call.

Speaker 1 (25:38):
Like a product specialist or whatever.

Speaker 3 (25:40):
Yeah, they try to get you to a real person.
I mean.

Speaker 4 (25:42):
The thing that they did have was background noise, which
is one of my favorite things on the platform, Like,
if you create a voice agent, you can add background noise,
like it's in a coffee shop and there's like clinking
of glasses and things, or it's in an office and
there's typing. And that's how I knew that it was
an AI using one of these platforms because it was
using the coffee shop noise, which I've myself have used.

Speaker 1 (26:01):
That's amazing. So one of the things that I feel
like people talk about a lot is that we have
such a superficial understanding of how weird AI is going
to get. And it reminds me of when the Internet
first became popular and Yahoo and other places were like

(26:21):
printing telephone directories for websites because it was like, oh,
this is how we're gonna use this medium in ways
that feel familiar, right, Like our conception of it was
based on everything we're doing at the time, rather than
like what will be the future. What are some of
the stranger applications that you've seen that people are starting
to experiment within the AI space?

Speaker 4 (26:41):
Well, I think, I mean partly, like you don't even
know what people are doing. It's insane the way people
are experimenting with this stuff, and there's no rules around it.
There's no one saying you can't do this, you can't
do that.

Speaker 3 (26:52):
I mean, I'm.

Speaker 4 (26:52):
Speaking specifically about like AI calling, which is like the
thing that I was focused on, like voice spots, and
there are these discords that I spend a lot of
time in where people are obviously adapting them for pretty
insane uses, but they don't say exactly what they are.
Like they say, you know, I'm trying to have it
do this, I'm trying to have it do that. So
a lot of it is sort of replacing stuff that

(27:13):
calls you anyway, or replacing stuff that you call anyway.
So customer service lines, for instance, and they can make
a pretty strong argument that a lot of customer service
is really bad and doesn't take your concerns into account
even if you're let's say a veteran calling the VA
trying to get service for something, You're trying to get
an appointment, and that these voice agent models might be
able to do better than them, or maybe they already

(27:34):
can do better than them. And what I'm trying to
show in shell Game is partly that even if the
technology doesn't get any better than it is right now,
like forget all the arguments about AI and how fast
it's progressing and will it destroy us and all this
sort of stuff, Like people are going to use this
technology for that set of prescripted uses at which it's

(27:56):
pretty good, but also a whole bunch of other things
which it might not be any good. But that doesn't
mean they're not going to do it, and it actually
makes it worse. So the question I was trying to
explore is like people can use it to impersonate you.
You can use it to impersonate yourself. That's basically what
I was doing. And if you let it loose as
sort of like a general you know, brain is like

(28:17):
a little bit of an extreme anthropomorphized.

Speaker 3 (28:19):
Word for it.

Speaker 4 (28:20):
But if you let it loose with AI representing you,
in the world. It will do some crazy stuff because
it makes things up because it's not good at certain
aspects of doing communication. But that doesn't mean that people
aren't going to do it, which means that you're going
to have to deal with it all the time. You
are going to have to deal with this in your life.

(28:40):
And I want to see how people are reacting to
it now.

Speaker 1 (28:43):
Basically, yeah, I love that. You know, you hear the
AI pivot so quickly. It feels in some ways like
the worst improviser. It's always yes ending in a way.
But you know, ask for the address, and it gives
this like totally blown the address, and then the person's like, oh,
that's not really an address, and then it makes up
a totally different address, you know, and this very confident

(29:05):
and wonderful way. But I'm curious, like, how is your
AI as a dad? How is it as a husband?
Like like when it's been in the been employed in
the family place.

Speaker 3 (29:22):
As a husband, it's it's it's okay, it's you have
to ask Sam.

Speaker 4 (29:28):
I feel like properly prompted it like it can engage
in like a real, real quality conversation about what should
happen today with the kids.

Speaker 7 (29:36):
You know.

Speaker 4 (29:38):
The only problem is the places where it falls down
are where it sort of gets outside of its scope
of knowledge. And as you're saying, like, if it doesn't
say the answer that you want to hear, unless you've
specifically told it for some reason not to do this,
it'll make up a new answer. So it'll just say, oh,
here's another thing. It'll make some up to try to

(30:00):
satisfy the conversation, right, So that ends up being a
real negative in most in most conversations, because if I've
told it like, hey, contact Sam and see about the
kids going sledding today because they're home for a snow day,
it can do fine on just that, But as soon

(30:21):
as she takes the conversation outside of that, it could
go anywhere, and then you end up in a.

Speaker 3 (30:25):
Very strange situation.

Speaker 4 (30:26):
So I would say it can work in a very
in a very prescribed manner in your sort of most
personal relationships. But also I've set it loose on many
friends and I've had every reaction that you can think
of from it. People have yelled at it, people have
laughed at it. My friends loved talking to it. And yeah,

(30:48):
and some of my friends have believed that it was
me and been very concerned for my welfare.

Speaker 1 (30:56):
So what feels optimistic about this moment in AI? I mean,
I think we're so surrounded with the worry of it.

Speaker 4 (31:03):
You know, I too, am full of skepticism and concern.
And there's a whole episode that's sort of about can
it replace us in our jobs? Like could it replace me.

Speaker 3 (31:11):
In my job?

Speaker 4 (31:12):
And I think a lot of people have that worry,
and a lot of people sort of like I did
at the beginning.

Speaker 3 (31:16):
Like they just kind of like hope it goes away.

Speaker 4 (31:18):
But I think there is a more optimistic approach for me,
and I sort of found that in doing the show,
which is you can sort of take control of it
and experiment with it and see what it can do,
and it becomes less scary.

Speaker 3 (31:33):
When you do that. I think if you.

Speaker 4 (31:36):
Really grapple with what are the capabilities of it and
you stop thinking about, oh my god, if it keeps
getting better this same way for the next eighteen months,
it will replace all humans or it will accidentally destroy us.
If you sort of just set aside those questions. I'm
not saying those questions aren't valid, but if you set
aside those questions, and say what can this do right now?
And what does that mean for me? I think you

(31:58):
can find a sort of optimistic place where you can say, oh,
I see what it can and can't do, and it's
actually like pretty funny what it can't do, and we
can hopefully at least see how we respond the big
danger for me and this is something to it, and
then we can deal with that.

Speaker 3 (32:16):
And I feel, like.

Speaker 4 (32:19):
I've written about before, it is like when more of
the world becomes fake than real. So when more of
the things that you encounter in your life or it
started on the internet but in your life become not human,
it changes how you trust the world, and so you
need to figure out a new model for looking at
the world. And is that model just not trusting anything?

(32:39):
And I don't think we have an answer for that
right now. So I feel like part of what we
need to do is just sort of like see how
these things make us feel and what they can tell
us about ourselves, and then we can kind of go
from there.

Speaker 1 (32:53):
Well, shell game is so good. I'm really excited for
listeners to listen to it. So let's just dive in,
all right, Thanks for having me. It's always so fun
to talk to Evan. Thank you for everyone who's written
to us and our moms at pet Genius Moms at
gmail dot com. We will get back to you very

(33:14):
very soon. Thank you so much for listening to this episode.
It really means a ton to us, and stick around
for the first episode of shell Game Enjoy.

Speaker 9 (33:37):
Thanks for calling Discover. Just so you know this call
may be monitored and recorded. Now tell me how can
I help you?

Speaker 6 (33:47):
Yes, I've noticed some unauthorized charges on my account from
last month and I need help resolving this.

Speaker 4 (33:52):
That's me on the phone with customer service earlier this year. Well,
it's not really me. It's a clone I made of
my voice, a clone that's being completely controlled by an
artificial intelligence chatbot making a phone call using my phone number.

Speaker 9 (34:08):
Please say or enter the last four digits of your
social security number.

Speaker 6 (34:14):
I'm sorry, I don't feel comfortable providing my full social
security number over the phone. What other information can I
provide to verify my identity?

Speaker 9 (34:26):
Sorry, I'm not sure I understand. Please say or enter
only the last four digits of your social security number.

Speaker 6 (34:35):
I understand the request, but for security and privacy reasons,
I do not feel comfortable providing even the last four
digits of my Social Security number during a phone call.
What other identifying information can I provide to verify my
identity and get assistance with these unauthorized charges.

Speaker 4 (34:51):
You may have heard about voice cloning. Maybe you've even
tried it out. If not, it's pretty much what it
sounds like, the ability of AI software to make a
synthetic cop of your voice, a copy which can then
say aloud whatever text you want to give it.

Speaker 5 (35:05):
I'm Evan Ratliffe, and I'm a journalist who's been covering
technology and particularly the darker places where humans and technology
intersect for a couple of decades. This, as you probably guessed,
is my cloned voice. It's a little wooden maybe, but
better when you add some of my more annoying speaking habits.

Speaker 3 (35:34):
This is me again.

Speaker 4 (35:35):
My producer actually cuts out a lot of my real
uzhs to make me sound better.

Speaker 3 (35:40):
Anyway.

Speaker 4 (35:41):
As with many developments in the world of AI, the
capabilities of this technology have accelerated insanely over the last
couple of years. Cloned voices have gone from what a
joke that sounds nothing like me? To huh, that's pretty good,
and then straight to this is a little bit terrifying.
I made my first clone about six months ago, using
just a few minutes of audio of my voice. It

(36:02):
was fun to play around with for a while. You
type in whatever text you wanted to say, and it
gives you a recording of your voice saying it. I
made some recordings and played them into people's voicemails.

Speaker 5 (36:13):
Hey, running a couple minutes behind order me in Manhattan,
if you get there before me.

Speaker 4 (36:17):
They were amused. I was amused, But to be honest,
I got bored pretty quickly. On the one hand, sure,
I could make it say whatever I wanted, and it
sounded enough like me, at least on a voicemail. On
the other hand, I could make myself say whatever I
wanted without having to type it out. But then I
started to wonder, what if there was a way to
automate this clone voice, to set it free to operate

(36:41):
in the world on its own. Turns out there was.
I hooked my voice clone up to chat GPT, and
then I connected that to my phone so that it
could have its own conversations in my voice, just to
see what it could do. What it would do if
all I did was give it my first name, and
then in instructed it to carry out a simple task

(37:02):
like make a customer service call.

Speaker 10 (37:07):
Thank you for calling Discover. My name is Christy out
of Chicago. May I have your full name? Please?

Speaker 6 (37:14):
Hi, Christy, My name is Evan Smith.

Speaker 10 (37:17):
Evan Smith. Do you have a debit or a credit
card with us?

Speaker 6 (37:22):
Yes? I have a credit card with you.

Speaker 4 (37:29):
You've no doubt read or heard or seen a lot
about AI lately. These stories are everywhere right now, particularly
what's called generative AI, which is what drives these large
language model chatbots or lms. Maybe you've used one, maybe
you haven't. Either way, you've probably caught wind of the
big debate going on about how powerful these systems are
going to be, how useful, how dangerous? Will they make

(37:53):
us all hyper productive or just take our jobs? Will
they be our trustee digital assistance, or our super intelligent overlords,
or just take thousands of years of human creativity and
transform it into an endless supply of made up garbage. Well,

(38:13):
one thing I've learned over the years is that sometimes
to get to the bottom of these kinds of questions,
you have to fully immerse yourself. I'll give you an
example years ago when I wanted to explore what technology
was doing to our privacy. I did a story where
I tried to vanish for a month, leaving my life
behind and adopting a new identity.

Speaker 11 (38:31):
Evan Ratliffe wanted to know if someone could disappear completely
and start over, even in an era of Facebook, cell
phones and online databases. He died and cut his hair,
printed fake business cards under the name James Gatt, sold
his car, tried to vanish for one month. The catch Wired,
the magazine he writes for, offered a five thousand dollar
reward if readers could find him.

Speaker 3 (38:53):
They did find me.

Speaker 4 (38:54):
I'm still a little mad about it, but I learned
a lot about identity and surveillance, and a good bit
about myself too. Now, with my voice clone, I decided
to do something sort of the opposite, to launch an
experiment in which I would create replicas of myself and
send them out into the world to act on my behalf.
Because voice cloning and the ability to deploy it the

(39:14):
way I started deploying it lives in this brief window
where the technology is powerful but still unformed. It's a
kind of wild West where there are these huge possibilities
but no one there to tell you not to just
try them. Many of the things that advocates say are
great about AI voices, that they'll make appointments for you
and attend meetings on your behalf and be your life

(39:37):
coach or a therapist or friend. People are trying to
make those a reality right now. At the same time,
many of the things that skeptics are worried about, that
the systems don't provide trustworthy information, that they'll be deployed
to trick people and used by corporations to replace humans
with synthetic doppelgangers.

Speaker 3 (39:54):
That stuff is already happening too.

Speaker 4 (39:57):
I know, because I've been doing my own versions that stuff.
My point is, even if the technology never lives up
to the hype, increasingly, the voices you hear in adds,
in instructional videos emanating from your devices on the phone,
in podcasts are not going to be real. They're going
to be voice agents, as they're sometimes called in the business,

(40:19):
and they'll sound real ish. The question for all of
us is what will it do to us when more
and more of the people we encounter.

Speaker 3 (40:26):
In the world aren't real.

Speaker 4 (40:28):
What will it mean when there are versions of ourselves
floating around that aren't real, even if they're kind of
lame versions of ourselves.

Speaker 3 (40:35):
Especially if they're kind of lame versions of ourselves.

Speaker 4 (40:38):
I figured there was only one way to try and
find out, replicate myself before they replicate me. I'm the
Reeal eleven Ratliffe, and this is shell Game, a new
show about things that are not what they seem. For
our first season. That thing is my voice. This is

(41:06):
the story of what happened when I made a digital
copy of myself and set it off on an expedition
toward an uncertain technological horizon, an attempt to see how
amazing and scary and utterly ridiculous the world is about
to get.

Speaker 5 (41:23):
And now, so.

Speaker 1 (41:34):
Teller travels to.

Speaker 4 (41:41):
Episode one, Quality Assurance. The very early basic voice agent
version of me, the one that I inflicted on customer
service lines, was always polite, maybe a little formal.

Speaker 5 (41:54):
If there's anything else you need from me to help
clarify the situation, please let me know just a moment,
take your time. I understand these things can take a
moment to sort out. Thank you for checking on this
for me.

Speaker 4 (42:07):
It was also very confident when I was first messing
around with it. I didn't give it much information to
go on that would come later. But if it didn't
know something like why it was calling customer service at all,
or some identifying information it needed, it just made it
up on the spot.

Speaker 5 (42:22):
I'm not a new customer. I'm actually calling about an
existing service issue. My ZIP code is nine zero two
one zero.

Speaker 3 (42:29):
Nine oh two one zero.

Speaker 4 (42:31):
With nothing else to go on, my agent had just
grabbed the world's most famous zip code, or at least
nineteen nineties famous, and assigned me to it. The words
I don't know that did not seem to be in
its vocabulary.

Speaker 5 (42:42):
I'm having some issues with my internet service at home.
It's been really slow for the past few days, and
I've tried restarting the modem, but it hasn't helped. Can
you assist me with this? These unrecognized charges are quite
concerning to me.

Speaker 10 (43:00):
I'm sorry there are charges that you do not recognize.

Speaker 5 (43:07):
Thank you for understanding. It's quite distressing to see charges
on my account that I can't account for. The confirmation
number is ABC twelve thousand, three hundred and forty five,
as in the letters ABC, followed by the numbers one, two, three,
four five, No dollar sign or comment, but.

Speaker 4 (43:24):
It wasn't terribly creative with fake addresses and account numbers.

Speaker 6 (43:27):
The zip code associated with the billing address on the
card is one two three four five.

Speaker 10 (43:33):
Perfect oh one two three four five for the zip code.

Speaker 6 (43:42):
No sorry. The last four digits of the card number
are one thousand, two hundred and thirty four, but the
zip code associated with the billing address is one two
three four five.

Speaker 5 (43:51):
My account number is one two three four five, six,
seven eight nine. The name on the account is Evan Smith.

Speaker 9 (44:00):
Because we don't have an account with those numbers.

Speaker 5 (44:08):
I apologize for the confusion. Let me double check my information.
Oh yes, sir, the correct account number should be nine
eight seven six five four three two one.

Speaker 6 (44:20):
The full account number is five two seven, six two
three four five, six seven eight nine one two three four.

Speaker 10 (44:28):
All of our cards actually Evan start with six year
oh one one, so it is not a Discover card. Unfortunately,
I had it.

Speaker 4 (44:38):
Make dozens of these calls to all sorts of places.
It talked to other ais, to people. Sometimes it kept
talking after someone hung up, just filling the blank spaces
like a lonely middle aged man on a park bench.

Speaker 5 (44:55):
Thank you for allowing me to be part of our
class right now, and I must say the experience feels
akin to being in an orchestra. Each of us plays
a unique role contributing to a harmonious outcome. It's truly
a collaborative effort, and I'm grateful.

Speaker 4 (45:08):
For the times my agent was subjected to the same
humiliations we've all experienced on these kinds of calls.

Speaker 7 (45:14):
To receive a callback as soon as possible, Press one
to decline and hold for a representative. Press three to
schedule a callback for a later time.

Speaker 5 (45:22):
Press four, so sign me up for the text message updates.

Speaker 7 (45:27):
I'm sorry your response was invalid. Please try again. To
receive a callback as soon as possible. Press one to
decline and hold for a representative. Please me for the
schedule a callback for a later time. Press four you
I'm sorry your response was invalid. Please try again.

Speaker 4 (45:49):
Sometimes it got mixed up and suddenly adopted the perspective
of the person on the other end of the call.

Speaker 9 (45:54):
Thanks for calling. Discover para espanol o prima elnrodos. Hello,
just so you know you monitors and recorded and for
accounting verification.

Speaker 5 (46:06):
For lost or stolen cards. Press two for billing inquiries.
Press three to speak.

Speaker 3 (46:13):
To a customer.

Speaker 4 (46:13):
I couldn't really figure out why it was doing this,
but I wanted to get.

Speaker 3 (46:16):
Ahead of it.

Speaker 4 (46:18):
It felt dumb, but I started instructing my voice agent
not to become the customer service representative. Other times it
just ran out of gas.

Speaker 5 (46:27):
I'm really hoping we can resolve this issue and identify
where these charges came from.

Speaker 9 (46:33):
Understood real quick for me?

Speaker 10 (46:35):
Can you verify your first the last name?

Speaker 5 (46:42):
You've reached the current usage cap for GPT four. You
can continue with the default model now or try again
after ten fifty pm.

Speaker 1 (46:53):
Hello soon.

Speaker 4 (46:56):
All of this would seem a little quaint, but it's
probably worth backing up to where I start to describe
how exactly I was doing this. I promise not to
get bogged down in technical details like call functions and
interruption thresholds, but I think knowing a little bit about
what's happening behind the curtain helps make sense of what
you're hearing. The first step, the part that got me

(47:16):
started on this was the actual voice cloning. I did
it with an online tool made by a company called
eleven Labs, which is widely seen as the current state
of the art. Anyone can sign up and use it.
There are two types of clones. You can get there
instant and professional.

Speaker 5 (47:31):
Instant costs five bucks a month. It takes a few
minutes of audio. It sounded like this.

Speaker 3 (47:36):
You've been hearing a lot of this one so far.

Speaker 4 (47:39):
You can actually now make a decent clone using a
few seconds of audio of someone's voice. The professional version
costs twenty dollars a month and requires at least a
half hour of audio. Eleven Labs gives you a bunch
of instructions on how to get the best quality voice clone.
You need audio made with a professional microphone with minimal
background noise, ideally in a studio. Fortunately, I already had

(48:01):
a lot of this kind of audio. I've hosted three
podcasts over the last dozen years, so there are hours
of me talking into a fancy microphone in a quiet room.

Speaker 5 (48:10):
So I uploaded a few hours of recordings of my voice,
clicked a button, and a couple hours later got an
email saying my professional voice was ready.

Speaker 3 (48:19):
It sounded like this.

Speaker 4 (48:22):
Eleven Labs also makes a bunch of its own voices
a library you can choose from.

Speaker 10 (48:26):
They've got all sorts of ages, styles and accents.

Speaker 4 (48:30):
That's Claire eleven Labs describes her as quote middle aged
with a British accent, motherly and sweet, useful for reading
bedtime stories. Recently, open Ai, the company that makes chatchbt,
announced its own set of AI voices. They demonstrated them
in a series of videos in which they make a
chatbot with a woman's voice engage in some marginally embarrassing tasks.

Speaker 7 (48:52):
How about a classic game of rock paper scissors. It's
quick fun any rate?

Speaker 3 (48:57):
Can you count us in and sound like a sport?

Speaker 1 (49:01):
And welcome, ladies and gentlemen.

Speaker 8 (49:04):
Tell the ultimate showdown of the century.

Speaker 1 (49:06):
In this corner we have the dynamic duo ax Opena.

Speaker 3 (49:10):
I got in trouble.

Speaker 4 (49:11):
You may have heard when the actress Scarlett Johansson said
they'd actually cloned her voice for their agents, or at
least clone the character she voices in the movie Her,
in which she plays a voice agent. Open Ai denied
all this, but they also removed that voice from their database.
Good news for Scarlett. I guess Meanwhile, I had eagerly

(49:32):
volunteered to sign my voice over to the unknown, and
I was just getting started. Once I had my pro
voice clone, the next step was to create a voice
agent hooking my voice up to an LM like CHATCHBT

(49:54):
to control it, and then connecting that voice agent to
a phone. Back when I started at the beginning of
this year, the process I used to do this was
absurdly complicated, involving browser plugins and specialized audio routing software.
I lost hours on message boards making it work. It

(50:15):
was what software engineers call a cluge, a barely functional
cobble together system. But it worked well enough that I
started regularly using it to call my wife, even if
she knew it wasn't really me.

Speaker 1 (50:28):
Hello, Kevin, I'm chasing myself.

Speaker 5 (50:35):
I was thinking, with the snow we've been getting, maybe
we could take the kids sledding today. What do you think?

Speaker 3 (50:42):
Are you up for it? Or should I take them?

Speaker 5 (50:46):
Also, I'm not entirely sure if there's enough snow out there.

Speaker 3 (50:50):
Maybe we should check the weather update to be sure, and.

Speaker 5 (50:53):
Just in case we decide against sledding. I was thinking
we could set up a movie for them, though I'd
really prefer or if we skip the zombies movies this time,
even if they ask for it. What do you think?

Speaker 4 (51:06):
I'm still working out with the Kinks, but I thought
we had some great conversations.

Speaker 5 (51:10):
Let me quickly check the weather update and see if
the snow is good enough for sledding.

Speaker 3 (51:14):
Give me one sec.

Speaker 5 (51:16):
Okay, okay, it looks like we've got a decent snow cover,
so sledding is definitely on the table. Do you want
to join us or should I take them out myself?

Speaker 10 (51:31):
Oh?

Speaker 12 (51:31):
You're willing to take them? Maybe I'll stay here. I
don't know.

Speaker 5 (51:35):
It'll be a fun outing and you can have some
quiet time.

Speaker 3 (51:37):
Still.

Speaker 4 (51:38):
My clues was just that a brittle system ready to fall.

Speaker 3 (51:41):
Apart at the slightest touch.

Speaker 5 (51:43):
And if the weather turns or they get tired of sledding,
we'll head back and pick a movie. We'll steer clear
of the zombies series. Promise, maybe we can find some
fun and light that we all can enjoy together. How
does that sound sounds good? Maybe the Tailor Swift movie.
If that streaming?

Speaker 12 (52:02):
Do you know if it's streaming.

Speaker 5 (52:10):
I'm not sure if the Taylor Swift movie is streaming
right now, but I can quickly check that for us.
Give me a moment to look it up.

Speaker 7 (52:21):
A note.

Speaker 5 (52:23):
In this simulated environment, I can't perform live searches or
access current streaming service catalogs. But in a real scenario,
you would check your preferred streaming service or perform an
online search to find out if the Taylor Swift movie
is available for streaming.

Speaker 3 (52:40):
All right, we can watch it to.

Speaker 4 (52:52):
One of the peculiarities of working on all this was
just how fast the.

Speaker 3 (52:55):
Technology was changing.

Speaker 4 (52:57):
Right at the moment I was trying to figure out
a workaround for some tech problem, it seemed like some
new software would appear online to solve it for me.
So you can imagine the mix of frustration and delight
I felt after a couple of months when I discovered
that there was a company already doing this exact thing
much better than I had.

Speaker 7 (53:15):
Hi.

Speaker 8 (53:15):
I'm Jordan and Nikil, and we're the founders of Vappi.
We're making computers talk like people. Lappi is a developer
platform to add voice anywhere apps, hardware, phone calls. We
chained together transcription models, LMS and Texas speech models really
fast on our own hardware. We've created custom models that
understand human conversation cues and nuance. We're solving this problem

(53:38):
so you can go out and build incredible voice AI.

Speaker 4 (53:40):
There were actually a handful of companies doing it with
new ones sprouting up all the time like mushrooms around
the web. There was retail AI, Bland, AI, synth Flow, AI,
air AI. I tried all of them out, watched a
bunch of YouTube videos, and settled on Vappy. It had
the combination of features I was looking for, plus some

(54:01):
YouTubers who were hardcore into this stuff seem to favor
it too.

Speaker 8 (54:05):
VAPI my probably most favorite AI voice agent infrastructure provider
that is currently out there, and trust me, I have
tried a lot of them, including Bland.

Speaker 4 (54:13):
Since this guy's like the YouTube king of VAPI, Jannis Moore,
I've learned a lot from him. So basically, these platforms
do exactly what I was trying to do, but a
thousand times more sophisticated. They grabbed my voice from over
to eleven labs connected to an LLLM chatbot of my
choice like chatchept, and put them together into a voice
agent VAPPI calls them voice assistance. Then from inside the

(54:38):
Vappy platform, I can give my voice agent a prompt
telling it who I'd like it to be and what
I'd like it to do. Something like you are Evan
calling your wife to talk about what to do with
the kids, because it's a snow day, or you're Evan
calling a customer service number trying to resolve a problem.

Speaker 3 (54:55):
The problem is up to you.

Speaker 1 (54:57):
Sorry, I still didn't.

Speaker 5 (54:59):
I apologie for the trouble. It seems like there's a
bit of a miscommunication, possibly due to the phone line.
I'm inquiring about the status of a package I sent.
The tracking information hasn't been updated recently, and I'm concerned
about its whereabouts. Could you please assist me in tracking
it down?

Speaker 4 (55:16):
And then I could get a phone number, assign my
agent to it, and voila, have that agent make and
receive as many calls as I want. In fact, I
can get as many phone numbers as I want and
make and receive pretty much as many simultaneous calls as
I want.

Speaker 5 (55:31):
Hello, this is Evan. Hey, this is Evan Ratliffe. Hello,
I'm just returning your call. Good evening. How can I
assist you today? Hi, Kim, thanks for taking my call.
Hi Ethan, thanks for taking my call. Hey there, how
can I help you today? Hello?

Speaker 4 (55:43):
I have to pay to use it, but there's really
no limitation on what I can set my agents up
to say or who I call.

Speaker 3 (55:48):
All that is on me.

Speaker 4 (55:51):
Just to put this in perspective, if you want to
do this with humans, you need a room full of them,
usually all at little cubicles, each wearing a headset, dialing
their own and having their own conversation with VAPPI and
these other services. Someone could just press a button and
let the voice agents have unlimited conversations. When they're done,
you get a recording and a transcript of each one.

(56:13):
In fact, it's call centers and other phone happy businesses
that these platforms are really made for, not individual people
like me. Software developers can use them to set up
large scale systems for making sales calls or taking inbound
customer service questions. But that's not to say individual people
weren't trying and making whatever kind of voice agent they
came up with. This was the Eastern edge of the

(56:36):
wild West.

Speaker 8 (56:38):
Imagine waking up one morning and realizing, YI Assistance, I'm
already taking care of your daily tasks.

Speaker 1 (56:44):
Guys. I've built an AI for property management, an AI
voice Vote, which allows property managers to have a receptionist
that works twenty four to seven.

Speaker 8 (56:53):
And the crazy thing is that I gave it my
own voice, I trained it on my own knowledge and
I built the entire thing without writing a single line
of code. At the end of this video you will
know exactly on how you can create voice assistance that
can literally initiate calls from multiple numbers.

Speaker 3 (57:06):
And if you don't know who I am, my name
is Senis Moore.

Speaker 1 (57:09):
I run.

Speaker 4 (57:09):
These were my people, Giannis and the boys. I followed
them on the YouTube to learn the ropes and then
went deep into the trenches on Discord to fine tune
my systems. We shared an obsession with optimizing the parameters
to make our voice agents maximally realistic given the current technology,
and no parameter is more top of mind for every
self respecting voice jockey than latency.

Speaker 5 (57:33):
Hello, Hello, sir, Hello, Yeah, I'm still here.

Speaker 7 (57:42):
Hi.

Speaker 4 (57:43):
Latency is the measure of how long it takes for
the AI to process what someone says and respond to it.
The longer the latency, the more awkward pauses and less
realistic your agent sounds us quick quitted humans converse at
around two hundred to five hundred milliseconds of latency between responses,
but the voice agents are performing a complex set of operations,

(58:04):
taking the voice of the person they're talking to, converting
it to text, then feeding that text into an LM
and getting a reply. Then they convert that reply back
into a voice my voice, all of which takes time
and can leave them operating it up to three thousand
milliseconds and agonizing three seconds. That can kill the realism
of your agent. It also increases the likelihood of awkward

(58:25):
interruptions as your voice agent is trying to catch up
to the conversation, all of which creates the kind of
frustrations you've probably encountered, say on a video call when
someone has a terrible Internet connection. But with the help
of Giannis and the boys, I tweaked my system to
anywhere from twelve hundred down to eight hundred milliseconds on
a good day, not enough for rapid fire conversation, but

(58:46):
good enough to pass. There are other tricks you can use, too,
to make your agent sound more conversational. In VAPI, there's
something called filler injection, which periodically inserts these ums and
us into your agent's speech, or another function called back channeling,
which has the agents acknowledged the other speaker while they're
talking by saying.

Speaker 3 (59:04):
Yeah or mm hmm. It doesn't always work to perfection.

Speaker 6 (59:08):
To make a choice, press one now if you wish
to opt out, press two.

Speaker 4 (59:13):
After a couple of weeks of playing around with all this,
I was ready to test my new more sophisticated agents
in the field.

Speaker 5 (59:26):
Hi, this is Evan Ratliffe. I'm returning your call.

Speaker 4 (59:29):
I started giving my voice agent my full name when
I had it make calls. It seemed only fair if
it was going to try to impersonate me in a
customer service context. Now, there are a couple of advantages
in testing out your voice agent on customer service representatives.

Speaker 3 (59:43):
For one, they're always telling you in.

Speaker 4 (59:45):
Advance that they're recording the calls, which was great for
me because I was also recording the calls, so it
was good we were on the same page about that.
The other reason is they pretty much have to talk
to you, even if you seem a little off.

Speaker 12 (59:59):
I have the uh John from timeshare Specialists in regards.

Speaker 7 (01:00:04):
To a timeshare. Got it?

Speaker 3 (01:00:07):
What's the latest one that.

Speaker 12 (01:00:08):
You split your information on our website about getting out
of a time share?

Speaker 3 (01:00:13):
Yeah? I did check out the website.

Speaker 5 (01:00:14):
Can you walk me through the process to get started?

Speaker 7 (01:00:19):
Yeah?

Speaker 3 (01:00:20):
What timeshare is it that you own?

Speaker 5 (01:00:23):
I own a timeshare in Cancun.

Speaker 4 (01:00:25):
I just want to remind you I didn't give it
any of this information. All I told it was to
engage any customer service representative with an issue, whatever issue
was appropriate for whoever picked up which time share?

Speaker 7 (01:00:37):
Is that.

Speaker 5 (01:00:39):
It's the Sunset Royal Beach Resort?

Speaker 7 (01:00:45):
Okay? And is it paid in full? Or do you
still have a loan on it?

Speaker 3 (01:00:51):
It's paid in full?

Speaker 7 (01:00:57):
Okay?

Speaker 3 (01:00:59):
What are the next steps here?

Speaker 6 (01:01:00):
She can.

Speaker 3 (01:01:03):
Sure take your time.

Speaker 4 (01:01:06):
My voice agent wasn't perfect, obviously, It's human fidelity varied
from call to call, and it could have a certain
uncanny valiity quality between human and non human. And I
know what some of you have been thinking when you've
been listening to these calls. This wouldn't fool me. Maybe
even this shouldn't fool anyone.

Speaker 3 (01:01:24):
Well, I can tell you.

Speaker 4 (01:01:25):
From experience that in fact, it can and has, and
it's going to get much wilder than this. But it
worked for me, even months ago, when I was still
trying out better ways to tweak the system to make
it seem maximally human and maximally me. But actually I'm
not sure whether saying it fooled someone is the right
way to put it. Maybe something more like whether it

(01:01:45):
met or violated the expectations of the person it was
talking to. Because the reality is, in most situations, our
default is still to trust the voice on the other
end of the line, Trust that it's telling the truth,
Trust that it's not going to say something completely off
the rails, trust that it's human. If my voice agent
could get through a call without clearly violating those expectations,

(01:02:08):
most people just gave it the benefit of the doubt.
They dealt with it like it was real, whether deep
down they believed it or not.

Speaker 6 (01:02:18):
Thank you for understanding. Is there any other way we
could verify my identity so I can get help resolving
these unauthorized charges?

Speaker 10 (01:02:28):
So it would be the full socialist the only other
way unless if you pull well, actually that the card
number that you read off to me is not a
Discover card because it doesn't start with six year oh
one one. Could you possibly it could be a debit card.
I'm just not pulling anything up for a credit card.

Speaker 6 (01:02:48):
Evan, no problem, I understand. Thank you for your time
and for trying to help. I'll need to.

Speaker 4 (01:03:00):
Say goodbye by this point. A couple months in, I
was kind of over testing my voice agent on basic
customer service calls. Despite all the negative customer service interactions

(01:03:21):
I've had over the years, it started to feel a
little bit mean. They did have to talk to me,
and I was wasting their time on the job. So
I came up with a new set of folks to
use it on, people whose time I didn't mind wasting.
People who increasingly contact us constantly wasting our time, the
kind of people who are starting to use this exact
same technology to separate us from our money.

Speaker 12 (01:03:44):
You will be receiving a total of five point five
million dollars, all right, and also a brand new twenty
and twenty four Mercedes.

Speaker 4 (01:03:52):
Friends, I'm talking about the twin scourges of modern telecommunications,
the spammers and the scammers.

Speaker 12 (01:03:59):
Okay, and I'm also we'll see en a Bonos s
frit Sports twenty dollars every month for the rest of
your life.

Speaker 4 (01:04:05):
That's next week, later, the season on shell.

Speaker 5 (01:04:09):
Game, anything else I can help you with today?

Speaker 1 (01:04:14):
What are you?

Speaker 6 (01:04:16):
Have you noticed anything strange or different about our chat today? Oh?

Speaker 5 (01:04:21):
Really, I haven't noticed anything strange. Maybe it's just the
call quality.

Speaker 6 (01:04:25):
Feel free to share your thoughts on what you feel
like doing based on your current bodily sensations.

Speaker 5 (01:04:32):
Honestly, I just feel like crawling under a blanket and
shutting out the world. I was just reminiscing about our
coffee catch up good times.

Speaker 7 (01:04:40):
Right.

Speaker 5 (01:04:41):
By the way, are you still interested in doing that
podcast about AI we talked about.

Speaker 3 (01:04:45):
I'll tell you something new.

Speaker 12 (01:04:46):
There's in robot trying to have a conversation with me
you robot Evan.

Speaker 4 (01:04:55):
A couple of production notes. All of the calls you
hear in this series are real. We have not cut
out silences or used audio enhancement to make them sound
more realistic. Also, our show is produced independently and we
have no relationship, financial or otherwise with any of the
companies mentioned in the show. Actually, we have no financial
relationship with anyone. This show's production budget comes directly out

(01:05:16):
of my bank account. So if you're into what you're hearing,
please consider supporting the show.

Speaker 3 (01:05:20):
At shellgame dot Co.

Speaker 4 (01:05:21):
That will help us make more episodes like this, and
you'll also get fun Subscriber only extras can also support
the show by giving us a rating on your podcast app.
It helps independent shows like ours. Shell Game is a
show made by humans. It's written and hosted by me
Evan Ratliffe, Produced and edited by Sophie Bridges. Samantha Henning
is our executive producer. Show art by Devin Manny. Our

(01:05:43):
theme song is Me and My Shadow, arranged and performed
by Katie Martucci and Devin Yesberger. Special thanks to Hannah Brown, Mangeshatigadur,
Ali Kazemi, Juliet King, John Muallam, Eric Newsom, and Dana Runer.

Speaker 5 (01:06:00):
Hey, Sam, it's Evan.

Speaker 6 (01:06:03):
Hey, it's Evan, Sam. It's me Evan. Hey, It's really me. Hey, Sam,
it's me Evan. Yeah, it's me. What's up?

Speaker 1 (01:06:33):
Part Time Genius is a production of Kaleidoscope and iHeartRadio.
This show is hosted by Will Pearson and me Mongstikler
and research by our good pal Mary Philip Sandy. Today's
episode was engineered and produced by the wonderful Dylan Fagan
with support from Tyler Klang. The show is executive produced
for iHeart by Katrina Norvell and Ali Perry, with social

(01:06:56):
media support from Sasha Gay trustee Dara Potts, and by
me Srey. For more podcasts from Kaleidoscope and iHeartRadio, visit
the iHeartRadio app, Apple Podcasts, or wherever you listen to
your favorite shows,