October 11, 2023 • 25 mins
What trade-offs exist between safeguarding privacy and fostering innovation in AI? Explore the delicate balance between safeguarding privacy and nurturing innovation in AI on Partners In Possibility, hosted by CEO of Plus Company Brett Marchand. Join us for the second part of our captivating insider series on the digital economy. We're thrilled to bring you Avi Goldfarb, a distinguished AI expert with unparalleled insights.
Brett and Avi delve into the realm of AI and prediction technology and how they shape the landscape of marketing. Tune in to uncover the essential role of human judgment in effectively harnessing this transformative technology. Additionally, we've got something special in store - an exclusive preview of our groundbreaking innovation, AIOS. This intelligent All-in-One System for Marketing is poised to unlock boundless potential in the consumer journey.
The AI revolution is here—subscribe to Partners in Possibility on all podcast platforms to stay ahead of the AI curve!
Brett and Avi delve into the realm of AI and prediction technology and how they shape the landscape of marketing. Tune in to uncover the essential role of human judgment in effectively harnessing this transformative technology. Additionally, we've got something special in store - an exclusive preview of our groundbreaking innovation, AIOS. This intelligent All-in-One System for Marketing is poised to unlock boundless potential in the consumer journey.
The AI revolution is here—subscribe to Partners in Possibility on all podcast platforms to stay ahead of the AI curve!
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:01):
And we can figure out how technology enables you to
deliver a new kind of value. That's when we can
really see the upside potential. The risk that so many
established companies have is they say, you know what, we're good,
we don't need to worry about it, and then a
startup comes along and this is where we get disruption.
Speaker 2 (00:24):
In an ever changing world. That's all about stained connected,
building connections and seeing where the next collaboration takes a
marketing campaign from an initial brief to the follow through.
What paths are going to make a campaign success more
than a possibility. Hi, I'm Brett Marshand, CEO of Plus
Company is Partners and Possibility. During this exciting and transformative period,
(00:49):
we're living in what Abby Goldfarb calls the between times.
We are witnessing huge advances in technology and machine learning.
Abby is the Rotmann Chair in AI and Healthcare at
the univer Risty of Toronto. It's also Chief Data Scientists
at the Creative Destruction Lab and co author of the
best selling book Prediction Machines. And that's follow up Power
and Prediction. Today, we continue our conversation about AI and
(01:12):
prediction technology and how it will change our industry. We
talk about the vital role that human judgment plays and
using this technology most effectively and how we here at
Plus Company are unlocking what's possible in the consumer journey
with our newest innovation AOS. Let's talk about digital marketing
(01:35):
and advertising. What further disruption do you think is coming?
Speaker 1 (01:39):
So more of a question of what part of the
industry isn't going to be transformed? And this is such
an information intensive industry. Yeah, I think there's two core
areas that were thinking into the future. The first one
is in content development. So right now, content is still
not personalized, so broadly you have humans developing content for
(02:01):
that is going to be appealing to a wide variety
of people. What AI can do is take that human
developed content and then adapt it to the particular needs
of users based on the various data coming in. And
so we can use generative tools, so image generation and writing,
(02:25):
to take an advertising campaign theme that would be human
generated and adapt it to the particular needs of a
user who might be seeing it an ad at a
particular time. So that's going to be there's going to
be a real transformation and content to content development. I
think the other side of it is on optimization and allocation.
(02:46):
So how do we how do we develop effective campaigns
that don't just get the right ad to the right
person at the right time, but do it in a
way that's optimal from a budget sense, so that the
company isn't just maximizing revenue per se or maximizing clicks,
(03:12):
but is actually getting a return for their ad dollars
in a way that right now is very very hard
to deliver and measure.
Speaker 2 (03:22):
You've picked on two areas by the way, that we're
putting a major focus on, both on the content development
and customization side, but also on row as. And we've
talked about in past podcasts we at this project called
Project Catalyst, But you've talked about prediction and separating prediction
in the decision making process. I think that's a really
important concept that people will be interested in hearing your
(03:45):
reflections on.
Speaker 1 (03:46):
Remember what today's AI does is it's computational statistics, it's
prediction technology, and what are predictions for? Predictions in and
of themselves have no use. What a prediction is for
is it helps you make better decisions, and decision making
is the big deal, especially in information intensive industries. That's
(04:08):
that's what the benefit of prediction comes from helping us
make better decisions. The hope for humanity, right is that
prediction isn't everything involved in a decision. There's all these
other parts of a decision that still require a human touch.
One of them is understanding what data to put into
the prediction and how that works. Another is an action
(04:30):
like actually doing something with your prediction. But the most
important one is this thing we call judgment, and judgment
at a high level is knowing which predictions to make
and what to do with those predictions once you have them,
and that remains inherently human. Yeah, my best explanation of
judgments from this. You see the movie I Robot. Yeah,
(04:54):
it's a pretty good science fiction movie. It's it's a
pretty good movie, and it's excellent science fiction. And and
what makes it excellent science fiction is this flashback scene.
So Will Smith is the protagonist and I Robot, and
he hates robots. So you can kind of see where
the movie's gonna go. And there's this scene about why
he hates robots. So he and this little girl, they're
(05:14):
driving in a car and they get into a car
accident and they both start sinking into a river. And
it's clear they're both about to drown, and then a
robot comes along and saves him and not the girl,
and that's why he hates robots, right. So it turns out,
because it was a robot, you could audit it, and
so he audited it and he figured out, well, why
did it save him and not the girl? And it
(05:35):
turned out the robot predicted that he had a forty
five percent chance survival and the girl had an eleven
percent chance to survival, and so Will Smith, the detective, says, well,
but eleven percent was more than enough, and a human
being would have known that. Well, I don't know if
eleven percent was more than enough. That's but that's an
inherently human statement. That's judgment. So the someone programmed that robot,
(05:59):
that fictional robot, to say, a life is a life,
forty five is more than eleven, so save him. When
you have an AI, the AI is providing a number.
It is providing a prediction that is only useful if
you then provide your judgment on what do you do
with that prediction. That's disruptive to industry. It's not disruptive
(06:20):
to industry because it's uncomfortable. We can figure that out.
It's disruptive to industry because the people who are good
at doing a prediction and judgment together, or the parts
of your organization that are good at making decisions where
you have that prediction and judgment bundled together, might not
be the same as those that are good when the
prediction is provided by a machine and there is a
(06:42):
human providing judgment.
Speaker 2 (06:43):
It's funny because we have an example of this. We
did a campaign for Google Cloud in the US where
during the Super Bowl, the AI engine at Google was
taught by every single game that ever been done in
the NCAA before, and then they took what happened in
(07:05):
the first half and at halftime made a prediction on
what would happen in the second half. So and it
was incredibly accurate. I mean, they guessed pretty much the score,
who would win, but also a whole bunch of other predictions.
You know, how many three pointers would be scored, et cetera.
And one of the judgments that we had to make
(07:26):
with Google is what because we then on the fly
put an add on air and said here's what's going
to happen in the second half, and of course predicting
the score in the second half. That's where we had
to use judgment to say no, that's not the right
thing to do, because you can imagine what it would
have done to you know, sports betting and people's anxiety
around who would win or not. I mean, there's all
(07:47):
kinds of issues there. So it's a good example. I
hadn't thought about it that way until you just put
it that way. But that was one of the core
things that we had to think about when we did
that campaign with Google. As we explore the innately human
role of judgment in making use of new prediction technology,
we can also begin to imagine how it will affect
(08:08):
the larger structures within different industries, and how do you
think that will help the potential of AI. This decoupling
you talk about.
Speaker 1 (08:18):
A lot of the potential is going to be through
a reinvention of the organization. Just like with electricity, the
decoupling of the power source and the machine enable the
reinvention of the factory. With AI, a decoupling of the
prediction from the judgment is going to enable reinvention of
(08:39):
the organization right and in particular because the people who
are good at doing the prediction and judgment together may
not be the best at doing judgment when the machine
provides prediction. We can see that in industry by industry.
You can imagine it in healthcare right now, doctors are
really good at diagnosis, and as part of that process
(09:02):
they also help the patient manage the stress of dealing
with the healthcare system, dealing with difficult times. Maybe with
the machine is providing the prediction on the diagnosis, you
still want someone with ten years of post secondary training
or more to help the patient manage the stress. But
(09:24):
maybe you want a totally different kind of person. Maybe
you want someone more like a social worker and less
like somebody who was at the top of their class
all the way through it, and that would lead to
a reinvention of the industry and an entirely new kind
of organization change. In human resources, we have a similar issue,
which is right now a lot of HR decisions are
(09:45):
made on the ground by individuals who know the employees
personally and have a sense of prediction and how they're
going to do in the company, and some judgment about
what the company values. AI enables is it can enable
the scaling of predicting who's going to succeed. Certainly in
(10:06):
companies where they have hundreds or thousands of workers who
are doing very similar jobs. But we still need someone
at headquarters to provide the judgment about what success looks like.
Speaker 2 (10:16):
Especially with bias, right, because if you AI would be
biased based on history only because it can only look
at the past and what's happened. Right, So you're right,
I mean there you need judgment to say, Okay, well
maybe the prediction says this, but we really got to
give this person a chance.
Speaker 1 (10:32):
Absolutely, if the leadership cares about reducing bias, I'm incredibly
optimistic that AI will do better than whatever human processes
people have in place. That's an important if I get that.
But if leadership cares about changing things, machine prediction allows
(10:52):
them to audit right, to figure out what's going wrong,
and also to proactively learn kind of system solution here.
That's that I've seen deployed in terms of HR is
to say, there's a big company that decided to use
AI to help them with their hiring processes. Okay, and
(11:14):
they never deployed because the AI said don't hire any
women or minorities. They looked at it and like, why
did the AI say don't hire women? Minorities and the
answer is, obviously, they hadn't hired very many women are
under representa minorities in the past, and so the AI
was just predicting based on that past data. But the
manager in charge said, you know what, that's that's not right,
(11:35):
and they dug into what was going on, and they
realized because it's not that the AI was predicted women
minorities would do badly, it was that the AI essentially
was making predictions with very high variants because it had
no idea, because there was no data on which women
are minorities might succeed in the in the company right.
And so the strategy was when new applicants come along
(11:57):
that have high potential, the best guess wasn't that good
but of high ouption value, that you should proactively hire
those people, recognizing that on average they might not be
as good. But in the process, we will learn which
underrepresented people will thrive in the company. And they did that.
(12:19):
They hired people who they were very uncertain about how
they performed, they learned who would succeed, and then over
time they no longer needed to do that and they
had good estimates and they're hiring practices became much much
more equitable.
Speaker 2 (12:37):
That's a great example. It's funny because you know, we
also saw it in digital marketing to some degree, this issue, right,
brand safety, which was a huge issue on platforms like YouTube,
et cetera, ran into the same problem, right because the
prediction models, which were just then buying advertising through programmatic,
were basically sending people to websites and and to watch
(13:01):
videos that you know, brands weren't very comfortable with. I
mean obviously, you know, training videos for al Qaida, you know,
porn sites, you know, a bunch of places where the
prediction would say, oh, that's probably that's the best place
to go, you know, to get the most optimal and
cheapest eyeballs and potential click through right rates, but obviously
not good for the brand. You know, morally, not the
(13:23):
right thing to do. And therefore we need to insert
a level of judgment so that a human could say, listen,
you know, even though that might be what the prediction
model says, that's not the right thing to do.
Speaker 1 (13:34):
Absolutely.
Speaker 2 (13:35):
Let's take a look a little bit at this point
and system solution idea and what the differences are and
what value it brings. Each of those brings to a company.
Speaker 1 (13:46):
So what I've seen most companies do is they look
for these easy wins. They identify their existing workflow, they
unpack it, and they can so step by step, what
does our workflow look like. They can find a task
done by a human that is a prediction task, tick
out that human. They drop it in AI. They keep
the rest of the workflow the same, and since they've allocated,
typically they've allocated that human to some other task, they've
(14:08):
saved a little bit of money on that particular task
that in some cases is useful, but it typically isn't transformative.
And so what I've seen is you do these easy
wins and you save some money. But if that's somebody
at the director level or below doing that kind of thing,
(14:29):
they can't get anybody else to care. Ok, the CEO
views it, Hey, you know, incrementally improving the workflows that
you already have, that's your job. Great, good for you.
You did it, and no one pays attention. It doesn't
lead to transformative change in the company, and it doesn't
even lead to transformative change in the careers of the
people implementing the point solutions, where much more value can
(14:53):
be unlocked. But it's much more risky, I don't want
to hide that point is when you can start thinking
about how does this technology enable delivering enable you to
deliver a new kind of value, and we can figure
out how technology enables you to deliver a new kind
of value. That's when we can really see the upside
(15:15):
potential and the risk that so many established companies have
as they say, you know what, we're good, we don't
need to worry about it. And then a startup comes
along and figures out that the value that you know,
a company, an established company and incumbent might think they're
(15:36):
delivering can be delivered much better in combination with with
prediction technology. And this is where we get disruption and
the real risk that new technologies come about for incombment firms.
Speaker 2 (15:52):
It's interesting because you know Michael Cohen, which is how
we met actually who's our chief data and analytics officer,
and he's built in a platform or we're building a
platform with his leadership, which we call AOS. That's that's
the product name we call Project Catalyst, which is basically
a platform and it builds on what you talked about
earlier for the for the digital and advertising community and
(16:17):
industry where we basically can use prediction to just to
figure out what message where, what target, what product, what geography,
and how do you optimize clients marketing spend in order
to get the best return is super powerful and and
(16:40):
because it's using generative AI, you can then also go
to the next stage, which is, you know, then use
it to also customize the actual content, you know, and
do a real time basis. You know, for us, it's
causing us to also think about workflow, and I mean it,
you know, because we're in an industry that has looked
(17:00):
very similar despite all the changes that you talked about
earlier for a long time. You still do a brief
and then from a brief you do the content, and
then from the content you figure out what the what
the media plan is going to look like, and then
you execute it. Then you do research and figure out
what worked and didn't work, and you go back and
you know that workflow is very similar and we are
(17:23):
having to think about maybe we have to turn the
entire workflow upside down. You know, you actually start with
what's in the marketplace now first and foremost in order
to create the prediction, and then that then leads to
a different set of content. You know, content development is
now on a continuous basis and you're not you know,
(17:44):
it's not a batch process anymore across anything you do,
including analog, et cetera. So it's just mind boggling how
how much change could happen in our company, but also
in our industry in the next few years, especially given
how evill and AI has been in digital advertising and
in marketing for the last twenty years.
Speaker 1 (18:05):
As you say, absolutely, I'm actually curious to hear what
you guys have been up to and what that transformation
is look like.
Speaker 2 (18:14):
We're going to take a quick break and when we
come back, we'll talk more about plus aos and how
we have plus company plan to use it to optimize
the consumer journey, plus some of the challenges we're facing,
and avi's predictions for the near future. Wow, welcome back
(18:38):
to Partners and Possibility. I'm Brett Marsch on plus aos
is how plus Company is exploring the ways prediction technology
and generative AI can improve the way we market. We're
at a point where we are now testing it. We're
about to launch this new platform, so we're testing it
with with with clients. I mean, you won't be surprised
(19:01):
to hear that the predictive model is incredibly accurate, you know,
and we're doing a lot of work around customer journey
instead of thinking sort of top down, you know, where
did you do your marketing and how did it impact
your sales, et cetera, but actually doing a consumer by
consumer and to your earlier point, not only does that
allow you to figure out how to optimize around consumer
(19:24):
journey and use synthetic data to fill in the banks
about you know, what's the consumer journey look like, but
also then to optimize the content in a way that's
going to make it as effective as possible. And so
I think this is where the power of AI and
generative AI is incredibly interesting. But it may mean for us,
(19:46):
you know, a pretty massive change in our workforce, you know,
because we're going to have to train them on how
to use these We're going to have to train people
on how to use judgment to your point, to make
sure that we are not just letting the MA to
drive all the decisions, because it could, you know, it
could lead to the wrong kinds of decisions for us.
(20:07):
We're using what we call a community of practice around
AI instead of instead of doing it centralized. And I
don't know if this is the right approach or not,
but I think it is, which is have a bunch
of experiments that are working on each of these areas
and working directly with clients at the agency level, at
the grassroots level, seeing what works and doesn't work, figuring out,
(20:30):
you know, how do we have to change workflow and
or our processes and our workforce. And we're starting to
learn that it will have some major impacts on what
the company looks like. And I think that for me
that's acting more like a startup and less like a
you know, a big company that's doing it top down.
But I don't know if you have a point of
view on whether you think that's the right approach or not.
Speaker 1 (20:52):
It's very hard for a big company to do it
top down without without allowing some experimentation in different parts. Right,
So the one of the one of the big challenges
in the between times is we don't know exactly what's
going to work. We have some hypotheses and some ideas,
and we know we kind of know what we're doing
right now isn't going to work, but exactly what the
(21:15):
better system is is hard to discover. And so enabling
your organization, your customers, your suppliers to experiment a little
bit in order to help you learn and figure out
what's best is it's clearly going to be better than
you know, you deciding okay, this is definitely this is
(21:35):
definitely what the future should be and just running with that.
Speaker 2 (21:39):
Yeah, and you asked this really interesting question in your book,
which is, if we knew then what we know today
about AI, you know, how we would behave differently? What
would we change? What do you think that looks like
for our industry.
Speaker 1 (21:52):
There's an extraordinary potential for personalization and to really help
help advertising do what it's supposed to in the sense
that helping people find the products that they want and
helping advertisers get in front of the people whose behavior
(22:18):
will be changed because of the ad. A lot of
our focus over the past few years has been, you know,
on the tech side, sure, and on the design side,
but a little less on how do we think through
what kinds of ads are going to not just get
(22:39):
people to notice them, but lead to the change in
behavior that helps the advertiser while also helping the user
consumer do you know, get what they want when they
want and exactly what the skills are. That's we're starting
to figure that out. I don't think I would have
known that five years ago, but this move toward personalization
(23:02):
and the opportunities around it has been extraordinary.
Speaker 2 (23:05):
If AI can help us influence the kinds of products
that we buy that are aligned with our values and
the things we think are important in the world. Right
who's making a difference in climate change, who's actually treating people,
you know, in an equitable manner, who's producing products that
are going to be good for us as individuals and
(23:30):
as communities. I mean, there are a whole bunch of
variables here that right now, as a consumer you really
don't have any sense of unless you dive in and
do a ton of research. But that's actually something that
could be incredibly beneficial, I think, with AI in the future,
so that you're actually making purchases and voting with your
dollars in a way that's actually going to make the
world a better place. And I know that sounds, you know, utopian,
(23:51):
but I actually believe that that's one of the potentially
beneficial impacts of AI on our industry and therefore on
company and on the world.
Speaker 1 (24:01):
Yeah, I love that vision. It's a really exciting future
for the industry overall.
Speaker 2 (24:06):
Ivan, this has been fascinating. You know, I could spend
hours with you talking about this. I'll ask you one
last question, which is one prediction for the future. One
thing you think is that we're going to see three
years from now, five years from now because of AI
and its power, what do you think that is?
Speaker 1 (24:29):
I think AI is going to reduce discrimination and bias
in a way that in retrospect will become obvious, and
the resistance to it is going to increasingly come from
the people who benefit from the biases in the current system.
And so looking back five years from now, when we
(24:49):
think through you, why didn't we adopt in this place
versus that place, and who is fighting it, will realize
that the people who thought hardest were not the people
necessarily who cared about reducing bias, but they were the
people who really benefited from the way the current system worked.
Speaker 2 (25:12):
Yeah. Well, thank you for that, Jeffrey. I look forward
to many conversations. Good luck with all your research. It'll
be fascinating to hear where you go next.
Speaker 1 (25:22):
Fantastic. Thanks for a great conversation.
Speaker 2 (25:27):
Thank you for listening to partners and possibility. That was
part two of my conversation with Avi Goldfarb. I hope
you enjoyed our conversation on what AI and prediction technology
means to the marketing field and the preview of our
new intelligence all in one system for marketing plus AOS.
Read the books Avi co authored for even more insights
into prediction technology and AI in the between times. We
(25:49):
have links to them in the show notes.
And we can figure out how technology enables you to
deliver a new kind of value. That's when we can
really see the upside potential. The risk that so many
established companies have is they say, you know what, we're good,
we don't need to worry about it, and then a
startup comes along and this is where we get disruption.
Speaker 2 (00:24):
In an ever changing world. That's all about stained connected,
building connections and seeing where the next collaboration takes a
marketing campaign from an initial brief to the follow through.
What paths are going to make a campaign success more
than a possibility. Hi, I'm Brett Marshand, CEO of Plus
Company is Partners and Possibility. During this exciting and transformative period,
(00:49):
we're living in what Abby Goldfarb calls the between times.
We are witnessing huge advances in technology and machine learning.
Abby is the Rotmann Chair in AI and Healthcare at
the univer Risty of Toronto. It's also Chief Data Scientists
at the Creative Destruction Lab and co author of the
best selling book Prediction Machines. And that's follow up Power
and Prediction. Today, we continue our conversation about AI and
(01:12):
prediction technology and how it will change our industry. We
talk about the vital role that human judgment plays and
using this technology most effectively and how we here at
Plus Company are unlocking what's possible in the consumer journey
with our newest innovation AOS. Let's talk about digital marketing
(01:35):
and advertising. What further disruption do you think is coming?
Speaker 1 (01:39):
So more of a question of what part of the
industry isn't going to be transformed? And this is such
an information intensive industry. Yeah, I think there's two core
areas that were thinking into the future. The first one
is in content development. So right now, content is still
not personalized, so broadly you have humans developing content for
(02:01):
that is going to be appealing to a wide variety
of people. What AI can do is take that human
developed content and then adapt it to the particular needs
of users based on the various data coming in. And
so we can use generative tools, so image generation and writing,
(02:25):
to take an advertising campaign theme that would be human
generated and adapt it to the particular needs of a
user who might be seeing it an ad at a
particular time. So that's going to be there's going to
be a real transformation and content to content development. I
think the other side of it is on optimization and allocation.
(02:46):
So how do we how do we develop effective campaigns
that don't just get the right ad to the right
person at the right time, but do it in a
way that's optimal from a budget sense, so that the
company isn't just maximizing revenue per se or maximizing clicks,
(03:12):
but is actually getting a return for their ad dollars
in a way that right now is very very hard
to deliver and measure.
Speaker 2 (03:22):
You've picked on two areas by the way, that we're
putting a major focus on, both on the content development
and customization side, but also on row as. And we've
talked about in past podcasts we at this project called
Project Catalyst, But you've talked about prediction and separating prediction
in the decision making process. I think that's a really
important concept that people will be interested in hearing your
(03:45):
reflections on.
Speaker 1 (03:46):
Remember what today's AI does is it's computational statistics, it's
prediction technology, and what are predictions for? Predictions in and
of themselves have no use. What a prediction is for
is it helps you make better decisions, and decision making
is the big deal, especially in information intensive industries. That's
(04:08):
that's what the benefit of prediction comes from helping us
make better decisions. The hope for humanity, right is that
prediction isn't everything involved in a decision. There's all these
other parts of a decision that still require a human touch.
One of them is understanding what data to put into
the prediction and how that works. Another is an action
(04:30):
like actually doing something with your prediction. But the most
important one is this thing we call judgment, and judgment
at a high level is knowing which predictions to make
and what to do with those predictions once you have them,
and that remains inherently human. Yeah, my best explanation of
judgments from this. You see the movie I Robot. Yeah,
(04:54):
it's a pretty good science fiction movie. It's it's a
pretty good movie, and it's excellent science fiction. And and
what makes it excellent science fiction is this flashback scene.
So Will Smith is the protagonist and I Robot, and
he hates robots. So you can kind of see where
the movie's gonna go. And there's this scene about why
he hates robots. So he and this little girl, they're
(05:14):
driving in a car and they get into a car
accident and they both start sinking into a river. And
it's clear they're both about to drown, and then a
robot comes along and saves him and not the girl,
and that's why he hates robots, right. So it turns out,
because it was a robot, you could audit it, and
so he audited it and he figured out, well, why
did it save him and not the girl? And it
(05:35):
turned out the robot predicted that he had a forty
five percent chance survival and the girl had an eleven
percent chance to survival, and so Will Smith, the detective, says, well,
but eleven percent was more than enough, and a human
being would have known that. Well, I don't know if
eleven percent was more than enough. That's but that's an
inherently human statement. That's judgment. So the someone programmed that robot,
(05:59):
that fictional robot, to say, a life is a life,
forty five is more than eleven, so save him. When
you have an AI, the AI is providing a number.
It is providing a prediction that is only useful if
you then provide your judgment on what do you do
with that prediction. That's disruptive to industry. It's not disruptive
(06:20):
to industry because it's uncomfortable. We can figure that out.
It's disruptive to industry because the people who are good
at doing a prediction and judgment together, or the parts
of your organization that are good at making decisions where
you have that prediction and judgment bundled together, might not
be the same as those that are good when the
prediction is provided by a machine and there is a
(06:42):
human providing judgment.
Speaker 2 (06:43):
It's funny because we have an example of this. We
did a campaign for Google Cloud in the US where
during the Super Bowl, the AI engine at Google was
taught by every single game that ever been done in
the NCAA before, and then they took what happened in
(07:05):
the first half and at halftime made a prediction on
what would happen in the second half. So and it
was incredibly accurate. I mean, they guessed pretty much the score,
who would win, but also a whole bunch of other predictions.
You know, how many three pointers would be scored, et cetera.
And one of the judgments that we had to make
(07:26):
with Google is what because we then on the fly
put an add on air and said here's what's going
to happen in the second half, and of course predicting
the score in the second half. That's where we had
to use judgment to say no, that's not the right
thing to do, because you can imagine what it would
have done to you know, sports betting and people's anxiety
around who would win or not. I mean, there's all
(07:47):
kinds of issues there. So it's a good example. I
hadn't thought about it that way until you just put
it that way. But that was one of the core
things that we had to think about when we did
that campaign with Google. As we explore the innately human
role of judgment in making use of new prediction technology,
we can also begin to imagine how it will affect
(08:08):
the larger structures within different industries, and how do you
think that will help the potential of AI. This decoupling
you talk about.
Speaker 1 (08:18):
A lot of the potential is going to be through
a reinvention of the organization. Just like with electricity, the
decoupling of the power source and the machine enable the
reinvention of the factory. With AI, a decoupling of the
prediction from the judgment is going to enable reinvention of
(08:39):
the organization right and in particular because the people who
are good at doing the prediction and judgment together may
not be the best at doing judgment when the machine
provides prediction. We can see that in industry by industry.
You can imagine it in healthcare right now, doctors are
really good at diagnosis, and as part of that process
(09:02):
they also help the patient manage the stress of dealing
with the healthcare system, dealing with difficult times. Maybe with
the machine is providing the prediction on the diagnosis, you
still want someone with ten years of post secondary training
or more to help the patient manage the stress. But
(09:24):
maybe you want a totally different kind of person. Maybe
you want someone more like a social worker and less
like somebody who was at the top of their class
all the way through it, and that would lead to
a reinvention of the industry and an entirely new kind
of organization change. In human resources, we have a similar issue,
which is right now a lot of HR decisions are
(09:45):
made on the ground by individuals who know the employees
personally and have a sense of prediction and how they're
going to do in the company, and some judgment about
what the company values. AI enables is it can enable
the scaling of predicting who's going to succeed. Certainly in
(10:06):
companies where they have hundreds or thousands of workers who
are doing very similar jobs. But we still need someone
at headquarters to provide the judgment about what success looks like.
Speaker 2 (10:16):
Especially with bias, right, because if you AI would be
biased based on history only because it can only look
at the past and what's happened. Right, So you're right,
I mean there you need judgment to say, Okay, well
maybe the prediction says this, but we really got to
give this person a chance.
Speaker 1 (10:32):
Absolutely, if the leadership cares about reducing bias, I'm incredibly
optimistic that AI will do better than whatever human processes
people have in place. That's an important if I get that.
But if leadership cares about changing things, machine prediction allows
(10:52):
them to audit right, to figure out what's going wrong,
and also to proactively learn kind of system solution here.
That's that I've seen deployed in terms of HR is
to say, there's a big company that decided to use
AI to help them with their hiring processes. Okay, and
(11:14):
they never deployed because the AI said don't hire any
women or minorities. They looked at it and like, why
did the AI say don't hire women? Minorities and the
answer is, obviously, they hadn't hired very many women are
under representa minorities in the past, and so the AI
was just predicting based on that past data. But the
manager in charge said, you know what, that's that's not right,
(11:35):
and they dug into what was going on, and they
realized because it's not that the AI was predicted women
minorities would do badly, it was that the AI essentially
was making predictions with very high variants because it had
no idea, because there was no data on which women
are minorities might succeed in the in the company right.
And so the strategy was when new applicants come along
(11:57):
that have high potential, the best guess wasn't that good
but of high ouption value, that you should proactively hire
those people, recognizing that on average they might not be
as good. But in the process, we will learn which
underrepresented people will thrive in the company. And they did that.
(12:19):
They hired people who they were very uncertain about how
they performed, they learned who would succeed, and then over
time they no longer needed to do that and they
had good estimates and they're hiring practices became much much
more equitable.
Speaker 2 (12:37):
That's a great example. It's funny because you know, we
also saw it in digital marketing to some degree, this issue, right,
brand safety, which was a huge issue on platforms like YouTube,
et cetera, ran into the same problem, right because the
prediction models, which were just then buying advertising through programmatic,
were basically sending people to websites and and to watch
(13:01):
videos that you know, brands weren't very comfortable with. I
mean obviously, you know, training videos for al Qaida, you know,
porn sites, you know, a bunch of places where the
prediction would say, oh, that's probably that's the best place
to go, you know, to get the most optimal and
cheapest eyeballs and potential click through right rates, but obviously
not good for the brand. You know, morally, not the
(13:23):
right thing to do. And therefore we need to insert
a level of judgment so that a human could say, listen,
you know, even though that might be what the prediction
model says, that's not the right thing to do.
Speaker 1 (13:34):
Absolutely.
Speaker 2 (13:35):
Let's take a look a little bit at this point
and system solution idea and what the differences are and
what value it brings. Each of those brings to a company.
Speaker 1 (13:46):
So what I've seen most companies do is they look
for these easy wins. They identify their existing workflow, they
unpack it, and they can so step by step, what
does our workflow look like. They can find a task
done by a human that is a prediction task, tick
out that human. They drop it in AI. They keep
the rest of the workflow the same, and since they've allocated,
typically they've allocated that human to some other task, they've
(14:08):
saved a little bit of money on that particular task
that in some cases is useful, but it typically isn't transformative.
And so what I've seen is you do these easy
wins and you save some money. But if that's somebody
at the director level or below doing that kind of thing,
(14:29):
they can't get anybody else to care. Ok, the CEO
views it, Hey, you know, incrementally improving the workflows that
you already have, that's your job. Great, good for you.
You did it, and no one pays attention. It doesn't
lead to transformative change in the company, and it doesn't
even lead to transformative change in the careers of the
people implementing the point solutions, where much more value can
(14:53):
be unlocked. But it's much more risky, I don't want
to hide that point is when you can start thinking
about how does this technology enable delivering enable you to
deliver a new kind of value, and we can figure
out how technology enables you to deliver a new kind
of value. That's when we can really see the upside
(15:15):
potential and the risk that so many established companies have
as they say, you know what, we're good, we don't
need to worry about it. And then a startup comes
along and figures out that the value that you know,
a company, an established company and incumbent might think they're
(15:36):
delivering can be delivered much better in combination with with
prediction technology. And this is where we get disruption and
the real risk that new technologies come about for incombment firms.
Speaker 2 (15:52):
It's interesting because you know Michael Cohen, which is how
we met actually who's our chief data and analytics officer,
and he's built in a platform or we're building a
platform with his leadership, which we call AOS. That's that's
the product name we call Project Catalyst, which is basically
a platform and it builds on what you talked about
earlier for the for the digital and advertising community and
(16:17):
industry where we basically can use prediction to just to
figure out what message where, what target, what product, what geography,
and how do you optimize clients marketing spend in order
to get the best return is super powerful and and
(16:40):
because it's using generative AI, you can then also go
to the next stage, which is, you know, then use
it to also customize the actual content, you know, and
do a real time basis. You know, for us, it's
causing us to also think about workflow, and I mean it,
you know, because we're in an industry that has looked
(17:00):
very similar despite all the changes that you talked about
earlier for a long time. You still do a brief
and then from a brief you do the content, and
then from the content you figure out what the what
the media plan is going to look like, and then
you execute it. Then you do research and figure out
what worked and didn't work, and you go back and
you know that workflow is very similar and we are
(17:23):
having to think about maybe we have to turn the
entire workflow upside down. You know, you actually start with
what's in the marketplace now first and foremost in order
to create the prediction, and then that then leads to
a different set of content. You know, content development is
now on a continuous basis and you're not you know,
(17:44):
it's not a batch process anymore across anything you do,
including analog, et cetera. So it's just mind boggling how
how much change could happen in our company, but also
in our industry in the next few years, especially given
how evill and AI has been in digital advertising and
in marketing for the last twenty years.
Speaker 1 (18:05):
As you say, absolutely, I'm actually curious to hear what
you guys have been up to and what that transformation
is look like.
Speaker 2 (18:14):
We're going to take a quick break and when we
come back, we'll talk more about plus aos and how
we have plus company plan to use it to optimize
the consumer journey, plus some of the challenges we're facing,
and avi's predictions for the near future. Wow, welcome back
(18:38):
to Partners and Possibility. I'm Brett Marsch on plus aos
is how plus Company is exploring the ways prediction technology
and generative AI can improve the way we market. We're
at a point where we are now testing it. We're
about to launch this new platform, so we're testing it
with with with clients. I mean, you won't be surprised
(19:01):
to hear that the predictive model is incredibly accurate, you know,
and we're doing a lot of work around customer journey
instead of thinking sort of top down, you know, where
did you do your marketing and how did it impact
your sales, et cetera, but actually doing a consumer by
consumer and to your earlier point, not only does that
allow you to figure out how to optimize around consumer
(19:24):
journey and use synthetic data to fill in the banks
about you know, what's the consumer journey look like, but
also then to optimize the content in a way that's
going to make it as effective as possible. And so
I think this is where the power of AI and
generative AI is incredibly interesting. But it may mean for us,
(19:46):
you know, a pretty massive change in our workforce, you know,
because we're going to have to train them on how
to use these We're going to have to train people
on how to use judgment to your point, to make
sure that we are not just letting the MA to
drive all the decisions, because it could, you know, it
could lead to the wrong kinds of decisions for us.
(20:07):
We're using what we call a community of practice around
AI instead of instead of doing it centralized. And I
don't know if this is the right approach or not,
but I think it is, which is have a bunch
of experiments that are working on each of these areas
and working directly with clients at the agency level, at
the grassroots level, seeing what works and doesn't work, figuring out,
(20:30):
you know, how do we have to change workflow and
or our processes and our workforce. And we're starting to
learn that it will have some major impacts on what
the company looks like. And I think that for me
that's acting more like a startup and less like a
you know, a big company that's doing it top down.
But I don't know if you have a point of
view on whether you think that's the right approach or not.
Speaker 1 (20:52):
It's very hard for a big company to do it
top down without without allowing some experimentation in different parts. Right,
So the one of the one of the big challenges
in the between times is we don't know exactly what's
going to work. We have some hypotheses and some ideas,
and we know we kind of know what we're doing
right now isn't going to work, but exactly what the
(21:15):
better system is is hard to discover. And so enabling
your organization, your customers, your suppliers to experiment a little
bit in order to help you learn and figure out
what's best is it's clearly going to be better than
you know, you deciding okay, this is definitely this is
(21:35):
definitely what the future should be and just running with that.
Speaker 2 (21:39):
Yeah, and you asked this really interesting question in your book,
which is, if we knew then what we know today
about AI, you know, how we would behave differently? What
would we change? What do you think that looks like
for our industry.
Speaker 1 (21:52):
There's an extraordinary potential for personalization and to really help
help advertising do what it's supposed to in the sense
that helping people find the products that they want and
helping advertisers get in front of the people whose behavior
(22:18):
will be changed because of the ad. A lot of
our focus over the past few years has been, you know,
on the tech side, sure, and on the design side,
but a little less on how do we think through
what kinds of ads are going to not just get
(22:39):
people to notice them, but lead to the change in
behavior that helps the advertiser while also helping the user
consumer do you know, get what they want when they
want and exactly what the skills are. That's we're starting
to figure that out. I don't think I would have
known that five years ago, but this move toward personalization
(23:02):
and the opportunities around it has been extraordinary.
Speaker 2 (23:05):
If AI can help us influence the kinds of products
that we buy that are aligned with our values and
the things we think are important in the world. Right
who's making a difference in climate change, who's actually treating people,
you know, in an equitable manner, who's producing products that
are going to be good for us as individuals and
(23:30):
as communities. I mean, there are a whole bunch of
variables here that right now, as a consumer you really
don't have any sense of unless you dive in and
do a ton of research. But that's actually something that
could be incredibly beneficial, I think, with AI in the future,
so that you're actually making purchases and voting with your
dollars in a way that's actually going to make the
world a better place. And I know that sounds, you know, utopian,
(23:51):
but I actually believe that that's one of the potentially
beneficial impacts of AI on our industry and therefore on
company and on the world.
Speaker 1 (24:01):
Yeah, I love that vision. It's a really exciting future
for the industry overall.
Speaker 2 (24:06):
Ivan, this has been fascinating. You know, I could spend
hours with you talking about this. I'll ask you one
last question, which is one prediction for the future. One
thing you think is that we're going to see three
years from now, five years from now because of AI
and its power, what do you think that is?
Speaker 1 (24:29):
I think AI is going to reduce discrimination and bias
in a way that in retrospect will become obvious, and
the resistance to it is going to increasingly come from
the people who benefit from the biases in the current system.
And so looking back five years from now, when we
(24:49):
think through you, why didn't we adopt in this place
versus that place, and who is fighting it, will realize
that the people who thought hardest were not the people
necessarily who cared about reducing bias, but they were the
people who really benefited from the way the current system worked.
Speaker 2 (25:12):
Yeah. Well, thank you for that, Jeffrey. I look forward
to many conversations. Good luck with all your research. It'll
be fascinating to hear where you go next.
Speaker 1 (25:22):
Fantastic. Thanks for a great conversation.
Speaker 2 (25:27):
Thank you for listening to partners and possibility. That was
part two of my conversation with Avi Goldfarb. I hope
you enjoyed our conversation on what AI and prediction technology
means to the marketing field and the preview of our
new intelligence all in one system for marketing plus AOS.
Read the books Avi co authored for even more insights
into prediction technology and AI in the between times. We
(25:49):
have links to them in the show notes.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
My Favorite Murder with Karen Kilgariff and Georgia Hardstark
My Favorite Murder is a true crime comedy podcast hosted by Karen Kilgariff and Georgia Hardstark. Each week, Karen and Georgia share compelling true crimes and hometown stories from friends and listeners. Since MFM launched in January of 2016, Karen and Georgia have shared their lifelong interest in true crime and have covered stories of infamous serial killers like the Night Stalker, mysterious cold cases, captivating cults, incredible survivor stories and important events from history like the Tulsa race massacre of 1921. My Favorite Murder is part of the Exactly Right podcast network that provides a platform for bold, creative voices to bring to life provocative, entertaining and relatable stories for audiences everywhere. The Exactly Right roster of podcasts covers a variety of topics including historic true crime, comedic interviews and news, science, pop culture and more. Podcasts on the network include Buried Bones with Kate Winkler Dawson and Paul Holes, That's Messed Up: An SVU Podcast, This Podcast Will Kill You, Bananas and more.