S2E26: "Building Ethical Machines" with Reid Blackman, PhD (Virtue Consultants)

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Reid Blackman (00:01):
LLMs don't deliberate. They don't weigh
pros and cons. They don't giveyou advice based on reasons.
What they're doing, in allcases, is predicting the next
set of words that is maximallycoherent with the words that
came before it. It's amathematical thing, right? So,
when it gives you thatexplanation, it's not actually
telling you the reason that itcame up with the output that it
gave you previously. It's morelike a post facto explanation -

(00:25):
an after-the-fact explanation,where it spits out words that
would look like it explains whathappened before (that coheres
with the words that camebefore), but doesn't actually
explain why it gave the outputthat it did. So, you might think
that it's deliberating and thengiving you the contents of that
deliberation, when in fact thatis not doing any of those

(00:45):
things.

Debra Farber (00:53):
Welcome, everyone to Shifting Privacy Left. I'm
your host and residentprivacyguru, Debra J. Farber.
Today, I'm delighted to welcomemy next guest, Reid Blackman,
PhD. Reid is the author of thebook, "Ethical Machines: your
concise guide to totallyunbiased, transparent and
respectful AI," which was justreleased last year by Harvard

(01:17):
Business Review Press. He's alsothe creator and host of the
podcast, Ethical Machines, andfounder and CEO of virtue, a
digital ethical riskconsultancy. He is also an
adviser to the Canadiangovernment on their Federal AI
regulations, was a foundingmember of EYs AI Advisory Board

(01:38):
and served as a Senior Adviserto the Deloitte AI Institute.
His work which includes advisingand speaking to organizations,
including AWS, US Bank, the FBI,NASA and The World Economic
Forum - all big organizationswe've definitely heard of - has
been profiled by The Wall StreetJournal, the BBC, and Forbes.

(02:02):
His written work appears in theHarvard Business Review and the
New York Times. Prior tofounding Virtue, Reid was a
Professor of Philosophy atColgate University and UNC
Chapel Hill. I have beenfollowing him for several years,
and I'm really excited to havehim on the show. Welcome, Reid.

Hey, thanks for having me.

I'm glad that we were able to book you I know
you're off busy, like writingand talking about AI. And the
whole field has blown up. So, Idefinitely want to talk about
some of the overlaps of privacychallenges and ethical AI and
Explainable AI. I'd love for youto just tell us a little bit
about your background, and howyou became interested in AI

(02:42):
ethics.

Sure, my background is likely to be
unlike the background of yourlisteners, from what I gather.
So, I'm not a technologist bybackground; nor am I something
like a privacy lawyer. I'm aphilosopher by background. So my
PhD is in Philosophy. I was aPhilosophy professor for 10
years. So, that means when youtake into account undergrad,
plus grad school plus being aprofessor, I've been doing
philosophy with an emphasis onethics for the past 20+ years.

(03:07):
So researching, publishing,teaching ethics for 20 plus
years. I don't totally know howI got into AI ethics, in
particular, to be totally frank.
Somehow - you know, I had thisidea for an ethics consultancy
many years ago, well, before Istarted it. And, at some point,
I don't know how - it was sortof in the ether. I became aware
of engineers ringing alarm bellsaround the ethical implications

(03:28):
of AI. And so, I just starteddigging in, I started reading up
on stuff and learning more, andI found it really interesting.
And then, a variety of othersort of both business and
personal factors came into play.
I left academia. I started thebusiness. I was particularly
interested in the ethicalimplications of AI, both because
of their impacts and because itstruck me as intellectually

(03:49):
challenging. And so, off I went.

I mean, that makes a lot of sense. I really like to
have varied guests. I mean, I'dlove the socio-technical
perspective that you're bringingto the space. So, if we only had
privacy engineers talking toother privacy engineers, I think
we'd be missing like broadersocietal challenges. And,
anything outside of just puretech and talking about tech
stacks, and how do you makethings work, so I love that.

(04:13):
What, then, motivated you towrite this book, Ethical
Machines? And then, I know yourintended audience isn't
necessarily Privacy Engineers,but who is your intended
audience, just to frame thebook?

Well, there's multiple audiences. I suppose
the most obvious or explicitaudience are senior executives,
at larger corporations who areresponsible for getting their
hands around this stuff, makingsure that things don't go
ethically, reputationally,regulatorily, and legally
sideways when it comes to usingAI and other digital
technologies. So, that's theprimary audience, but I really

(04:45):
wrote it in a way that I hopethat basically anyone can
understand. I gave it to acolleague of mine at AWS, who's
a senior leader on responsibleAI there, and she said, "Oh,
this is great," after she readit. "This is great. I can give
this to my grandmother andshe'll finally understand what I
do." And I thought, "Excellent!I want the senior executives,
the Board members, the C-suite,to understand what I'm talking

(05:07):
about. I also want a layperson.
I want students, I want dataengineers, privacy engineers,
etc. I want everyone to be ableto understand it.
And, the reason I want everyoneto understand it - the reason
why I tried to make it, to someextent everyone my audience, is
it's part of what explains why Iwrote the book to answer your
the first part of the question,which is that I just thought the

(05:28):
issues were ill-understood, notwell understood by the vast
majority of people Iencountered. Everyone knows the
headlines; or everyone knowsCambridge Analytica; everyone
knows, I don't know, maybe noteveryone, but a lot of people
know about biased ordiscriminatory AI, especially
from ProPublica's account in2016. So,we sort of know that
headlines - the headlines aroundblackbox models, you know, scary

(05:51):
blackbox AI. But, I didn't thinkthat anyone actually understood
the issues very deeply. Ithought that the understanding
was very superficial. And thenpeople went from that
superficial understanding to tryto remedy the situation. And
then everyone started screaming,"What good is ethics? It's too
it's too thin. It's too flimsy.
It's too subjective. You can'toperationalize it; we need
action!" And I just thought,"Okay, if people actually

(06:11):
understood this at a deeperlevel, action would be
relatively clear to see." Andso, that's why I set out to
write the book.

That's awesome. I think that's so needed in this
space, just making it moreaccessible to people. It's
really awesome. I have not readthe book yet, but I look forward
to doing so. I just ordered it.

Yeah, great.
Yeah, one thing, it's not merelyaccessibility, although that's
part of it. Part of what makesit accessible is the language I
use. So it's not filled withjargon. But what makes it
accessible, I think primarily,is the way (at least I hope, if
it is accessible, if I'vesucceeded and what I attempted
to do), it's showing people thelandscape. So, rather than
seeing a set, or a motley crew,of risks that are associated

(06:52):
with AI, I really wanted to showpeople the landscape - the
ethical risk landscape; show itto be a kind of coherent whole;
and show people how to navigatearound that landscape, both in
thought and action. So, that'spart of the goal, so that it's
not just sort of like, "Oh, thisis bad! What do we do about it?"
but "Okay, let's take a biggerpicture approach here. Let's see
the system. Let's see the partsof the whole, how the parts

(07:14):
relate to each other and to thewhole; and I think that gives a
better grip on how to actuallyaccommodate or manage these
risks.

Yeah, I totally agree on that. That makes a lot
of sense. I've heard you in aprevious podcast, when you refer
to two types of privacy, yourmental model: an 'active
privacy' versus 'passiveprivacy.' Can you just share
your thoughts on that?

Yeah, sure. So, when it came to talking to data
engineers, or people on the techside of the house, and they
talked about privacy, they tookit that, "Oh, okay, we respect
people's privacy on thecondition that (or maybe it's a
sufficient condition to respecttheir privacy), that their data
is suitably anonymized orpseudo-anonymized or it's
protected, eople can't getaccess to it, et cetera. And so,

(07:58):
that's a way in which you canthink of privacy in a passive
sense. So, you respect myprivacy on the condition that
you do certain things with mydata responsibly: you anonymize
it, pseudo-anonymize, et cetera,you know, behind the the correct
security walls, so to speak.
And, I am then passive withrespect to whether or not my
privacy is respected. I'mrelying on you to make sure that
my privacy is respected. Idon'tdo anything. But that's

(08:18):
different from exercising aright to privacy. If I'm
exercising my right to privacy,I'm doing something. I'm not
passive, I'm active. And so,what would one be talking about
if you're talking about dataprivacy from a perspective of
exercising your right toprivacy? Well, you're presumably
talking about something like mehaving control over who has

(08:38):
access to my data, under whatconditions, etc.
So, for instance, if I cancontrol whether or not you see
my data based on whether or notyou pay me some money for it,
that would be an example of meexercising my right to data
privacy. So, it's much moreactive. Usually, tech types
don't think about the act ofconception of privacy. They

(09:00):
think almost exclusively interms of the passive conception.
They're not explicit about this,of course, but that's the way
they think about it. So, thenyou find talk about
anonymization issues, likethings like or techniques like
differential privacy, that sortof thing. And that's all for is
important. Don't get me wrong. Ithink the passive conception of
privacy is, in some broad sense,a legitimate one. We need
technologists thinking about itand how we can do it better;

(09:22):
but, it's not the same thing asan act of conception of privacy,
and I think we need to add thatto the mix if we're going to get
governance, right.

I totally agree.
So to add on to that, I think wereally need to remember that
privacy is about individuals,and not about compliance from
the perspective of a company'sposture on privacy and their
compliance with it. Very oftenyou can get into a passive
privacy mindset where you'rejust thinking, "What is the
minimum I need to do to notviolate a law regarding privacy

(09:53):
and data protection?" as opposedto you know, really, thinking
about the customer experienceand making sure they feel trust.
And that can take an activestance on, and take action on,
the rights but also theirpreferences and they're
elevating the the individualback into the center of the
mental model of a business, asopposed to just their data that

(10:16):
a company is hoovering up aboutthem.

Yeah, that's right. And those decisions can
get played out in variousconcrete ways. Like, are we
collecting certain kinds of databy default? Or, do they have to
opt in in order for us tocollect it? If we do collect it
by default, do we give them theability to opt out of collecting
that data? If we do give themthe ability to opt out of
collecting that data, how easydo we make it for them to do it?

(10:44):
There's lots of decisions to bemade along here around thinking
about what does the individualneed in order to exercise their
rights.

Right? So, I've also heard you say that the fuel
of AI is people's data. Why isthis a problem?

It's not a problem in itself as it were. I
mean, machine learning - whichis the vast majority of AI we've
got around now - machinelearning gets fed on data.
Everyone knows that. All elseequal, the more data you have,
the better your AI gets. That'sbecause the more data it trains
from, the more it learns -almost equal. But, that means
that there is an unofficialincentive for organizations to

(11:19):
collect as much data about asmany people as they possibly
can. Right? Because they want togo far, as it were, with their
ML. They want to get as many MLmodels as they can running that
will actually solve theirbusiness problems. They want
them to be as accurate aspossible. And so, they're
incentivized collect as muchdata as they can. So, what can
happen in that pursuit of dataso that they can get their ML up

(11:41):
and running and trained well, isthey could inadvertently or
intentionally, in some cases,collect data the possession of
which itself constitutes aviolation of privacy. Relatedly,
it can be that the use of thatdata (forget about the
collection of it or the storageof it), just the using of it can
itself constitute an additionalor novel breach of someone's

(12:01):
right to privacy. So, just thenature of AI that incentivizes
organizations to breach people'sprivacy.

Yeah, I think that's been one of the toughest
things for a privacyprofessional to change a culture
within a very engineeringfocused company, BigTech
included, especially. Right?
Where the goal is collect asmuch as possible ever since the
Big Data days. Big data now toAI - it's still it's still
incentivizing engineers tocollect as much data as

(12:29):
possible. So, it definitely isexpanding the risks, and I think
it's really hard to change thatculture in industry. I mean, do
you have any ideas on how wecould get business leaders to,
you know, rein that in a littlebit?

Well, that's sort of a question around how do we
strategize to get buy in forprivacy- preserving, governance,
policies, practices, culture,etc. There's different ways. The
thing I like to say is thatdifferent people have different
motivational constitutions.
They're, they're made up ofdifferent motivational
cocktails. And so, differentpeople are going to react
(within an organization aregoing to react) to different

(13:07):
kinds of reasons for puttingguardrails around what kind of
data we collect. Some people,you say, "It's the right thing
to do. We've got to respectpeople's privacy," and they're
on board. Although, that'sprobably somewhat the minority.
I think it speaks to mostpeople, though, to varying
degrees. Then there's justthings like reputational risk,
regulatory risk, legal risk, andthose different kinds of risks

(13:30):
are going to speak to differentexecutives to different extents.
You sort of switch up, as itwere your sales pitch, depending
on who you're talking to.
My general experience is, itnever pays to go in there as an
activist. "It's the right thingto do. We're going to do only
the right thing. It's all aboutprivacy and respecting people's

(13:50):
privacy, and giving them theability to control what data we
collect about them and what wedo with that data. You know, we
need informed consent. We needdynamic consent...." So, even if
you're right, it's going to be ahard sell because the truth of
the matter is that you'retalking to people who have other
kinds of priorities, other kindsof things they need to think
about, including just frankly,straightforward bottom line
concerns that they'reresponsible for. And if they

(14:12):
don't hit their numbers,literally or metaphorically,
then their job is in peril. So,trying to figure out how your
privacy agenda gels with theiragenda is crucial, just like any
kind of negotiation orcollaboration.

Yeah, obviously, that makes sense. How can we get
customers to trust us more,which will translate into a
stickier customer that buys morefrom us or really going further
to explain how privacy is goingto enable the business as
opposed to just be some add on -add a box of privacy to
something. Right? It's kind ofabout baking it in.

Yeah, I put it slightly differently. I mean,
most people put it the way thatyou did. It's a perfectly fine
way of something like how do weget their trust, how do we
increase their trust? And to me,I don't know why (I do you know
why), but I focus on thenegative. Let's not cause
distrust. Right? Because, aleader could be like, "Yeah,
that's good. Increased trust isgood. That's a really nice thing
to have." But, while increasedtrust is "nice to have," loss of

(15:09):
distrust - avoiding loss ofdistrust - is a "need to have."
Avoiding distrust. So, you know,"Let's do this because
otherwise, we're going to losetheir trust. We will violate
their privacy and thereby losetheir trust. They will stop
working with us. They will shoutabout us on social media. We
might invite regulatoryinvestigation for our practices.
Let's at least define a floorthat we're not comfortable with

(15:31):
and make sure that it'soperationalized."

I think that's actually really brilliant - the
power words there. Because then,it really kind of hangs a
lantern on it. Most executivesfeel like they're building trust
in some way or another. So, Ilike that, where this way you're
focusing on you don't want tolose that. And then, it's like,
what is losing that look like?
What's the metric? How manypeople are leaving? Or, then you
can actually action upon that,and it also seems a little

(15:56):
scarier, in a good way, to keeppeople on task about not wanting
to lose that trust.

Yeah. I always go for avoiding the bad first. We
can strive towards the good; butfirst, let's make sure that we
avoid the bad.

Yeah, I think one of the reasons as a privacy
professional that I've stayedaway from that is that I have
seen - we've had very fewmetrics over the years. It's
getting better now that we movemore into the technical, but
we've had very few metrics otherthan breaches. And then, the
security budget kind of tookthat away from the privacy folks
like 15 years ago, and thatbecame a security magic as
opposed to really about privacy.
And so, we had very few and theones people would really use to

(16:32):
try to move the needle would bearound like, "Oh, you don't want
fines" or "You don't wantregulatory enforcement." Right?
And, I have found over the yearsthat just talking about not
having fines and regulatoryenforcement doesn't move the
needle. Too few companiesactually are stuck with those
giant finds; a lot of them arereally big, huge organizations.

(16:53):
It is not what moves the needle,and what does is really aligning
more with what the businessneeds. You don't want to lose
sales. You don't want to haveyour insurance provider not not
insure you because you didn'tmeet some threshold - like other
things. And, you just gotta getcreative around how to get there
and from using negativelanguage, but I really liked the

(17:14):
way you did it.

Again, I think it varies. So, some people are
going to like to like negativelanguage, some won't. So, figure
out what works for that person.

Yeah. And the org and the culture.

Yeah, exactly.
What you know, one thing I wantto highlight, by the way is, you
know, we talked about thetraining data. So, companies
are, because of AI, companiesare incentivized to collect as
much data about as many peopleas they can so they can get more
accurate models, yada, yada,yada. But, one thing that we
didn't touch upon is the"inferred data" - privacy as it
relates to that inferred data.
Because ML is in the business ofbusiness of making certain kinds

of (17:47):
creating new data. And, that data is often about people and
so that's another significantway in which privacy people
might not have their eye on theinferred data at all, when they
should. Training data it reallyimportant, but what you infer
about people can be just asmuch, if not more than, a
violation of privacy compared tothe original data you collected

(18:08):
about them.

Yeah, I totally agree. Can you expound upon that
a little more in terms of someuse cases you're seeing out
there?

Yeah, I'll give you my favorite toy example. So,
suppose you're atelecommunications company.
You've got lots of data aboutpeople and their whereabouts.
Right? Because you've got celltower data, where their phone
is, et cetera. So, let's supposethat you're an organization and
you have data about where Itravel, where I am, throughout
New York City, which is the cityI live in. So, it's not as such

(18:38):
a violation of my privacy foryou to have that data, let's
say. Let's say that you've alsohave some data that's perfectly
not a violation of my privacy,like the addresses of cancer
treatment centers throughout NewYork City, or the addresses of
therapists throughout the city,whatever it is. And suppose
you've got those two data sets,you throw them into your AI, and

(18:59):
what the AI "notices" is that3:30 pm every Wednesday, Reid
goes to this cancer treatmentcenter - goes to this address,
which is a cancer treatmentcenter or a therapist's office,
whatever it is. So, what you'vedone is you've inferred, "Oh,
Reid has cancer or he's atherapist," or whatever it is.
That's a piece of inferred data.
Let's just say it's true. It'snot, but for the sake of
argument, let's say it's true.

(19:20):
What you've inferred about me isyour knowing that, as it were,
is a violation of my privacyeven though you're violating my
privacy by virtue of any of thetraining data that you used.

Right. That's really important to call out,
especially in the privacyengineering space, where we -
you know, this is a GDPRrequirement, not only a
requirement, but the definitionof personal data includes
inferred data; but, it's greatto have the reminder for privacy
engineers that we're alsotalking about and for datasets,
which could also be biometricdata, too, right?

Oh, absolutely.

Yeah, that makes a lot of sense. What would people
need to know regarding thetraining data and input privacy
and does that effect at all theoutput of inferred data?

I mean, there's lots of things to say about the
kinds of things that you need tolook out for when you're
collecting training data andways in which the collection of
that data may constitute aviolation of privacy. But
oftentimes, it might not be thecase that the people who are
collecting the data are at allresponsible for the inferred
data. And so, it's not just amatter of telling everyone
privacy matters and watch outfor the training data and for

(20:23):
the inferred data. It's going tobe a matter of assigning
particular roles,responsibilities related to the
privacy of data throughout theAI lifecycle, which might mean
different teams are handling ormonitoring the data acquisition
at different stages of the AIlifecycle, and so they're gonna
need different kinds oftraining. So, it might be the
case that the team that'sresponsible for collecting the
data did their job; but itwasn't their job to figure out

(20:46):
whether the inferred data willultimately constitute a
violation of privacy becausethey might not even know. They
might not know if thecommunication is bad exactly
what they're collecting the datafor what it's going to be used
for. Or, a team in anorganization might take that
data that wasn't originallycollected for their intended
purpose, but they found a newpurpose for it. They didn't tell
the team who originallycollected that data about it,

(21:06):
but they better have someone ontheir team who was responsible
for checking to see whether theinferred data constitutes a
violation of privacy.

Yeah, that was actually is a really good point.
And I think, you know, mytakeaway there is in the past,
you know, we talked about thesoftware development lifecycle.
And now, in my shifting privacyleft and my looking closer to
development of how products andservices get made, and making
sure privacy is baked in, we'retalking about the "DevOps
lifecycle;" but then there'salso the "personal data

(21:34):
lifecycle." So there's thelifecycle of how personal data
flows through organizations. Ithink you make a great point
that there's now a lifecycle ofAI training, and it sounds to me
there should be some futureframework or something that
holds all these life cyclestogether, so that you can make
sure that you're, in an agileway, being able to hit all of

(21:56):
the requirements through allthese various life cycles.
Right? So that it's ethical,everyone understands what is
expected. What are therequirements and when in each
phase of each lifecycle?

Yeah, I mean, a lot of you know, a lot of the
work that we do with clients isto make sure that some of them
have a well-defined AIlifecycle. Some do not have -
and I'm talking about Fortune500 companies don't have a
defined lifecycle, and thenmapping that lifecycle to your
RACI matrix so you can definewhat roles are responsible for
what throughout that lifecycle.
At what stage in the lifecycleis really important because most
organizations don't have a gripon that stuff. And then, you

(22:30):
also need processes where, youknow, the one team at Phase 1
communicates the rightinformation to the other team at
Phase 2, communicates to Phase3, etc. So, you've got to assign
responsibilities at each stageand make sure that there's
communication between thoseteams for handing off to other
teams.

Right, Right. Make sense. So, I often
hear in the ethics space,especially around AI about human
in the loop. Can you describewhat the human in the loop
actually means and why you thinkit's necessary to have a human
in the loop with respect to AI?

So, the idea that there's a human loop just means
something like there's somethinggoing on in the world, there's
inputs into the AI, there'soutputs of the AI, and then
there's something that resultsfrom the AI having an output.
Having a human in the loop meansroughly that you've got some
human somewhere in that loop(it's not really a loop that's
more of a linear set ofoccurrences, but then it loops
back into the more data intoAI). The general point is that

(23:27):
something like we have a humaninvolved; the AI is not just
doing its own thing withoutanyone looking. So, for instance
- what's a good example - maybethe AI gives a judge a
recommendation about whetherthey're high risk and deserve a
certain kind of sentencing orwhether they deserve probation
or something along those lines.
Nonetheless, you might think,"Well, we want a human in the
loop," which means it can't justgo straight from AI risk rating

(23:48):
to decision. It has to be AIrisk rating to a judge that
considers that risk weightingalong with other factors, and
the judge making a decision.
That would be an example of ahuman in the loop.

Got it. So not an automated decision making. It's
avoided.

Solely or exclusively automated decision.
Yeah, I do think, in manyinstances - again, this is going
to be context sensitive, so I'mnot going to say you always want
a human in the loop. There arecases in which it's not good to
have a human in the loop, evenreally ethically scary cases or
gross cases. Suppose you're inmilitary and you have the other
side using AI in a variety ofways, making phenomenally fast,

(24:25):
"decisions." Suppose you've gotAI as well, but you want to make
sure that you always have ahuman in the loop so that when
your AI responds to their AI,before your AI responds, there's
a human there to validate orassess the outputs of your AI.
Well, that can be reallydangerous because you're slow.
Humans are really slow.
Meanwhile, the other side ismaking more and more decisions,
doing more and more stuff. So,you're going to be at a place in

(24:47):
that particular instance wherethere's no time for human
decisions because the opposingparty made the case that you're
just too slow now for theirtechnology. So, you could put a
human in the loop but that mightlead to absolute disaster.
On other cases, you better havea human in the loop, or that
will lead to ultimate disaster.
So, whether to place the humanin a loop, and where to place
them, and how they interact withthe outputs is all going to be

(25:10):
context-sensitive and it'scomplicated.

Yeah, I think it's fair to say that the instances
where you would not want a humanin the loop are probably the
outliers, or I should say theexceptions to the general
thinking. If someone's bringingsomething to market, for
instance, versus military; but Iotherwise agree. Again,
everything's context-dependent.

This is another, not corporate example, but
criminal justice. So, I had aconversation with a professor of
law who thinks that, plausibly,we are either at or will soon be
at a place where AI makes betterrisk judgments about, say,
criminal defendants than peopledo. And that's not surprising,

(25:52):
he says, when you realize justhow bad people are at judging
the risk of other people. We'rereally bad. So, since we're
really bad at it, it's not thatdifficult for an AI to be better
than us because we're bad atstuff. I mean, we're, you know,
limited beings. We've got thesefinite brains. We're tired.
We're hungry. We'resleep-deprived. We're stupid or

(26:13):
irrational or we have ulteriormotives, but you know....
Yeah, we're kindof a mess. So, there might be

...we're distracted....
loads of instances in which anAI systematically performs
better than, in some casesbetter than the average, in some
cases better than the best. Andit might be the case that you
have a human in the loop, andthey just screw it up because ML

(26:37):
was in the business ofrecognizing phenomenally complex
patterns that people can't see.
In some cases, those patternsare really there, and when the
person says, "No, no, no, no,that person doesn't have cancer.
The ML is wrong. The AI iswrong!" Actually, that's the
human expert that's wrong. Theydon't see the pattern as it were
that the AI sees. In otherinstances, the AI has gone
haywire. And, "recognizes" thepattern that's not actually

(26:58):
there, and the expert has rightto step in. So, how we interact
with these things is reallycomplicated and it's not a given
that humans are usually betterat it than the AI. Often, maybe
for now. 5 years from now? 10years from now? They get dicey.
Interesting. Yeah.
Well, definitely. You know, weall have front row seats to see
how that's gonna turn out. I'vebeen watching play out in my

(27:20):
LinkedIn feed as a privacyethicist - I've been following a
lot of AI ethicist and I have alot of the AI ethicists in my
feed that are pushing backagainst LLMs and maybe even
potential for a lot of risks;and they're explaining how
there's a lot of risks aroundbias today - that we really need

(27:41):
to put guardrails around things.
But, at the same time, there'sso many calls from industry to
move forward with certain AItechnologies, certain new
companies. I mean, it's like acrazy hype cycle right now. But
then, you've also got the AIethicist who are trying to
verbally corral that and makesure that we've got guardrails.

(28:01):
So, I guess the question I wantto ask is, how do we avoid AI
ethics fatigue because we wantpeople to take action, but how
do we do that so that peoplearen't like, throw their hands
up and say, "I'm tired of this,I don't know how we can ever
address the problem" and thenavoid it?

Okay, so there are a couple of things to say
here. So one is that, usually,when you hear people in the AI
ethics space who are railingagainst LLMs, they might be
doing a couple things. The onethat might be saying, stop
paying attention to this'godfathers of AI' who say that
this is going to lead to anexistential threat, or AI does
pose an existential threat, whenwe have to address that. Don't

(28:40):
pay attention to them becausethat stuff is BS; it's not going
to happen; or it's so far offand we have real problems here
today, right here right nowlike: biased AI,
privacy-violating AI, etc.
That's sort of one camp. I'mrelatively sympathetic to that
line of thought.
You then have some people whoare either genuinely concerned
about the existential threatstuff, or they're very concerned

(29:03):
about the economic impact of jobloss from AI, if that's going to
happen, they think that's goingto happen. You also have some
people who are really worked upabout the ease with which
misinformation / disinformationcan be created and scaled using
generative AI. Those are alllegit concerns - well, the
misinformation is certainly alegit concern. Job loss -

(29:23):
possibly. I think it's too soonto tell. But, I don't think that
any of those concerns -theexistential threat one or the
job loss one or themisinformation one - is the kind
of thing that the vast majorityof corporations need to do
anything about, or even can doanything about, including the
job loss one because I don'tthink we could talk about this
if you want, but I don't thinkit's the responsibility or the
obligation of businesses to hirepeople, or an obligation to

(29:45):
maintain people when they canwhen they have more efficient
means by which they can do theirjobs, do their work. But
,another thing to say is thatthere are some ethical risks
that I think are particular tothings like LLMs that
enterprises should pay attentionto. So here, I'm thinking about
things like the so calledhallucination problem, which is
that LLMs output falseinformation; what I call the

(30:07):
deliberation problem, whichmeans it looks like MLMs are
deliberating and giving youadvice for good reasons, when in
fact, they're not; what I callthe sleazy salesperson problem,
which is, you can make reallymanipulative chatbots; and the
problem of sharedresponsibility, the fact that
there's a small set of companieswill make foundation models like
LLMs and then downstreamdevelopers, or enterprise or

(30:29):
startups or whatever, tweakthose models or fine tune them.
And then there's a questionabout who's responsible things
were ethically sideways? Thosekinds of concerns really do
matter quite a bit. Oh, and thenyour question was, how do we
avoid ethics fatigue?
Yes. AI ethics fatigue.
There's a number of things. Itdepends on what the source of
the fatigue is, of course, butif the if the source of the
fatigue is people screamingabout job loss and an

(30:51):
existential threat, et cetera,then the fatigue is, "Hey, don't
worry about that stuff. That'snot for you." If you're Kraft
Heinz, if your U.S. Bank or JPMorgan, you're not going to
solve for the allegedexistential risks of AI. So, one
thing to stop the fatigue is tomake sure that corporations
focus on the risks that actuallypertain to their organization.

(31:12):
Another source of fatigue that'soften cited is not the set of
risks or the kind of risks, butthe frustration with trying to
translate ethics principles intoaction. So, people talk about
justice and transparency, andprivacy included. "We're for
privacy. We respect people'sprivacy!" And then no one knows
how to operationalize thosethings. I should say no one -

(31:32):
lots of people don't know how tooperationalize those things.
They conclude from that, "Oh,ethics is just so fluffy. We can
operationalize it. I'm tired ofall this talk about ethics. We
need to implement stuff,solutions. What does that look
like?" And in that case, it'snot the solution to the fatigue;
it's to actually do the ethicswell. It's not to abandon it.
The problem is that you're atsort of layer 1 of 1000 of how

(31:54):
to think about ethical risks ofAI, and you've got to go deeper
if you want to actuallyunderstand what you're dealing
with here if you want tounderstand your problem. So,
design the appropriatesolutions.

Yeah. And in fact, I love this. It's not a direct
quote, but from one of theappearances you had on I think
someone else's podcast, you hadsaid something to the effect of
when you articulate a value onthe front end, you must explain
a guardrail that you'veimplemented to effectuate that
value. For instance, in yourprivacy policy, or in your if

(32:24):
you have an AI ethics policy orwhatnot, if you just say we love
privacy, we respect yourprivacy, but you can literally
show nothing, you're justrepeating hot air. You're not
doing anything and taking anyaction. You're just placating.

Yeah, exactly.
Whenever I work with clients,and we start working on what
their values are, how they wantto write their AI ethics
statement, as it were, we're notallowed to just say "We're for
privacy. We're for fair and justoutputs and against biased AI."
What the hell does that mean?
You've got to put some more meaton the bones. And one way is to
articulate what red lines, thosevalues create. If you value X,

(33:02):
does that means you'll eithernever do Y are you'll always do
Z, something along those lines.
So to take one of my go-to toyexamples, iff you just say we're
for privacy, it doesn't meananything. If you say, "We
respect people's privacy and sowe will never sell user data to
third parties." Okay, that'ssomething. Not all companies
will sign up for that. Somewill. Some won't. There's more
to say, but that's at least aguardrail. Now everyone knows,

(33:24):
and you have to talk about howto operationalize the
guardrails, but at least inprinciple, you can sort of see
how that gets done roughly.
Okay, I understand now what theymean when they say they value my
privacy, it means they're notgoing to take my data and sell
it to a third party. Now, Idon't think it should end there.
I think there should probably be3 to 7 guardrails per value
because we value privacy, we'llalways X; we will never Y; we'll

(33:46):
always Z. You know, those setthe guardrails. If you have you
know, 3 to 7 - let's call it 5 -let's say got 5 of those per
value, let's say you have 5values, that's 25 guardrails of
action, already from the startjust from, you know, from your
statement. And now, you can seethe road to implementing such
guardrails. You can measurecompliance with those

(34:07):
guardrails. And, when peopleread about these things, it's
now a much more crediblestatement than, "We love your
privacy."

Exactly. Makes sense. So responsible AI, or
ethical AI? What's thedifference? Is there any? And
then why does it why docompanies try to avoid the word

Well, they try to avoid the word ethics because
ethical?
they don't know what to do withit, because they think it's
squishy or subjective orsomething like that. And so,
they abandon it. I think that'sa mistake, although it does
explain why I primarily speak interms of ethical risks and not
just ethics. So I'll say, we'regoing to build an AI ethical
risk program, as opposed to anAI ethics program, because

(34:45):
businesses could at least speakand understand the language of
risk in a way that they don'tunderstand the language of just
ethics. That's one thing to say.
I have been talking about AIethics and AI ethical risks for
a while now. What I've seenhappen is that evolve into, as
you've said, corporationstalking about 'responsible AI.'
For some reason, they're morethey're more comfortable that
are responsible and they arewith ethical. But, what I see

(35:08):
gets done when people use theword or the phrase responsible
AI is it encompasses a wholebunch of stuff. Yes, we're for
responsible AI. That includesthe ethics stuff, but it also
includes regulatory compliance,say with the GDPR or CCPA. It
includes regulatory complianceand of course, legal compliance
more generally. It includesengineering excellence, like

(35:29):
model robustness andreliability. It includes
cybersecurity practices, youknow, best practices for
cybersecurity or something alongthose lines. So, it's a bucket
of stuff. Its security. Itsregulation. Its ethics stuff.
It's engineering excellence. So,it's a bunch of things. And
then, when they start buildingout that responsible AI program,

(35:50):
they don't really know what todo with the ethics stuff. So,
they focus on regulatorycompliance and cybersecurity and
engineering excellence, and theethics stuff gets short shrift.
Usually, it'll just be a focuson bias, as though that were the
entirety of AI ethics or AIethical risks, and they carry on
building this responsible AIprogram that ignores the vast

(36:12):
majority of the ethical risksbecause they don't know what to
do with it. So, I thinkresponsible AI is a fine thing
to do. If you want to use thephrase, nothing intrinsically
wrong with the phrase. What'sproblematic that use of the
phrase has led people to ignoreethical risks and pay more
attention to the things withwhich they're already familiar,
which is, that's a problem.

You threaded some things together for me there
that really like underlines thepoint. So, thank you for that.
In fact, to extend on thatpoint, I read your recent
article, "Generative AI-xiety,"which was published in Harvard

(36:46):
Business Review, I think 10 daysago or a week ago, something to
that effect, and that walksbusinesses through AI risks as
you see it. You mentioned before- you list four main areas that
people should pay attention to.
You mentioned them before. I'mjust going to say them again,
just to make it clear that theseare the four areas you really
feel like are the main risks,and then if there's anything you
want to expound upon, I'd loveto hear it. The first is the

(37:08):
hallucination problem. Thesecond is the deliberation
problem. The third is the sleazysalesperson problem. And four,
the problem of sharedresponsibility. Do you want to
go into little more depth onthose?

Yeah, sure. So, one thing I want to highlight is
that I think those are the, ifyou like, "the Big Four" that
pertain to Generative AI inparticular.

Oh, yes. I'm sorry. Yes. Generative AI.
bias, privacy, explainability.
Those are still around andthey're not going anywhere. They
apply to Gen AI just as much asthe non-Gen AI. But, those are
the big four for Gen AI, Ithink. Yeah, I could talk about
those. I mean, people know thathallucination problem, LLMs
output false information. Onething that I tried to highlight
in the article, though, is thatthe problem is not merely the

(37:51):
fact that it outputs falseinformation. It's that people
are lazy and they suffer fromautomation bias, and so they
tend to believe the output. So,you can tell someone, "Hey,
listen, this thing, you know,sometimes outputs false
information." But, it's reallyeasy to rationalize, like, "I
know what this thing outputsfalse information, but the
answer that it just gave meseems really reliable...it seems
Great. And then the deliberationproblem?

(38:12):
true. That makes sense. Thatresonates. I don't think this is
one of those cases in which it'shallucinating. This seems right.
It's easy to make thatrationalization when: A) we're
already inclined to defer to theoutputs of machines, because for
some reason, we think thatthey're more reliable because
they're mathy, or something likethat. So ,we suffer from
automation bias. We're lazy. So,having to actually verify the

(38:33):
outputs takes a lot of work,that or we just think that it's
already got it right. And, wemight just want a quick answer.
Doing research takes time. Thisthing just gave me an answer.
That seems good. All right, I'mjust gonna go with that. So,
it's not merely the fact that itoutputs false information. It's
that we humans are bad at reallychecking our own biases against
- biases in favor of it, andmaking sure that we do our due

(38:54):
diligence. That's thehallucination problem,
The deliberation problem isharder to see. But I think it's
a pretty significant one. So,let's say you're in financial
services, and you create a anLLM that gives financial advice.
So, let's say you're interactingwith an LLM that goes financial
advice and say, "Hey, listen,here's who I am, blah, blah,
blah," LMMs don't deliberate.
They don't weigh pros and cons.

(39:17):
They don't give you advice basedon reasons. What they're doing,
in all cases, is predicting thenext set of words that is
maximally coherent with thewords that came before it. It's
a mathematical thing, right? So,when it gives you that
explanation, it's not actuallytelling you the reasons that it
came up with the output that itgave you previously. It's more
like a post facto explanation -an after-the-fact explanation,

(39:41):
where it spits out words thatwould look like it explains what
happened before (that cohereswith the words that came before)
- it doesn't actually explainwhy it gave the output that it
that it did. So, you might thinkthat it's deliberating and then
giving you the contents of thatdeliberation, when in fact that
is not doing any of thosethings.
So basically, it's not thinking. It's perceived

(40:04):
follow up answer could becompletely another fabricated
response or it is fabricated.

Yeah, exactly.
Yeah, can be completelyfabricated, especially if it's
in the realm that requiresexpertise. Right? It produces
some seeming reasons for whygive that output that look to
reflect an appreciation of thecomplex world of financial
services or financial advisoryor something along those lines.
And, to an expert, they're gonnabe like, "Yeah, that sounds
good," but they don't know.
They're not experts. That's thewhole reason they went to the
LLM in the first place. And,problem is compounded by the

(40:36):
fact that LLMs often evince itsrecommendations and so-called
explanations quite confidently.
It's easy to be taken in bythat, even though you did well
to overcome the problemsassociated with the
hallucination problem by saying,"Yeah, I don't know about this.
Let me dig deeper. You then digdeeper, and now you're giving an
explanation - that's not reallyan explanation - for why it gave

(40:56):
you the advice that it did, andyou can be taken in by that.

Right. It's just more and more risks compounding
one another. Okay, the third onethat we should pay attention to
for LLMs - the third risk is thesleazy salesperson problem,
which you did touch upon before.
But if you could give maybe ause case.

Yeah, this one is easier to get a grip on. It's
just the fact that you can finetune your LLM. You can further
train your large language model,your chatbot, to interact with
people in various ways. So, forinstance, you can say, "Go read
this negotiation book, and thenengage in conversation with
customers in a way that reflectsthe lessons or the teaching of
that negotiation book, or ofthat book on how to manipulate

(41:35):
people - that book on how topress people's emotional buttons
so that you get the sale."Right? And so, then what you're
doing is you're training yourchatbot to be manipulative in
various ways. And so, then itgoes out, talks to your clients
or customers, and negotiateswith them in ways that are
perhaps sleazy, manipulative,ethically objectionable, and
that result in loss of trust.
That can easily happen.

Got it. Thank you.
And the fourth risk is theproblem of shared
responsibility; and, I thinkthat one is particularly
interesting to my audience, ifyou could give a little more
detail there.

So, the issue there is, let's say it's OpenAI
or Google or whomever develops aLLM, a foundation model. They
sell it to whatever, yourorganization - healthcare and
life sciences, financialservices, whatever your industry
happens to be, they're sellingit. And now, your engineers,
they fine-tune the model fortheir particular use case for
whatever you're trying to buildin-house. So you're building on

(42:31):
top of that foundation model.
Now, if things go ethicallysideways - who knows what
happens maybe it starts givingreally bad recommendations to
customers, gives recommendationsthat are bad internally that
were supposed to increaseefficiency when instead of
decreases efficiency, who knowshow it goes sideways, but it
violates people's privacy inways that you didn't anticipate,
so on and so forth. And now yousay, "Well who's responsible for
it's going sideways? Is it thepeople who built the foundation

(42:55):
or the people who built on topof the foundation? Is it the
organization from what yousourced that LLM? Or is it your
organization?" And so there's aquestion about how do we
distribute responsibility whenthings go ethically sideways?
Generally, what I think here isthat there's a question, and
it's an empirical question on ause case basis, which is, did
the foundation model companygave the downstream developers

(43:18):
sufficient information such thatthey could do their due
diligence - they had enoughinformation such that they could
check that this thing is safeenough to deploy?

What does that look like?

Well, that's gonna vary by use case. I mean,
one thing is that you might needto know... For instance, let's
just take a sort of an abstractexample: "Hey, listen, in order
for us to really do your duediligence here, we need to know
a lot more about the trainingdata that you use to train this
LLM."

Oh, I see.

And then they say, "Nope, we're not going to
give that to you." Okay. If theydon't give it to you, let's say
it's Open AI, and say, "Look, wetell you what kind of training
data we've got. That's our IP.
We're not giving that to you.
You're not allowed to knowthat." If your organization
says, "Okay, well, we need it inorder to do our due diligence to
make sure that this thing issufficiently safe to deploy, but
they're not giving it to us, sowe're gonna move forward anyway.

(44:08):
Well, in that case, yourorganization certainly bears a
lot of the responsibility whenthings are ethically sideways
because you chose to go forwardknowing that you didn't know
enough to do your due diligence.

I see a lot of that happening right now as
people are like guinea pigs inthis new world of LLMs.

I think this gets really complicated because,
probably I could be dissuaded ofthis, but I am inclined to think
we need a regulation around thissort of thing. Suppose that you
are the government. If you'vegot any function, if the
government has any functions,it's to protect people - to
protect its citizens. If it hasany function, it's one of
protection. Suppose thegovernment says, implicitly or

(44:47):
explicitly, to the foundationmodel builders, suppose the
foundation model builder say,"Listen, it's 'buyer beware.'
We're going to build our stuff;there's a lot of software to
keep secret; and it's buyerbeware. If you want to buy it,
and then fine tune it, and thenuse it, and things go sideways,
that's on you - that's not onus. What's going to happen is
that the companies with thelargest risk appetite are going

(45:07):
to be the ones who use it. So,of course, what are we going to
see? We're going to see peoplehurt We're going to see people's
privacy violated. We're going tosee people discriminated against
- all the bad stuff. That's areally bad outcome, if you're
government, right, to allow amarketplace in which only the
riskiest players are going toplay. That's bad. That's bad for
society. So, it looks like weneed certain kinds of

(45:28):
requirements around foundationmodel companies disclosing
enough information to companiessuch that they can do their due
diligence. And if they won'tdisclose it to Company A, let's
say in financial services, andthen they can't also sell it to
Company B, and not give thatinformation.

Right. That definitely makes a lot of sense.
I guess that's a big part ofresponsible AI is being able to
explain it, too, and haveexplainable AI. So, I guess, as
we're getting to the end of thisamazing chat, I'm definitely
going to put in the show notes alink to your book, Ethical
Machines, which I know is forbusiness leaders who are

(46:06):
thinking about incorporating AIinto their organizations. What
I'd love for you to do is giveany advice you have for
technical staff, so like datascientists, architects, product
managers, devs, when they'retrying to build products and
services that leverage LLM's ina responsible manner. It could
be high -level advice, but whatshould technical staff think

(46:28):
about as they're bringing thistech to market?

If you're a privacy engineer, then you know,
one thing to think about is whatare the various ways that we're
going to need to protectpeople's privacy? How are we
going to ensure, for instance,that when people throughout our
enterprise use it, how do wemake sure that we're not loading
into it or not putting into theprompts sensitive data, personal
information, personal healthinformation (if it's a
healthcare company), sensitivecorporate data.... One thing to
think about is how do we makesure that we educate people

(46:52):
throughout the organizationabout how to use this stuff.
There's also questions thatprivacy, people need to think
about with regards to whetheryou want your people using LLMs
in particular by way of an API -e.g., to an open AI or to a
Microsoft or to a Google, orwhether the LLMS should live on
premises, whether should be onyour own server, so no data, no

(47:13):
prompt data goes out to anyother company. So, thinking
about how that data gets handledis going to be one of the big
things. The truth is that ifyou're a more junior engineer,
the best thing you can do isjoin with others to push for
stronger, more robust governancefrom the top down.

Yeah, you know, as you were talking, I was even
thinking to myself, Oh, okay, soit's, this is more data
governance. It's just datagovernance for AI, and you can
add to the data governanceprocesses for one organization
that they've got set up already,or that they're putting in place
(or that they should have inplace), it's not like an
independent thing you now haveto do this for AI. There's
probably processes that alreadyexist for data governance that

(47:55):
you can then make sure the AImodels, you know, and then the
whole lifecycle for AI isincorporated into the data
governance.

Yeah, that's exactly right. So, that's pretty
standard project for us to do isto do an analysis, you know a
gap and feasibility analysis ofan organization as it stands
now, with regards to AI ethicalrisks. One thing we're certainly
looking at are their datagovernance structures,
processes, policies, workflows,RACI matrices, etc. to see what
is there that we can leverageand just augment such that it

(48:25):
includes AI ethical riskmanagement, as opposed to
building things from scratch.

Right. You always want to reuse what's already in
the org that's working. Is thereanything else you'd like to
leave the audience with? Whetherit's something inspiring or a
tip or go to XYZ conference?

Yeah, I don't know. Inspiration - I think I'm
bad at inspiration. I'mnot...it's not my, you know,
motivational speaking, is not mything. One thing to walk away
with when it comes to theethical risks of AI is to
understand that it's simply nottrue that we don't know what to
do about these things. Somepeople like to say, "Oh, we got
to do more research. We've gotto look into this. Everyone's
trying to figure out what to do.
It's the Wild West. No one knowsthe answers." And it's sort of
like, "Well, no, that's notreally true. There are some

(49:07):
people who know the answers, andpeople who have been working on
this stuff for a long time. I'mnot saying people like me know
all, but it's simply not thecase that we don't know what to
do here. If your organization isnot doing things, it's not due
to sort of humanity-wideignorance; it's due to a lack of
political will within theorganization, which means that
the thing to do is to figure outhow can we generate that

(49:28):
political will to engage inappropriate ethical risk
mitigation.

Awesome. Well, thank you so much for joining us
on Shifting Privacy Left today,and you're discussing your work
with ethical AI and overlappingprivacy issues. Hopefully, we'll
have you back in the future.

Sounds great.
Thanks for having me.

Until next Tuesday, everyone, we'll be back
with engaging content andanother great guest.

All Episodes

Episode Transcript

Popular Podcasts

Dateline NBC

24/7 News: The Latest

Therapy Gecko

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}S2E26: "Building Ethical Machines" with Reid Blackman, PhD (Virtue Consultants)

Episode Transcript

Popular Podcasts

.css-r6mb8g{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;}Dateline NBC

24/7 News: The Latest

Therapy Gecko

All Episodes

S2E26: "Building Ethical Machines" with Reid Blackman, PhD (Virtue Consultants)

Dateline NBC