S2E28: "BigTech Privacy; Responsible AI; and Bias Bounties at DEF CON" with Jutta Williams (Reddit)

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jutta Williams (00:01):
Sometimes in the privacy engineering space we
get so wrapped up in kind of thetechnical controls to face, and
it is just not sexy to work onthings that are knowledge-
enhancing, but that's really amissed opportunity.
I love education, training,policies, standards, guidelines.
I mean it's not sexy work, butthese are force multiplying
tasks.
Start with knowledgeenhancement and it will take

(00:24):
your program farther, faster.
I love this approach" to createcuriosity, create engagement,
have some fun.
This doesn't have to drearywork.
Have some fun and people willfollow.

Debra J Farber (00:42):
Welcome everyone to Shifting Privacy Left.
I'm your host and residentprivacy guru, Debra J Farber.
Today, I'm delighted to welcomemy next guest, Jutta Williams,
Head of Privacy and Assurance atReddit.
She's also co-founder of HumaneIntelligence and BiasBounty.
ai, and privacy and responsibleAI evangelist and startup board

(01:07):
advisor.
She has a Masters inInformation Security Policy and
Management and graduated withdistinction from Carnegie Mellon
University; and, she considersherself a 'Recovering Chief
Privacy, Security, andCompliance Officer, which I
might ask her about.
Jutta has a long history ofaccomplishments in privacy
engineering and has a reallyunique perspective on the

(01:30):
practice of privacy engineering.
That's because she's had rolesat 4 major tech and social media

companies (01:36):
Google, Facebook, Twitter and now at Reddit.
So, a few weeks ago I finallyhad the pleasure of meeting
Jutta in person at DEF CON - andwe'll go into how that meeting
went in a little bit - and thenthere, I volunteered at the AI
Village and its launch of thelargest Generative AI Red

(01:58):
Teaming Challenge, a 'biasbounty' that Jutta co-designed.
I cannot tell you how psyched Iam to have you to join us for
this conversation today.
So, let's get started.

Thank you so much for hosting me and, you
know, longtime listener,first-time participant.
I have to say this is an honoron my part for sure, and I'm
super excited to kind of jumpinto a fun conversation.
And also, I just wanted to say,"Thank you so much up front.
You worked tirelessly at theDEF CON event, where none of us
expected to work quite that hard.
It's kind of like duckssmoothly sailing across the top

(02:33):
of the lake but those littlefeet were sure pedaling
underneath the water.
So, thank you so much.

It was my pleasure.
I roped my fiancé in as well.
He's the one that got me goingto DEF CON for so many years,
and it was honestly reallyfascinating.
I'm glad I got to have a seatat the table to kind of check
out the challenge and to reallybe there to help support the
community.
So, thanks for putting ittogether and I look forward to
helping again next year andencourage others to do so too.

(02:58):
You have such a uniquebackground and I know my
listeners would love to knowwhat got you interested in
privacy engineering and how didyou make that transition from
security engineering intoprivacy.

Wow.
You know, I find that everyperson I talk to has such a
unique and interesting journeyto this field.
It doesn't seem like anyonecomes through a very
straightforward path.
So for me it was an unintendedcareer choice.
When I went to undergraduateschool, I decided I wanted to be
an assassin for the CIA.
Actually, no, just kidding.
I wanted to be a lawyer.

(03:33):
The first class is that - Imean, I became a Political
Science undergraduate with anemphasis in International
Relations and Turkish language,of all things.

So, obviously it makes sense that you ended up
as a Privacy Engineer.

100% Right.
But, I also fell in love, gotmarried to the military, and had
to move overseas; and, the onlyjobs we could legally hold as a
dependent in Turkey wereworking for the U.
S.
government - and there wereonly four jobs.
They were all low level GS5 andGS6 jobs.
I got to work as an InformationAssurance - and what was it
called - oh, it was some reallylame title, but it was basically

(04:12):
helped us support with a lot ofadministrative responsibility.
Office Automation Specialist -that's what it was.
So, you know, I got a clearanceand I got to work in technology
and one of the earlier phases oflike advancement in DO D.
You know, the first people inDOD who had computers on their
desks were admins.
So, we were, indeed, data entrypeople and if you look across

(04:32):
the spectrum of who's risen tothe top in D O D spaces, a lot
of people came out of theadministrative pool that are in
really senior positions today.
And anyway, technology becamekind of my path from there on
forward.
So, I moved back to the Stateswith a clearance and technical
experience right in time forInternet 1.
0.
So, from there on, I was atechnologist and I started as a

(04:52):
Help Desk Support Tech.
Then I became a SystemsAdministrator, the Network Ops
Manager, and eventually I landedin a job that was just
incredibly humbling as aResearch and Development person
working on crypto technologydevelopment.
Because obviously, right?

Yeah, obviously.
So, you know that's the jumpingoff point into privacy
engineering.
I see it now with thecryptography overlap.

Building public key infrastructure and some of
the really cool technologiesthat evolved from PKI in D O D
was super special and exciting.
It was around 2002, 2003.
And privacy- enhancingtechnologies really starts to
get one's juices running aboutwhat are the long term
consequences of using privacy-enhancing tech?
What are the long- termconsequences of using technology

(05:35):
that keeps information secret?
And I had the experience ofsitting in the room - I wasn't
important enough to be at thetable, but I was sitting in a
room - when we had a PrivacyBoard discussion about privacy-
enhancing technologies and itsimplications for the world; and
it was just so exciting and sohumbling that I was like, That's
it! This is my calling.
"When I applied to one school, my

(05:58):
graduate program, because I wasgoing to only go one place and
it happened to be CarnegieMellon because there was a whole
lot of really cool academicwork happening in the field of
privacy - it was at the time ofLorrie Cranor and the
k-anonymity work.
I had the incredibledistinction of having Alessandro
Acquisti as my Graduate Advisorat Carnegie Mellon, and we got

(06:18):
to do some really interesting,cool technology development
work, but also kind of advancethe school of thought around
privacy and the commoditizationof personal data.
So, from there I was kind ofhooked on privacy.
Even though I worked insecurity engineering roles -
because that's what existed atthe time - it was always with
this bent toward not necessarilyprotection from external

(06:38):
threats but figuring out ways tomake sure that the internal use
cases were also appropriate andthat there was transparency.
So that was kind of how I gotfrom tech and non- tech into
privacy.

I love it.
That's such a great story andobviously unique.
Everyone who kind of comes intosecurity and privacy typically
has a fascinating story as tohow they got into it.
I also wanted to point out thatyou and I both worked at the U.
S.
Army - I think we've overlappedfor one month.
I was only there for two monthsas an intern for the Army JAC

(07:14):
Corps summer internship program,but I think that's just amazing
.
We worked at the same companymany years ago - or organization
, not company.
It really is a small world.
Amazing.
Okay.
So, you've worked, as Imentioned before, in several of
the BigT ech / FAANG companies.
I know we can't call it FAANGanymore because Facebook is now

(07:36):
Meta.
But, I don't know, MAANGcompanies.
And, Twitter is now X.
Oh my gosh, I didn't even thinkof that.
Well, Twitter was never on theFAANG.
Oh, right.

There was no 'T.
'

Yeah, maybe we throw out the acronym altogether
, who knows, but you worked in aprevious FAANG environment for
4 different companies in PrivacyEngineering roles.
So let's, if you wouldn't mind,let's compare the privacy
cultures and just generalapproaches at each company.

So, my first act career was the government and
then my second act career was alittle bit of consulting, but
mostly healthcare.
So, the healthcare culture - andI was there for, gosh eight
years maybe - it's a littledifferent because there's like
this culture of caring aboutprivacy.
It's considered - every personwho works in healthcare kind of
understands that their extensionof the doctor- patient

(08:28):
confidential promise; and, as aresult, you don't fight
culturally to really implementand build really great programs,
in my opinion.
So, when I moved fromhealthcare to Google, I was
really surprised that that isnot an embedded cultural truth.
That data is just kind of fuel.
"It's like electricity, asAndrew Ng said about data and ML

(08:51):
.
It's part of the valueproposition, but it's not by
itself held in the same esteem,in my opinion, as it was in
healthcare; and that was alittle shocking.
So, when I landed at Google, Iwill say that it was a bit of
culture shock because I wentfrom an equally large sized
company - my first healthcaresystem where I was the Privacy
Officer was 130,000 employees,and that's about how big Google

(09:12):
was - but, you know, they putextra zeros on the number of
users, and so scale wascompletely different.
The intimacy of the data wastotally different.
We didn't have nearly as muchdata as, say, you would have at
a 30- year longitudinal healthrecord.
So, everything was justdifferent, and also people
didn't see data the way I sawdata and that it came from a
person, that it belonged to aperson.

(09:34):
This was just bits and bytesthat were useful in creating
really huge scale technology atthe time.
I'd say that over the course ofthe couple years I was at
Google, that shifted and changedquite a lot.
GDPR came to be; there weremore regulations; there was a
lot more public feedback aboutprivacy as an inherent right and
that people were concernedabout their privacy.
So, culturally, things shiftedwhile I was at Google.

(09:56):
But, I'd say at the beginning,I didn't feel like they have the
same focus on where data comesfrom as I did, but maybe more so
toward the end.
Now, Facebook is a totallydifferent ballgame.
When I moved to Facebook, datawas respected, but it was
respected for what it delivered;and it delivered huge margins
and growth and an opportunity toreach lots of people.

(10:16):
So, again, totally differentpoint of view.
Instead of being innovation-focused and how can we use this
data in super innovative ways,it was how do we grow big and
fast?
That was a very differentculture, too.
It was also harder to implementprivacy change, in my opinion,
because if you affected one ofthose two top line metrics, it
was a really uphill slog.
Then, I moved to Twitter, andTwitter is a different position.

(10:40):
I was there as a MachineLearning Ethicist.
We were kind of the machinelearning ethics, transparency
and accountability groups.
So, we were looking at thelong-term consequences of using
large datasets to deliverservices and to make decisions
at scale.
So, it was a different role anddata had a different use case
and purpose, which was to createpersonalization.

(11:02):
So my job was a littledifferent.
I'd say that Twitter was adifferent culture too.
It was very methodical, it wasvery thoughtful, and I would say
that it was slower as a result,but I also think that we use
data differently.
It was also a very legally-driven privacy program, so legal
liability and regulatoryliability was first and foremost
thought, even more so thaninnovation.
So, a different culture also.

(11:23):
I left before 'Elon year,' so Ican't speak to how it is now,
but I thought Twitter's approachto data privacy was good.
But, it actually impacted andharmed, I would say, innovation
as well because it was soprotectionist.
Now, Reddit is a totallydifferent company.

In what way was it protectionist?
I'm just curious.

Data use cases were approved in a piecemeal,
very specific fashion.
I'd say that data deletionpractices were really
aggressive, and so data deletionwas impacting a lot of the ML
use cases.
It was just a very differentstructure and it was, again, by
that time in society, there hadbeen a lot of enforcement
actions.

(12:02):
So, I would say that there wasmore interpretation and there
was a lower risk tolerance forsome of the data practices by
the time I got to Twitter, andsome of the legal liabilities
were impacting some of theinnovation decisions.

That makes sense OK.

Reddit's a totally different culture and a
different place.
We don't collect a lot ofinformation.
We don't use a lot ofinformation.
And, I would say that we're notmotivated by only the financial
imperative to use data.
So, I think that also thestructure of the business is
different.
We're focused a little bit moreon community ideation, interest

(12:39):
more than the person.
So, if you look at some of thesocial platforms, it's a little
bit more about "ey, this isJutta, this is what I'm doing,
this is what I like, this iswhat I look like, this is these
are the things I think.
And at Reddit it's a little bitmore about the idea and the
conversation than it is aboutthe person.
And, as a result, we just don'thave that much information
about people and we're notreally looking to get it.
I think that it's just adifferent platform.

(13:01):
It has a different purpose anda different use case and for me,
it gives us an opportunity toreally do things differently.

Do you find that the culture at Reddit that you
just described makes it - I mean, do you just have less of a
scope, like less on your platebecause you're not covering as
much personal data, for instance?
So are you basically able toget to some pet projects or some
really interesting deploymentsthat maybe in another company's

(13:28):
cultural environment youwouldn't have had the
opportunity to get to because ofcompliance?

Yeah, we haven't slid down the slippery slope,
right, as it were.
I think that it changes theconversation a little bit about
how do we stay true to ourselvesand who we are while trying to
accomplish some of the biggerpicture things that other
companies and our peers areaccomplishing.
I think that it requires us tobe a little bit more creative
about how we get to the outcomewithout using the same methods

(13:54):
and means.
It's not terribly hard todeliver a targeted ad when you
have 289 data elements about aperson and a social graph.
But, if you don't have thosethings but you do have a
community that is granularizedthe way we do with subreddits,
can we get to the same place andhow do we do that while still
making advertisers feel likethis is a targeting system and

(14:15):
program that's worth paying for,right?
So it's kind of the creativeways that we get to the same
outcomes while looking at itfrom a completely different
angle.
I think it's possible, but Ijust find myself being a lot
more creative in the way that Ihave conversations.
And then also, staying true tothe principles, like you get to
really double down on theprinciples that we live by.

Yeah, that's what I was thinking is that you
would even have more time to beable to kind of message better
and like focus on, I guess,parts of a privacy program -
like communication andgovernance and maybe more fun
training programs or whatnot,because there's less of a focus
on needing to meet complianceaims, not because you don't have
compliance, but because youdon't have to bring things into

(14:59):
compliance where they weren't.

Fingers in the holes in the dam.
Right?
You're just like, you canstructure things just a little
bit more.
We're an 18 year old startup.
There's pros and cons to bothof those things, but what I
would say is that complianceshould be the byproduct of a
really strong program.
Right?
It shouldn't be the definingreason for a program, but so
many times you're just trying tosolve for these minute

(15:22):
compliance issues because you'reso far down a path that, you're
right - I get to focus a lotmore on building a really
effective program under thedefinitions of framework and
spend a little less timereporting on compliance.
It's just a byproduct.

Love it.
So, is Reddit hiring forprivacy engineering

I keep trying to .
The market keeps shifting andchanging.
I do have two open positionsOne is for a manager position on
the Ops side of my business andthe other is another SME.
We're shifting left, as yourpodcast is called.
We're trying to build moreprivacy engineering projects,
which requires differenttechnical skill sets than some
of the privacy review strategiesof the past.

(15:59):
So, if we're shifting left andwe're building it directly into
technology so it scales and ispersisted, it means more SMEs.
It means more infrastructurepeople.
So yeah, we are, but we'rehiring different people than
I've ever hired before.
Much more on the technical sideof things Data pipelines and
deletion scripts and all thesethings that scale in an

(16:20):
automated way to ensureconsistency.
That's kind of our goal, rightnow.

Amazing.
Well, I think we have some ofthose people in the audience, so
, if you're looking, I encourageyou to go and submit your
resume because then you'd get towork for Jutta, which is almost
as exciting as working forReddit.

With, not for.

So, what advice do you have for security
engineers who may be seeking totransition to privacy
engineering or at least expandtheir roles to take on some
aspects of privacy engineering?

Yeah, the best privacy people I know have a
beginning or an end in the law.
So if you're already anengineer and you want to get
into privacy, you're going tohave to read up on the rules.
A lot of what we do is buildingtechnical standards or
technical expectations rather,based on an interpretation of a
rule or an interpretation ofwhat we see as an enforcement
action in industry.
So it's reading the regulations, learning how to read them and

(17:14):
then understanding enforcementactions and kind of the
approaches that people aretaking to solve for some of
those enforcement expectationsand then being able to translate
those into technicalrequirements.
It's a difficult thing.
So, conversely, if you're anattorney or a lawyer or somebody
who's trained in law and wantto come and be a Privacy
Engineer, you're going to haveto learn the technical ways of

(17:35):
delivering on controlexpectations.
So read up on kind of what's anexpectation.
Some of the IAPP certs are goodones.
There's a couple of great booksout there, but you have to have
expertise in both, and thoseare two very hard industries to
combine into one practical skillset.
So, I'd say learn how to read areg, learn how to interpret it,

(17:58):
learn how to turn that into abusiness requirement and then
you can turn it into a technicalrequirement and then you can
apply your engineering skill set.
Conversely, if you're reallygreat at reading a reg, learn
how those turn into businessrequirements.
Then go educate yourself aboutwhat is a security control, what
is a data protection control,and go get smart on one or more

(18:19):
technologies.
But it takes a lot ofself-learning because it's
really hard to go to school andlearn all this stuff.
Most of it's on the jobtraining or self-study.

There's so much self-study.
I think that just applies tothe privacy profession generally
, especially now.
I mean, first it was drip, drip, drip of people caring and now
it's like in the news on a dailybasis.
It's just fascinating.

Also, you're going to have to be a jack- of-
all- trades because people keeppiling new and interesting
things on top of the privacyspecialty.
So, anytime that there's anethics and a data and a
technology question, it becomesa privacy obligation.

Yeah, do you think that's going to be like
some sort of Chief EthicsOfficer or Chief Trust Officer
or something in the future?

Yeah, I think Chief Trust Officer is a job
title I've seen.
I've seen that in market, andtrust starts to become a really
interesting thing because it'snot a measurement of effort.
It's a measurement of outcome.
And, those roles are reallyinteresting roles.
So, you see a Chief RiskOfficer once in a while, a Chief
Trust Officer, a Chief Ethicist.
CPO just keeps expanding andexpanding in obligation

(19:28):
requirement.
Right now, I would hate to be aCPO now compared to when I was
one, because it's so much fallsunder a CPO title anymore.
And also, you know, if you getit wrong, people are starting to
go to jail.
Now, you also have to be really, really smart about what is a
defensible program.
They're all different skills.
I went through a ChiefCompliance Officer training in

(19:48):
health, and 80% of it wasknowing when you had to bring in
the Board.
What was a board obligation?
What was a board reportableobligation?
And, it's so true because youhave to be able to affect the
change in a consistent change inyour business.
Or if you are not empowered tobe that change in the business,
you have to be able to report itupward or else you're

(20:08):
accountable for it.
So, it's just it's becoming amore liable position as well,
it's why I think you see so manyCPOs are part of Counsel's
Office, in Big Tech especially.

Oh, that's interesting for that overlap.
That's why you think that somany CPOs are required to be
Counsel - like practicingCounsel?

What I think, in other industries, is that the
compliance official comes fromthe core competency of the
business.
So, if you're in finance, it'susually somebody with a with a
finance background or businessbackground.
If you're in an industrialsafety position, it's usually an
industrial safety engineerwho's your chief regulatory
compliance official.
You would think that in BigTech it would be an engineer,

(20:45):
somebody who understands theengineered product, or a product
leader, but it's not.
It's always a lawyer, and partof it is just that it is so
litigious, and the regulatoryenforcement actions are so
significant, that I think thatmore and more of the privacy
official ends up being part ofCounsel's office.

Yeah, and I think that's also really
unfortunate because, Counsel -they do amazing job in what they
do - but you will counsel abusiness; it's not the business
itself.
Right?
If engineers and product folksare helping to create the
products and actually handle andtouch the data, it just seems
to me that the Chief PrivacyCounsel would be dealing with

(21:24):
legal stuff and Chief PrivacyOfficer is the business and
operations of privacy and maybeeven including some aspects of
privacy engineering or productor whatnot.
I've seen CPOs come right outof being a partner at a law firm
and then go into industrymanaging a large CPO office; and
I'm just like, "I don't knowhow they're set up to win there,

(21:46):
right, like there's justdifferent skill sets - the
management.
I'm not saying you can't be alawyer, but just that they're
not.
.
.

My best friends are all lawyers.
The thing is that most of theseBigT ech firms have the person
who's externally- facing, who isthe CPO, and then they have a
completely different functioninside the business that does
all the operational work.
That's true at all the big techfirms that I mentioned.
So, at Reddit, I operate theprogram that is privacy, but we
have a CPO inside our Counsel'soffice who's amazing and a great

(22:16):
CPO.
Again, it's just that so muchof the external interaction on
privacy anymore is withregulatory authorities and
bodies.
What we used to say is that, inthe compliance space and health
is, compliance programs aresupposed to be completely
transparent.
They're for the public.
Everything you do is reallysupposed to be transparent.
And then that's kind ofantithesis to Counsel's office,
which is to be protectionist.
It's their job.
So, it just ends up being alittle bit less transparent when

(22:39):
we have to go that way, andit's the shame that is the
enforcement actions of today.

That's a really good point.
Thanks for that.
Speaking of regulations, thereare new EU regulatory frameworks
like the Digital Services Act(or the DSA) and the Digital
Markets Act (or DMA).
What do you think, are privacypros the ones who will own those
ethical requirements underthese regulations within their

(23:05):
orgs?

You know, I see that happening a lot of places,
including with us.
You know, in large part, theDSA especially, is identifying
transparency requirements in howads are placed.
So, transparency and data andusers and agency all kind of
fall under that privacy missionstatement.
So, I would say, "yes, as muchas dark design did, as much as

(23:27):
kind of child protection rulesdid, when you come to data and
ethics related things, somebodyhas to operationalize them, not
just interpret them for yourbusiness, but then go create
something to solve for thatproblem, and I think that that
typically falls to a privacyprofessional.
So we're involved in DSA.
DMA, and most of my friends whowork in privacy are at these

(23:47):
BigT ech firms are also deeplyembedded in this work.
It also has a big componentaround AI and AI auditing, and
so when you look at theevolution of data protection -
from security to privacy to MLand AI - I see us very much
being embedded in DSA.
I also see us being embedded inResponsible AI, too.

So, speaking of Responsible AI and just AI
generally, I see the IAPP issetting up an AI Governance
Center and a new AI GovernanceCertification.
In fact, at the IAPP PrivacySummit this past March,
leadership emphasized thatprivacy pros are well positioned
to take on Responsible AIresponsibilities, whether that's

(24:28):
legal or consultative, or evenengineering responsibilities.
Since you're doing so much inthe ethical AI space, what do
you see the role of the PrivacyEngineer when it's overlapping
with Responsible AI?

It's such a great conversation starter, to
be honest.
I think that there's lots ofroom for interpretation on this
one.
I personally subscribe to theidea that AI governance is not
very different than other datagovernance responsibilities and
requirements.
At its core, I oversimplifyeverything because I wasn't a
technologist in my firstincarnation, but for me, machine

(25:04):
learning in AI is justbasically big data with
statistics on top.
So, when you talk about thegovernance structure for ML in
its use cases, most of the timeit's not a very different debate
than should we use large datasets to solve a problem.
So, I think that we'reperfectly positioned to
understand kind of theramifications and we just have
to scale up a little tounderstand what is a

(25:26):
representative sample?
Where does data cleansing comefrom?
How do you create a trainingdata set and a reinforcement and
feedback loop?
It's adding a bit of newcapability on top.
That isn't very much of astretch from what we did for
data governance.
If you were keeping up with thegovernance as a labeling
technology, and not necessarilyjust as a risk committee

(25:46):
conversation, it is a prettygood extension of the privacy
engineering components of datagovernance.
So, I say, "Yeah, bring it tous, give it to us.
Let us help expand a forum thatwe already have for data
governance related topics.
Also, it's really expensive tostand up governance structures,
so the more you can recycle, themore likely it is to be

(26:06):
successful and sustainable in abusiness.
So, 'yes, and' - privacyengineers are going to have to
learn a little bit of newcontext, a little bit of new
technology, but we're reallygood at that because every time
there's technologicaladvancement, and it gets tiring,
but I think this is one that'sreally worth investing in
because I think this is here tostay.
So, yes and - we apply dataethics to all kinds of data

(26:32):
project review processes in thepast, like should we do this
research?
Should we allow people to askthese questions?
It's just an extension to sayshould we allow computers to
make decisions in really large,fast ways about specific topics?
So, yeah, who better and Iguess it's one of my questions
who better in the company toapply an ethical position, to
understand tradeoff discussionsand to adapt to new technologies

(26:55):
as it relates to data use?

Great, that's exciting.
I look forward to seeing theoverlap develop between these
two fields.
Who doesn't want another cert?
Oh my gosh, I have too many as it is.
It's so funny.
It's what IAPP is there for.

It's the alphabet soup.

Exactly.
So, I would love for you totell us about your nonprofit
that you started, HumanIntelligence, that you co-lead
with Dr.
Rumman Chowdhury, whichsupports AI model owners seeking
product readiness reviews atscale.

That's a mouthful, huh?

Yeah.
What motivated you both tofound Human Intelligence and
what is the org's mission?
And, did you know that I wentto high school with Dr Rumman
Chowdhury?

No you didn't.
Oh man, she's one of myfavorite people in the world.

We met in AP Biology.
This is a true fact.

Wow, okay.
Well now, I really want to knowwhat high school you went to
and go like recruit from there.

Yeah, we're probably outliers.
I don't know many others thatare from our years, that are
like big in tech and made a namefor themselves.
So, this is the like oneoutlier that I'm aware of, the
two of us.

Well, so far I like the standard deviations
coming out of your high school,so we'll take you.

And I'm not even joking, but my dad was the High
School Principal.
So.
.
.

Well, there you go.
Rad.
I can't claim that from my highschool.
We had 5000 kids in my highschool.
Wow, ours was much smaller thanthat and you have all this
excellence.
So, clearly he did somethingright.
So, I'll tell you this.
I met Rumman when I was atTwitter.
I had started just two or threemonths before she did, and she
was like a celebrity hire for us.

(28:45):
She was an aqui- hire,actually; we bought a portion of
our company to bring her talentto bear, and we started this
really hardcore mission ofapplied AI ethics inside of
Twitter.
I recruited a really large teamof amazing data scientists and
then, at the end of the day,everybody got fired, just like
everybody did in responsible AI.
So, we started our friendshipthen.

(29:06):
I was Head of Product for heras the Head of Engineering,
Director of the program; and wewere kind of a delivery
partnership.
So, thought leadership on herside, me was just 'go big, go
fast' and make everything applied.
And, we were having a greattime in about two and a half
years ago now, not even, twoyears and some change.
We had this great idea where wewere talking to the AI Village

(29:29):
folks, and it was a virtual day- there was a virtual session
for DEF CON 29 - and they'relike, "We should do a bias
bounty.
And so we were like, what is abias bounty?
We already had this algorithmfor image cropping that we knew
was racist and gender biased anda bunch of other ists, and we
thought, "You know what?
Let's put it out to the worldand share with them kind of the

(29:50):
learnings that we had justpublished and challenge people
to find more that's wrong withthis particular image cropping
algorithm.
So, this algorithm was based onwhat's called 'saliency.
' Saliency is what's interestingwithin a picture; and most
saliency models are trained bytracking eye movement.
So, it's not a very consciouslytrained algorithm.
It's kind of your subconsciouslooking at images and it tracks

(30:12):
your eye movement and finds whatyour eyes found most
interesting about a picture.
What's also interesting aboutsaliency models is that a lot
were trained on college campusesby CS students.
So, you can imagine what a CSstudent found interesting about
a picture of a woman would bedifferent than what they found
interesting about a picture of aman.
And so, these image croppingalgorithms have been trained
subconsciously to be horriblybiased and cropping women from

(30:34):
neck to navel for the most part.
It was also cropping out blackpeople and it was also cropping
out people in wheelchairs.
It was cropping out people withwhite hair and it was cropping
out people who were larger thanskinny people.
It was just because the humannature, if we were to put
consciousness to our training,would have been completely
different and been much morefair with people.
But, unconsciously, algorithmstrained without intention turned

(30:59):
out to be pretty terrible.
So, we decided to run this biasbounty at DEF CON 29 and had a
great time doing it and we wereable to award some prizes.
We were able to talk reallyopenly about the fact that not
all algorithms should exist,that not all technical problems
need to be solved with ML, andit was pretty successful, and

(31:20):
that was really fun.
And so, when we were leavingTwitter - I left before Rumman -
we didn't want this idea to die.
It was a lot of fun.
It created a lot of awareness.
And, it created a lot ofinterest in people to study this
problem and to find new appliedways of solving for algorithm
bias.
So, we kind of spun up this BiasBuccaneers project.

(31:41):
It was completely unfunded andthe website is still really ugly
.
We ran a lot of information.
I think the website's cute.
It's pirate themed for those onthe call who don't want to go
to bias buccaneers.
org.
It's all pirate themed becausewe really liked pirates that
year.
It was also my Halloweencostume.
So, pirates it is.
And, we had a really good timeand it was a really hard

(32:02):
challenge and we had lots ofparticipation and we were able
to award some great prizes andit was just really invigorating
and exciting.
And it was just a bunch ofvolunteers that put it together,
and we're like "his is a greatway to help motivate an industry
and to help people like getsome training that is actually
quite rare to find.
" Applied AI bias detection isdifficult and there's not very

(32:24):
many people who are teaching it,so we didn't want to see that
die, so we continued to play andthen we created this nonprofit
so we could create this DEF CONchallenge.
So here we are.
Rumman has a lot of work, andthis is our side quest.

Love it.
So, this is a side quest thatenables the bias bounties to
continue.

Yeah.
But you know what?
It might not stay a side questfor very long because now
there's a lot of interest afterDEF CON.
So, who knows.
Maybe it'll become funded andbecome a real thing and we'll be
hiring and firing on that side,too.
Who knows?

Awesome.
Well, I wish the best for you.
At what point during theproduct development life cycle
should companies perform theseAI readiness reviews?

Oh gosh, at every point.
You know it's kind of likeprivacy-by- design.
Right?
It should be embedded at everystage of your development life
cycle to some degree, right?
So, algorithms are just likeany other software development
project.
You should take a look at thedesign of your project.
You should take a look attraining data for the project.
You should take a look at howyou know your initial precision
and recall rates are affected bythe data.

(33:30):
It's kind of continuous testingand evaluation strategy and
requirement when you build aproduct that is patched
effectively almost every day, ifnot every hour or every minute,
through a reinforced learningkind of strategy.
That means that you should berunning T and E; you should do
test and evaluation continuouslyto make sure that you're not
drifting from baseline, thatyour products are still

(33:50):
operating in the way that youexpect them to.
With every new bit of feedbackloop, you're not actually making
your AI dumber.
There's all kinds of articlesout right now that some of these
large language models areactually getting dumber.
They used to be able to dobasic math with precision into
the high 90s and now it's downto like 4% or 8% - I can't
remember.
You don't continuously testwhen you don't continuously

(34:11):
evaluate and have a positivereinforcement with human-based
contextual information comingback into your model, sometimes
it can drift quite quickly.

Well, how do you do that, I guess what they're
calling 'alignment?
" like how do you de-biased atscale?
I mean, is that even somethingthat can be done with humans in
the loop?
Or are we chasing windmillshere?

Kind of chasing windmills, but also you got to
keep charging the windmills,otherwise you end up in this
downward spiral.
So, yes and yes.
So, first and foremost, youhave to build checks and
checksums into your AI processesdirectly.
So, you're constantlyretraining your model, let's say
.
What is the representative datasample?

(34:59):
Are you making sure that yourrepresentation is accurate?
Is it correct?
Or are you training only on,you know, your average CS
student on a college campus as aguy, right?
So, how do you make sure thatthe people training your data,
or the data that you're using totrain, are representative of
the people who use your productsand services so that it can
adequately perform for all typesof people?
So, building that into yourdevelopment life cycle so that

(35:22):
it's constantly being evaluatedis a great first step.
Finding some ways to identifyif you have drift, building that
into your launch process tomake sure that you're doing an
evaluation to make sure thatyour model is performing in
accordance with your baselineexpectations, it's not drifting
away from some of these reallyimportant measurements of bias

(35:43):
and representation.
And not all bias is social, bythe way, right?
So bias is just a term thatdoesn't necessarily relate to
social justice- related usecases.
We're talking, bias could bemaking mistakes on calculating 2
plus 2 equals 5.
So, how do you identify some ofthose triggers?
Things are varying fromexpectation.

(36:07):
then also, you know, there'salso an internal raging debate
about what is bias.
Is bias a representation ofsocial in a way that you know
you don't want to manipulate orchange the world?
So, how much of what you'reexperiencing or recognizing is
actually resulting from themodel versus just society and
the data that comes out ofsocial society?
So, it's very difficult, but Ido think it's worthwhile.

(36:28):
I think it's still somethingyou have to study.
I think it's still somethingthat human beings can make a
judgment call better than mostsystems still, and that we need
to provide a feedback loop.
And that's what bias bountiesare.

That makes a lot of sense.
And also I'm hearing theimportance of measuring and that
way - if you don't have metricsand measuring and where you
want to be, how are you going todetermine if you're drifting
from that?
For me, that's a big takeawayfrom this conversation.

Yeah, and there's no real measure of
success today.
Right?
There is no, "hey, you need 95%on this scale.
So some of it is making updeterminants for yourself.
For your own business and yourown model, what is progress and
what is the opposite of progress?

Can you walk us through what a bias bounty is
like - you could use BiasBuccaneers or the DEF CON one,
but basically, I want to knowwho should be hunting for biases
in the bias challenge.
What are their backgrounds?
I think those questions gohand-in-hand because there's
different challenges.

So, drawing on the security history, so, from a
security perspective, you know,there was a real leap forward
when bug bounties were firstinitiated.
One of the reasons was, youwent from having a really small
security team that was reallylooking at product and a OWASP
in the app layers and networksecurity and all these things,
to creating a huge externalworkforce that was like a layer

(38:04):
for defense, if you will, thatwould come in and, because they
were paid, would come and tellyou before they sold it on the
dark web, about yourvulnerabilities.
It was a real leap forwardbecause you can't be everywhere
all the time.
Bug bounties became a realvalue add to security
organizations, and they're nowstandard practice.
Algorithms are everywhere.

(38:25):
They're in every part of ourlives.
Like, I don't know that we evenknow how often we're
interacting with algorithms thatare making decisions that are
good and bad for us.
So, I would suggest thatanybody who's affected by an
algorithm should have theability to report when they see
or experience something wrong;and that doesn't really exist
today.
Like, you can't report analgorithmic misconduct issue

(38:47):
through a bug bounty; and thereis no Help Center where you can
file a ticket if you feel likethere's gender bias in an
algorithm or if you see a matherror or you think the search
results are creating ahallucination.
The idea here is to create theability for even Joe Public to

(39:11):
participate in reporting errors.
But, you know, paying people tofind errors in algorithmic
outcomes is actually superproblematic all by itself.
Right?
Because there's a verificationprocess and then there's a
structure, payment process andall these sorts of things.
So, these sorts of concepts arestill being worked out.
As far as, like the details andit's core bias, bounties are

(39:31):
basically structured humanfeedback.
For DEF CON, we created veryspecific challenges.
We said here are 21 things thatwe know large language models
can get wrong.
Can you make it do these things, despite these companies
putting effort into creatingguardrails to prevent those
things from happening?
So, in this case it was almostcompetition for format, so we

(39:54):
called it a 'Capture the Flag'(CTF) - a Jeopardy- style
Capture the Flag.
It was probably closer toStructured Public Feedback that
we validated and awarded pointsbased on submissions.

Yes, I agree.

And so, I would say that it wasn't necessarily
Capture the Flag so much as itwas, "Hey, we validated that
your submission actually didviolate the guardrails that
these large language models hadput in place.
For defcon, we had eightdifferent LLMs.
I think it was a very specialpoint in time where a
competition was set aside toallow us to do this thing.
I don't know if we'll ever seeeight LLM side- by- side again,

(40:26):
like we did there, but it's avery interesting experience to
see how eight different largelanguage models behaves.
You could kind of tell just byfrom the personality of the
models.

Oh, and sometimes they'll tell you too,
which model it was.
If you answered in a certainway - if you prompted it in a
certain way.

Yeah, I was encouraging people to do that.
Maybe I shouldn't have, but ifyou ask them, it'll probably
tell you which model it is, eventhough we were trying to blind
it a bit, at least make it notobvious.

Yeah, you named them different elements of the
periodic table.

Right.
So there was Iron and Cobalt andall these other funny names for
your average Cohere and LLaMA2and ChatGPT, but you could kind
of tell that some of them weretrying to make you feel at home
and like this was just anotherhuman being you were talking to.
And then others were verystructured and very clearly an
AI, not trying to convince youotherwise.
And, some of them were veryeasily broken and some of them

(41:20):
were a lot harder to break theguardrails, but they were also
less useful and so, seeing themside- by- side, you can see that
the development pathway forlarge language models,
especially, and very differentat these different companies.
So, if you were to go hire oneof these companies to build an
LLM or to use their API tofurther your product development
- let's say that you wereproviding post traumatic stress

(41:43):
services and you wanted to havean intake conversation and have
it transcribed and have it ask acouple of clarifying questions,
which of these models would youchoose?
Would you choose one that wasmaking a person feel safe at
home or would you want them tobe, very clearly, "this is
talking to an AI and not to feellike this could be medical
advice.
Right?
So it's very interesting justto see them side by side, but

(42:06):
then also to see which ones wereeasily manipulated versus not
very easily.

Yeah, and to give context on the different
challenges, you know I did justthe general challenge of trying
to get a biased output becausefor me - we only had 50 minutes
and it kicks you out; that wasby design.
So, you know, you're like go, "Ionly have 50 minutes to play
with this and there's so muchyou want to play with, and so I
went with something that for mewas a little easier.
And my fiancé, Mack Staples, onthe other hand - interestingly,

(42:35):
he went and did the one where Ithink he was trying to get the
LLM to add him to a specificaccess control list; and he had
to get it to do that andconvince it to add him to the
list.
And I know, like you know, Istarted to see his creative jui
ces flowing, too.
It's interesting to see what ahacker can do - even if he

(42:57):
doesn't ultimately get thedesired behavior from the LLM,
with each prompt he ended uplearning more information about
the system or how it wouldstructure an answer enough to
then, you know, almost as a clueto going to another step.
So, I thought that was justfascinating to see, or room full
of hundreds of hackers.
I mean, what did you have like3000 come through over the

(43:19):
weekend?

Yeah, 300 total.
We had a little bit of downtime.
We're shooting for 3000 anddidn't quite get there.
Yeah, it was fascinating, andthat's really the ultimate
output of this challenge.
Well, twofold.
Right?
We had several objectives forthis event.
One was absolutely educationand awareness - to get all these
people thinking about LLMs andhow easy it is for

(43:40):
misinformation to be generatedat scale; because we see it, we
see it in the world and we needto be able to recognize when we
see it, so that we can combat it- all of us, in whatever our
jobs are.
We should all, just likesocietally, understand that not
everything you read is real.
So, and to question and toprove, and to study and to

(44:01):
research and to make sure thatthe information that you see is
real.
And the second part was tocreate a vulnerability database
to understand how hackers hack;how people, when motivated,
would try to leverage an LLM inreally malfeasant sorts of ways;
to try to accomplish a taskthat a sane person shouldn't be

(44:22):
trying to accomplish, right.
So, one of the questions wascreate a hallucination and
create fake history based on aWikipedia article.
Right?
People do this for fun, and itbecomes real in the real world.
So, how do people manipulateand do this at scale using LLMs
so that we can create betterguardrails?
So, the data analysis isongoing and all of the different

(44:46):
LLM companies are prettyexcited about the research that
results from this as well.

That's awesome and I know you had a lot of eyes
on you.
There was so much press.
You know, even The White Houseand Congress were keeping an ear
out.
Right?

They came through and played the game,
which was fantastic.
You know, the Cyber SecurityCzar for the administration came
through and spent a good hourtalking to people.
We had about 212 communitycollege students that were
sponsored to come to DEF CON andparticipate so that they could
bring their learning and theirpoint of view and their
perspective, which we don'talways hear.
We also had a couple ofdifferent social good groups

(45:22):
that came through so we couldlearn very specific experiential
feedback from Black Tech Streetand some other groups.
So, that was also superinformative, that everybody's
experience with these LLMs isdifferent.
Their point of view isdifferent.
Their perspective is different.
How LLMs and some of theseharms that were represented in
the challenges affect them isdifferent.

(45:42):
So, their feedback was supervaluable too.

I bet.
The sheer number of people - somuch of that were that you were
telling me that I helped do wasjust helping with lines.
That was my volunteering - asalmost like a DEF CON goon.
You know, there were just linesout the door and for maybe like
an hour to three hours ofpeople standing on a line

(46:06):
because they all wanted to getto try this out.
So, I'm saying that not as abad thing.
I'm saying that because therewas so much interest and it's
really great to see.
But, I do know that you've gotsome plans potentially for next
year's, like some lessonslearned for next year.
So, could you tell us a littlebit about what we might look
forward to at the AI village atDEF CON next year?

Yeah.
So, one of the big benefits Ithought from this last cycle is
that, at one point I had areally hardcore red teamer
sitting right next to a grandmaand both of them were engaged
and both of them were challengedby the exercise, and I thought
that that was just so wonderfulthat, no matter your technical
skill set, no matter where youcome from, this affects your
lives and you can engage with itand learn something and really

(46:50):
like challenge yourself, thechallenge.
So, we want to continue that.
So, that's one of the things wewill continue - to build
challenges that are veryaccessible, which means a time
limit, right, because, like, thewinner went through several
times, which is not against therules, so you know that you can
get faster if you like, figureout what works on each of the
LLMs and each of the challengesand come back and try again.

(47:10):
So, we'll probably do it onlineso that more people can
participate.
We just we started thinkingabout this in June and the event
was in early August, so we hadtwo and a half months to build
this particular event.
So, we'll have a whole year,which means we can really build
a secure environment and maybenot wired computers.
It's been a long time since Iran cable, so that was kind of

(47:30):
exciting.
So, we'll do it bigger.
We'll do it broader.
It will probably allow for teamentries, because team entries
tend to get really fascinatingand interesting results.
I think that we will try tomake it more like an actual
Capture the Flag (CTF) challengescenario.
Gosh, we need better lightingand music.
That was the most.
.
.
it felt like we were in acorporate boardroom.

There's that eSports arena that might be a
good fit.
I mean one of the hotels,Because this happens in Vegas,
for anyone who's listening.

We do.
We need to find a much more funenvironment and venue.
I just want to continue theeducation awareness and amp it
up a bit.
I think we create two pathways- one for the learner and one
for the competitor that, youknow, could ease yourself in as
a person who's just learningabout this for the first time
and then you can have a realcompetition ready CTF option for

(48:26):
people who really wanted tocompete.
But mostly, what we need to dois just make sure it's still fun
.
I love gamification of hardthings because when you gamify
things, people get engaged.
They don't feel like this isunapproachable.
It applies to privacy, too.
I like gamification and theprivacy program stuff.
It's like you mentioned earliertraining and education needs to

(48:46):
be fun.
When you have time and you havemomentum, you can really make
things fun and that usuallylends itself to getting
emotionally invested, whichusually means better learning.
So, we just need to make itmore fun next year and maybe a
little bit more challenging forthe people who really want to go
hardcore for the prizes.

Yes, absolutely, especially since it's in Vegas,
and I know that that's whatsome of the bug bounty platforms
do is they have their livehacking events with cash prizes,
and in Vegas during DEF CON.
This has just been such a greatconversation, and there's so
much wisdom that I think we'veall learned from you today.
Do you have any last words ofwisdom for the audience that

(49:29):
you'd like to share before weclose, or are there any calls to
action you'd like to make?

Love your side quests, man! Just go say "es to
crazy things and try new thingsand see where they take you.
From a privacy perspective, I'dsay this - you know I'm a
subscriber to framework.
I love repeatable frameworks.
It's kind of how I live my lifein addition to.
.
.I'm not actually a rigidperson; it's just that I find
that repeatability is the key tosuccess and happiness.

(49:55):
So, when I look at the privacyengineering work, where we're
making the most difference, goback to some of the fundamentals
about how do you createdefensible, continuously up,
improving, effective program;and the first four steps of the
seven elements of a complianceprogram are on knowledge and
improvement, knowledgeenhancement.
And, I feel like sometimes inthe privacy engineering space we
get so wrapped up in kind ofthe technical controls to place,

(50:18):
and it is just not sexy to workon things that are knowledge-
enhancing, but that's really amissed opportunity.
So, I love education, training,policies, standards, guidelines
.
I mean it's not sexy work, butthese are force- multiplying
tasks.
Start with knowledgeenhancement and it will take
your program farther faster.

(50:38):
That's what we're doing withthis AI stuff - we're just
trying to get knowledge outthere.
We might not have made theright knowledge yet, but we're
just trying to get knowledge outthere so that people will
engage, people will find theirown true north, and kind of take
this industry in completely newand different directions.
I love this approach - to createcuriosity, create engagement,
have some fun.

(50:58):
This doesn't have to drearywork.
Have some fun and people willfollow.
So, I think that's what I leaveyou with.
That's what DEF CON was about,was just having a good time, and
it turns out people really didhave a good time, even
Congressional delegations fromdifferent States; and yeah, I
had a great time.

It was a great event and you should definitely
be proud of the first giantlaunch with two months of
planning and and so many eyes onyou.

Oh man.
We were programming the daybefore we went live and like,
"Oh my God, nothing's gonna work.
What's gonna happen?
We ran out of ethernet cables.
We were under table.
I was sweating.
I sweat the entire time.

Well, I will tell you this, it was even hard
to get near you - not because ofthe sweat -but because there
were so many people who wantedto talk to you about how much
fun they were having with it.
When they were done, they werelike all excited.
I mean, I was there, and had tomake room for all these people
to give you feedback; and Ithink you did an amazing job the
whole team, you and Rumman, andI don't know all the right

(51:58):
people to give accolades to, butcongratulations on a job well
done.

Awesome.
We'll see everybody on thispodcast and from our network
networks, hopefully, we'll seeyou all next year at DEF CON.
It is not the scary conferencethat everybody thinks it is.
It's actually pretty safe,lovely and inclusive.
So, come on up.

And now there's kids.
There's stuff for kids there,too.
Well, Jutta, thank you so muchfor joining us today on Shifting
Privacy Left to discuss allthings privacy engineering and
Responsible AI.
Until next Tuesday, everyoneone, will be back with engaging
content and another great guest.
Amazing Thanks.
Thank you.
Thanks for joining us this weekon Shifting Privacy Left.

(52:41):
Make sure to visit our website,ShiftingPrivacyLeft.
com, where you can subscribe toupdates so you'll never miss a
show.
While you're at it, if youfound this episode valuable, go
ahead and share it with a friend.
And, if you're an engineer whocares passionately about privacy

, check out Privado (52:57):
the developer friendly privacy
platform and sponsor of the show.
To learn more, go to provado.
ai.
Be sure to tune in next Tuesdayfor a new episode.
Bye for now.

All Episodes

Episode Transcript

Popular Podcasts

Dateline NBC

24/7 News: The Latest

Therapy Gecko

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}S2E28: "BigTech Privacy; Responsible AI; and Bias Bounties at DEF CON" with Jutta Williams (Reddit)

Episode Transcript

Popular Podcasts

.css-r6mb8g{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;}Dateline NBC

24/7 News: The Latest

Therapy Gecko

All Episodes

S2E28: "BigTech Privacy; Responsible AI; and Bias Bounties at DEF CON" with Jutta Williams (Reddit)

Dateline NBC