December 11, 2024 • 33 mins
In this episode of the Product Experience Podcast, we speak with Kasia Chmielinski, co-founder of The Data Nutrition Project, who discusses their work on responsible AI, data quality, and the Data Nutrition Project. Kasia highlights the importance of balancing innovation with ethical considerations in product management, the challenges of working within large organizations like the UN, and the need for transparency in data usage.
Featured Links: Follow Kasia on LinkedIn | The Data Nutrition Project | 'What we learned at Pendomonium and #mtpcon 2024 Raleigh: Day 2' feature by Louron Pratt
Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. She’s worked on a diverse range of products – leading the product teams through discovery, prototyping, testing and delivery. Lily also founded ProductTank Bristol and runs ProductCamp in Bristol and Bath.
Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Randy Silver (00:00):
Hey, it's the Product Experience Podcast, and
I'm Randy and I'm Lily, and it'sbeen a while since we did an
intro together.
Lily Smith (00:08):
Yeah, we were lucky enough to join Pendemonium
earlier this year emceeing theMind the Product stage, while
the other person interviewedpeople for the podcast, and it's
been great to meet people faceto face, but we've missed being
together, so we promised to domore intros together.
Randy Silver (00:23):
We met a lot of amazing people in North Carolina
, but today's guest might havebeen my favorite.
Don't tell any of the others.
They're an advisor to the UN,they've worked with the White
House, they were part of MIT'sScratch which might be my
favorite part of the whole thingand they also helped scale the
US's COVID response.
Lily Smith (00:40):
Kasia Kermelinsky gave a great talk on stage, then
sat down with Randy to dig evendeeper on data nutrition.
In this interview, let's getright to it.
The Product Experience Podcastis brought to you by Mind, the
Product part of the Pendo family.
Every week we talk to inspiringproduct people from around the
(01:02):
globe.
Randy Silver (01:03):
Visit mindtheproductcom to catch up on
past episodes and discover freeresources to help you with your
product practice.
Learn about Mind, the Product'sconferences and their great
training opportunities.
Lily Smith (01:16):
Create a free account to get product
inspiration delivered weekly toyour inbox.
Mind, the Product supports over200 product type meetups from
New York to Barcelona.
There's probably one near you.
Randy Silver (01:33):
Hey, we're here in Raleigh at Pandemonium and I'm
here with Kasia Chmielinski andthey just got off stage and
gave this amazing talk and wewere hanging out yesterday and
had some really good things totalk about as well, and we're
going to get really into allthis stuff, but first, for
anyone who isn't here and didn'tget a chance to see you, can we
just do a quick introduction?
Tell us what are you doingthese days and how did you get
(01:55):
into this world in the firstplace?
Kasia Chmielinski (01:57):
Yeah, thanks for having me.
It's great.
The energy here is fantastic.
For those who can't see us,we're surrounded by small,
bright pink dinosaurs which Ilove.
Lily Smith (02:05):
It's a good vibe.
Kasia Chmielinski (02:06):
Yeah, so my name is Kasia.
I am a technologist.
I've been building products for20 years which is a scary thing
to say as a product manager,and I've worked in a number of
different places.
At this point, I'm reallyfocused on responsible AI and
data and ethics, and I do thisas a consultant.
So I work with a number oforganizations, including the
(02:27):
Diet Nations, and I also run asmall nonprofit called the Data
Nutrition Project.
We build nutrition labels fordata sets.
Randy Silver (02:33):
Which is fantastic , and we will definitely get
into all of that.
Let's start with the UN, though.
Tell us a little bit about thework you do with them.
Kasia Chmielinski (02:41):
So the UN is massive.
So, shorthand, I say I workwith the UN, but there's
actually so many differentcomponents and I'm still
learning the system.
I've been an advisor with themfor about three years.
I want to say two differentparts of the UN, both of which
are really thinking hard aboutdata quality, data standards and
(03:02):
then how to use data eitherinternal UN data or external
third-party data to build anddeploy algorithmic systems that
will benefit the sector.
So what that really means isI'm a team of one, but I work
with folks who are build teams,who are on the ground, and we
work together to try tointegrate some of the best
(03:23):
practices from academia andindustry into what they're doing
at the United Nations.
Randy Silver (03:27):
For those of us who struggle in just you know,
single corporate bureaucraciesor working in our own civil
services, we know how hard itcan be to have influence at that
kind of scale.
And you're trying to do it onan even larger scale and more
interconnected.
How do you make it work?
Kasia Chmielinski (03:44):
Oh, it's definitely not just me and I
would say, I'm still trying tofigure out how to have the kind
of impact I want to have.
It always comes down for me, itcomes down to people.
It comes down to meeting theright people at the right time
who are doing things inside theorganization and borrowing
heavily and being humbled bytheir deep knowledge of the
(04:05):
system.
So this happened as well.
When I worked in the federalgovernment, I realized that I
don't speak federal government.
Randy Silver (04:10):
I don't speak UN.
Kasia Chmielinski (04:12):
I mean, it's truly another language and I
don't think that I will everbecome fluent.
And so what I've done is I'vetried to find native speakers
and I go to them and I say, look, I don't understand what this
directive means.
Or can you boil this down forme into something that makes
sense?
Or what do you want me to do inthis situation and that's how I
actually end up having animpact is I find the amazing
(04:32):
teams doing things.
I convince them and charm theminto letting me work with them,
and then we build thingstogether.
Randy Silver (04:38):
That is a really smart way of making it work.
That's very cool.
Okay, I wanted to talk aboutsomething that you mentioned in
your talk.
You were talking aboutresponsible AI, or a responsible
approach to AI, and you kind ofput it straight out there that
the way that we approach productmanagement as a whole is kind
(04:58):
of antithetical to beingresponsible, and that's because
we work really hard to maximizewhat we consider value, so we're
trying to go for the biggestham and we're doing things which
inherently leaves thingsmissing.
Can you talk a little bit moreabout this dilemma?
Kasia Chmielinski (05:16):
Yeah, I think it's kind of at the core
of product management and theway we build things, and it's
both the process that allows usto make small innovative changes
quickly that then we can scale.
So it's definitely beneficialto building innovative products,
but it also gets in the way ofbeing thoughtful and moving
(05:37):
slowly and with intention, and Ithink that it really ends up
being a question that PMs haveto face is how do I balance
these things, how do I balancethese approaches?
And so what I mean is that, asa product manager, you know, I
have certain goals that I needto hit.
I need to reach my OKRs.
I have some you know indicatorsthat I'm tracked against.
(05:58):
I want to get my bonus, youknow, or?
if it's a really small company.
I need to make sure that welaunch on time so that we get
the next round of investment.
I mean, these are really bigproblems, right, and they're
non-trivial.
Randy Silver (06:08):
And it's not an academic exercise.
Kasia Chmielinski (06:10):
It's not academic and it's fun.
It's super fun.
I think PMs were kind of likeMacGyvers we're just in there
trying to make things work, youknow, and that's okay, because
I'll get to them one day, butthe decisions they make in the
(06:31):
very beginning they end up beingthe DNA of the product, and
that means that all those otherpeople that were left out of the
initial set of users willalways be secondary.
You know, maybe eventuallythey'll be brought into the fold
, but it's always going to bekind of updating or refactoring
the product as it currentlystood, and that's where a lot of
the gaps emerge that end upbeing identified later as bias
(06:52):
or discrimination, or even justyou know folks that end up using
your product off-label, right?
They kind of take it, they dosomething else for it and they
find a way to make it work,because people are geniuses,
people are really smart, but itwasn't meant for them to use the
product in that way, and that'sbecause it wasn't designed for
them, and that's the kind of anissue that I'm highlighting here
.
Randy Silver (07:10):
So how do we deal with it?
I mean again, in a perfectworld, we would build things
more deliberately, morecarefully, we would take as much
time as we need.
We have enough of a problemwith getting accessibility baked
in at the beginning and doingit right, but we're always in a
hurry to.
Let's just get it out, let'sbuild something that doesn't
scale, then let's add on thisother stuff on top of it to make
(07:31):
sure it scales and we'rehitting all the regulatory
compliance and everything elsethat we need to do.
How do we actually deal withthis?
Kasia Chmielinski (07:39):
There are a number of approaches.
I think if someone had solvedthis, I wouldn't be here.
I wouldn't be here, I wouldn'tbe talking, I'd just be
following what they say.
And I've done it a number ofways, sometimes more
successfully than others.
I think we can.
(08:00):
Obviously, if you're startingsomething from the beginning,
you have a great opportunity tocenter marginalized users, move
the center right and actuallybuild for folks who are at the
margins.
And you can say I'm going toactually build for folks that I
would usually cut out of thefirst round because I'll
actually end up building abetter product for everyone if I
build accessible first right,for example, and so you can do
that if you have the luxury ofstarting from scratch.
Even then you're still makingchoices.
And so I think that, regardlessof whether you're starting at
the beginning or you're comingin later, a really important
(08:22):
thing is to actually bemonitoring and tracking how
things are going and to makesure that you have real feedback
loops with people, not justkind of check box.
I asked somebody did some usertesting.
I built a product, I launchedit.
Well, did you go back to themand show them what you ended up
building?
Did you get feedback on whatyou actually launched or did you
just do it as part of thedesign thinking process?
Right, and I think if we buildin places where we can actually
(08:46):
get meaningful feedback andtrack the use of these products
and change them over time, wecan start to address the gaps in
retrospect.
It's not perfect, but I thinkthat that's another approach if
you're kind of jumping in in themiddle as opposed to starting a
redesign.
Randy Silver (08:59):
Okay.
So a lot of times we talk aboutI'm getting started with
something and let's thin slice,let's find one use case that
we're going to do, and usuallywe choose the easy one.
And what you're advocating foris don't choose the most obvious
easy one, choose something alittle more on the edge, whether
it's marginalized people ordiversity, or whether it's
(09:19):
special needs of some sort,special needs of some sort,
because if you solve that andone of the classic examples
there is, you know, the use ofclosed captioning and things has
made video better for everybodynot just for people who are
hearing impaired.
Kasia Chmielinski (09:36):
Yep, exactly , and it's a trade-off again.
So if you want to design forthe most complicated
accessibility case, you want todesign for, you know, users who
are using five differentlanguages instead of just one,
and all of a sudden you have tohave a multilingual interface.
I mean, these things can getmore and more and more
complicated.
You have to decide, kind ofwhere your line is going to be.
To your point, you want toactually launch something.
I get that, but I think thatthere are ways that we've just
(09:59):
done things and we assume thatit's the right way to do it.
We should start with a processof saying what are we actually
trying to solve and are thereways that we can increase the
footprint of who will be able touse this product and it'll make
it easier for us down the line,right?
So if I build something andit's just this is a dumb example
but single tenant, right, soit's just for one user, but I
(10:20):
know that I'm going to beonboarding multiple At some
point, I'm going to have to makeit multi-tenant.
That's painful.
Do you want to refactor lateror do you want to make some
really basic database choices inthe beginning to make it easier
down the road.
These are the kinds of things Ithink that we can dial up and
down when it comes to the typesof trade-offs we make from the
very beginning.
Randy Silver (10:37):
So being deliberate with your
architecture around things of wemay not be turning this on now.
We know we're going to have to,so we're going to leave space
for it, even if we're not doingit today.
Kasia Chmielinski (10:47):
Yes, where it makes sense.
Randy Silver (10:54):
Where it makes sense.
Yeah, of course.
Okay.
And then you also talked a lotabout when you're in these
situations.
You have to sometimes make somehard choices about how you're
going to approach existing as aproduct person in a company that
is trying to realize value atpace and be responsible.
So one of the things that yousaid is one of the options we
have is that we can refuse tobuild and sometimes redirect.
(11:15):
Yeah, Tell us a little bit moreabout that.
Kasia Chmielinski (11:17):
It's kind of the product manager version of
voting with your feet.
Randy Silver (11:20):
Yeah.
Kasia Chmielinski (11:21):
You know, you're presented with some kind
of a product plan that perhapsyou did not build.
It comes as an edict fromsomebody up in an office
somewhere and they say thatshall build the following thing,
and you go oh my God, that'sterrible.
For whatever reason, it's moralor otherwise, and one option
that you always do have is torefuse to build it.
(11:41):
Now, this can be a disastrousdecision for, for example, your
career.
Right, Also, it's a privilegeto be able to quit a job,
because that assumes you can getanother job.
Right, Maybe you need apaycheck and all these things,
and so those things are real.
But there is the conceptualoption, at least, of saying I
will not build that thing.
A way to soften that sometimesis to say I hear you, that's
(12:04):
really interesting, Let mecollect some information or do
some research.
And then to go back and say thething that you want to build is
not the right answer.
Assuming this is true, theright answer would be the
following right, and you canactually redirect and say I
won't do the thing.
Essentially, you're saying Iwon't build your thing, but I'll
build my thing and it willsolve the same problem, assuming
you agree with the problem, andthat, I think, is one kind of
(12:25):
initial approach that you cantake.
That is, I would say, it's abold choice but it's definitely
one that we have.
Randy Silver (12:37):
But we talk a lot about.
You know, people are alwayscoming and saying can I have
this?
And one of the core skills of aproduct manager that you have
to learn early in your career ishow to say no without using the
word no, and I think that'sessentially what you're saying.
Kasia Chmielinski (12:48):
Sometimes there is power in saying no, and
I think that's essentially whatyou're saying.
Sometimes there is power insaying no, yeah, like I won't do
it, but that I think should beused sparingly, otherwise you
get a reputation for being theone that yeah, you can only do
that so many times.
Exactly, and so I do think youcan take a stand sometimes and
say I do not think this is theright thing to build, I will not
build it.
But again, that is very bold.
I've done it a few times, but Iwouldn't overuse it.
Mostly it is this other thingyou're talking about, which is
(13:10):
let me show you what I would do.
That gets to the same problem.
Or let me explain to you whythe thing that you want to build
is maybe not the right answerright now.
There's all kinds of linguisticand communications tactics you
can take there to basically movethem off of the path that you
don't want to be on and movethem onto the path that you do
want to be on.
Randy Silver (13:28):
We did an episode a few months ago with a guy
named Steve Hearsom who's anorganizational design and a
development consultant, and hewas talking about.
One of the real challenges forus is that in product,
essentially you're middlemanagement let's not beat around
the bush but you're being askedto influence and act and work
with people multiple levelsabove you in an organization and
(13:49):
above your pay grade, and it isa real challenge.
Kasia Chmielinski (13:53):
Yeah, and not only that, but you're not
actually in charge of anything.
Yeah, but you're accountablefor everything.
You have no real power.
You can't make anybody doanything really, but you need
them all to do things right, andso I think that there's a lot
of nuance in the role.
Randy Silver (14:11):
Yeah, if it was easy, everyone would do it right
, okay, another thing that youidentified as things that we can
do is we can build better.
What do you mean by that?
Kasia Chmielinski (14:22):
So for that one, what I mean to say is that
let's say that you're in thecontext of organization or a
product or a roadmap and youkind of disagree with some of
the decisions that are beingmade.
Well, you can decide how tobuild that thing in a way that's
going to be less harmful ormaybe more beneficial, right,
and so it's kind of an extensionof the redirect, but maybe a
(14:45):
little bit less brutal.
The redirect is they're goingin one direction and you shift
them to another direction.
I think the build better is yousee the product, you see what
the gaps are and then you try tokind of shift so that you can
fill those gaps and you canaddress those errors.
You can buy yourself more time,you can get the resources you
need, you can get the expertiseyou need, whatever it is to
actually address those issues asyou go.
(15:06):
And I think you know whateverit is to actually address those
issues as you go.
And I think in the world of AI,if we want to specifically talk
about AI, there's a realopportunity here to not see AI
as a product but rather as aprocess and, if you broaden the
aperture, to think about AI as aprocess that involves use case
selection, training, dataselection, the training of the
model, the development, thedeployment, the updates and then
(15:29):
eventually the decommissiontraining of the model, the
development, the deployment, theupdates and then eventually the
decommissioning of the model.
And you think of this as many,many steps where things can go
right and things can go wrong.
Then you're not just at the endof the whole process saying, oh
God, we have a problem, right.
You're able to then jump in andbuild better by saying let's
now identify issues earlier inthe process, let's have ways to
check the data, let's actuallyknow what we're looking for,
let's have a series of metricsthat we're tracking over time.
(15:52):
When we launch something, let'sassume that's the beginning of
its life as opposed to the endof its life.
Right, and now it's live.
Now we have to train it, wehave to make sure it's okay,
it's out in the world, it'saffecting the world, is it
drifting?
Are things?
I don't mean to antipomorphizebut, you know and I think these
are all now places where we canbuild better, if we have a more
nuanced and a broaderperspective of what the product
(16:13):
is.
It's more of a process than aproduct.
So these are the ways I thinkthat we can build better.
Randy Silver (16:17):
So when we are building without AI, we have
developed lots of tools, verypractical approaches to doing
testing and QA along the way.
We've got BDD, behavior-drivendevelopment, we've got
test-driven development, we'vegot unit tests.
We've got all kinds of otheressentially tick-listy kind of
things and processes.
With AI it tends to be a blackbox without an audit trail.
(16:40):
Is there anything practical ifwe're taking a responsible
approach that we can do, as yousaid, if we're making a process
of things we should be doingalong the way?
Kasia Chmielinski (16:49):
Yeah Well, ai is a broad term.
So when you say it's kind ofblack boxy, it depends on what
kind of system, it depends onwhat algorithm, it depends on
the deployment method and alsohow it's contained.
So I agree, if you go and youget an AI system from a vendor,
that's really black boxy becauseyou might not even be, it might
be in a cloud environment youmight not even understand how
(17:10):
it's working.
If you're building somethinginternally and you're using a
statistical system, an AI system, machine learning system that's
actually interpretable in someway, then there are definitely
approaches that you can take tounderstand the inputs and the
outputs, or at least thedistributions of the inputs and
the outputs, against some kindof benchmark or against some
kind of test that you wouldexpect over time.
When you start moving into thekind of generative AI,
(17:32):
probabilistic, you're buildingsystems where you're, you know,
cobbling a lot of thingstogether.
I would say my best kind ofsuggestion right now is we need
to be thinking about evaluations, we need to be thinking about
red teaming and these kinds ofabout evaluations.
We need to be thinking about redteaming and these kinds of
newer approaches to basicallyprodding and testing the models
Another way to do it from or inaddition to that makes a lot of
(17:55):
sense to make sure that you'rebuilding in a componentized way
so that you can isolate eachcomponent and test it for
certain things.
So, for example, if you'regoing to use an LLM for
something, instead of having theLLM do the retrieval and all of
the analysis and also the inputand the output and send
everything back to you, so itmakes all these decisions
(18:15):
internally, split it up intopieces so that you can actually
test each piece for what.
The one thing is that it'ssupposed to be doing, and that
way you can kind of isolateproblems at least at that level
too.
So as the technology isevolving and we're finding new
ways to measure bias and issueswith the outputs, you can at
least limit the surface area orthe footprint of each component
(18:38):
and test them separately.
Randy Silver (18:48):
Product people.
Are you ready?
The word on the street is true.
Mtp Con London is back in 2025.
We're very excited for Mind theProduct's return to the
Barbican next March.
Lily Smith (19:01):
Whenever I hear people talking about the best
product conferences, Mind theProduct is always top of the
list.
If you've been before, you knowwhat's in store.
Oh, that rhymes.
New insights, strategies,hands-on learnings from the
absolute best in the field, plusgreat networking opportunities.
And if you're joining us forthe first time, I promise you
(19:21):
won't be disappointed.
Randy Silver (19:23):
We've got one speaker already announced.
That's Leah Tarrin, who you'veheard on this very podcast.
With more to come.
From the likes of WhatsApp, theFinancial Times and Google, you
know real people working in thefield who will share real
actionable insights to level upyour game as a product manager.
Lily Smith (19:40):
Whether you're coming to the Barbican in person
on March 10th and 11th ortuning in digitally, join us and
get inspired at MTP Con.
London Tickets are on sale now.
Check out mindtheproductcomforward slash MTP Con to find
out more, or just click onevents at the top of the page.
Randy Silver (20:07):
So you talked about it as a process and when
you are being responsible withthe process or deliberate with
the process, anyway, you havethe ability to control and put
responsibility in at each ofthese steps.
But a lot of the times nowwe've got people coming in
saying can we just add AI tothis?
Or we've got third-partyvendors coming in and people
(20:28):
frustrated with how long ittakes us to do things in product
development.
So they're just buyingsomething in and I'll pick on
marketing or sales, because,well, we always pick on
marketing and sales, but theyhave vendors coming to them
saying, yeah, we can do that foryou and it's a shadow IT
function that comes in.
How do we add responsibilityinto our companies when we're
competing with those pressures?
(20:48):
Or is that just outside what wecontrol?
What we control and don't worryabout the things that are
outside our scope?
Kasia Chmielinski (20:55):
We should definitely worry about them,
because we'll probably beresponsible for the things that
they do.
So, ultimately, if yourliability is sitting with you I
mean, if your customers areinteracting with your system and
your system as AI that youdon't understand, your customers
aren't going to yell at thethird party.
They're going to yell at you,right?
Because?
Yeah.
So I think there should, firstof all, be a kind of vested
interest in understanding howthis stuff works.
(21:17):
I also am not againstoutsourcing things as a
principle.
You know, if your real valueadd is one kind of feature set
and the other features you couldget by, you know, contracting
with some vendor who's going touse AI, like that's not
necessarily a bad answer, but Ido think that at this point, we
should be talking aboutprocurement and we should talk
(21:39):
about the types of questionsthat you would expect a vendor
to be able to answer.
So, how does their system work,right?
Well, I mean, first of all, doyou even need AI to solve the
thing that you're trying tosolve?
So is there a need for them?
Okay, let's assume that youactually do want to work with
them.
Great, what is the systemtrained on?
How do they train it?
How does it work?
How are they measuring success?
(22:01):
What's their ground truth thatthey're comparing it to to tell
you whether or not their modelis accurate?
How do they plan to update andmonitor this once it's in your
environment?
How often can you expect to bereceiving updates from them?
At what point would theyconsider decommissioning the
program?
Or would you considerdecommissioning or turning off
the contract and then makingsure those things are really
(22:21):
baked into the contracts and theprocurement process so that
there's some accountabilitythere between you and the vendor
?
I think these are the kinds ofthings that we have to consider
if we're going to be contractingwith third parties.
Randy Silver (22:32):
Yeah, sounds outside the scope, unfortunately
, of what most product managershave in their job.
Kasia Chmielinski (22:37):
I mean, a lot of things are outside the
scope of what's there.
What even is our jobdescription?
It's not very clear.
Randy Silver (22:43):
Fair enough, but yeah, as you were saying,
systems thinking is incrediblyimportant and the ability to
discuss this with other peopleand bring it to light is still
incredibly important regardless.
Kasia Chmielinski (22:53):
Yes, I mean, if you are a product manager
whose function in the companymight be replaced by some kind
of outsourcing, I think you dohave, then, a bit of a
prerogative to highlight.
Hey, before you decide to putme on a different project, let's
make sure the thing thatthey're doing is actually
similar to the problem thatwe're trying to solve and that
we're doing this in aresponsible way.
(23:14):
I think you have a little bitof power there, but you know
it's tough being a productmanager.
It really depends on thesituation.
Randy Silver (23:19):
Not for the timid.
Yes.
No but being a PM probably isnot for the timid, so you might
want to find a different job.
Okay, you also said there wasanother one about creating new
solutions, and this is whereyour other project comes in.
Really well, so talk about thata bit issues in the product
(23:47):
development process.
Kasia Chmielinski (23:47):
You can refuse to build things right If
someone shows you somethingthat's morally contentious.
You can redirect people, youcan decide to stay within the
organization and build better,but in some cases, the gap is
not even attempting to be filledby anybody, and so your only
option at that point is tocreate something new.
And the good news is that, asproduct managers, I think we're
really well positioned to buildstuff.
The good news is that, asproduct managers, I think we're
really well positioned to buildstuff, and we already understand
(24:11):
that we need to have, you know,a multi-stakeholder, kind of a
panel of folks and voices tohelp us build things.
We know how engineers think anddesigners, and we can bring
together, you know, the higherups and the folks who have
funding and all of this tobasically become little mini
CEOs of things, and so this iswhat I did with some folks in
2018.
My team's going to be so upsetI never remember.
But it was a while ago we allhad a fellowship at MIT and
(24:33):
Harvard.
It was a joint fellowshipcalled Assembly and we were
charged with thinking about theethics and the governance of AI.
And they said you've got fourmonths, here's a bunch of coffee
and some cookies, and come upwith something anything.
So we were puzzling over whatwe thought the kind of problem
space was, and at that timethere were a bunch of news
cycles about the problematicoutcomes of AI.
(24:56):
Bias and discrimination is kindof rising social consciousness
around this and what we noticedas practitioners was that these
media articles were reallyfocused on the output.
They're saying these systemsare deployed, they're in
production and they're hurtingpeople.
But all of us could kind of seethat most of these issues were
(25:18):
issues that were just carboncopies of issues in the training
data and nobody was focusing onwhat about the data?
Why is no one talking about thedata?
It's not public, it's nottransparent, no one knows what's
going into it.
But then they're shocked whenbad things come out the other
side.
Is anybody looking at the data.
And we started digging aroundand even from our own experience
, we knew that there are nostandardized data tools and no
(25:42):
standardized data approacheswhen it comes to identifying a
data set for use in AI.
And we were again in anacademic institution and we're
drinking a lot of coffee, we'reeating a lot of cookies, and
there was a box of cookies and Idon't remember who it was, and
they turn it over and they werelike there's a nutrition label
on this box of cookies andbefore I eat the cookies, I can
(26:02):
see what's inside.
Shouldn't we like have the samething for data?
Just like something like that?
And so we call ourselves theData Nutrition Project.
We emerged from that fellowship.
We make nutrition labels fordata sets and the idea is that
you have a standardized tool anddesign that allows you to
understand what's in a data setbefore you use it.
And for a practitioner whowants to do the right thing you
(26:24):
know we're not certificationbody, you know we don't validate
all the things but for somebodywho wants to do the right thing
, we give them an easy way tocommunicate what's in the data
and then an easy way to digestwhat's in that data.
Randy Silver (26:36):
What kind of things do you have on the label?
Kasia Chmielinski (26:38):
There are a bunch of different components
and we're actually in chats tosee whether we need to expand
this for different domains ofdata, like large language, model
data sets, you know.
But the kinds of things that wehave are actually more
qualitative, because it turnsout that qualitative information
is almost more important inthis kind of a scenario than
(26:58):
quantitative, quantitative stats.
People will run a datascientist will run those quickly
and say what's the distribution, show me this number of missing
fields, blah, blah, blah.
There's some kind of core setof things you're taught to do,
but qualitatively, I want tounderstand how did you fill in
that missing data?
How much was missing?
How did you clean it?
Who paid for this?
All of these kinds of things youwon't see in the data set.
(27:20):
So we surface things like that.
We also have a little modulethat explains to you what the
intended use is.
So this is what the data wasintended to be used for.
This is the way that it hasbeen used.
This is what you should not dowith the data.
These are known issues andareas for practitioners to
surface whether the data set hasundergone ethical assessment or
technical assessment, thesekinds of things.
Randy Silver (27:41):
And how are you seeing it being used?
Kasia Chmielinski (27:43):
There are a few prominent ways that people
use the label.
One is in kind of a industrydata set context, which is
honestly what we thought ofinitially, and we've partnered
with organizations like the UN,microsoft Research, various
places at Harvard to talk aboutdata and talk about labeling
that data and improving themetadata that already exists on
(28:06):
those data sets, and that'sreally cool.
So in some cases, folks haveactually used our schema In
other places.
We've taken components of oursand added it to theirs and those
are live and out there.
There's also an entirely secondapproach that we didn't really
consider, which is that thelabel is very, very useful for
(28:27):
education and for teaching andso there are a number of
curricula out there that use thelabel to basically teach data
science students how to thinkabout data and how to build data
sets and what a good data setwould look like.
And DNP Data Nutrition Projectdoes a lot of kind of tours
(28:47):
where we give lectures and wework with students and we kind
of jump in give workshops,because it ends up being a
really great teaching tool.
Randy Silver (28:55):
Fantastic, yeah, and have you seen it solve any
problems?
The difference in output andoutcomes in this one.
Kasia Chmielinski (29:04):
Yeah.
Randy Silver (29:05):
Usage is an indicator that it's being used,
but what for?
Kasia Chmielinski (29:09):
Well, here's an interesting one.
One kind of thing that wedidn't really expect was that
the quality of data output goesup when someone knows they have
to put a label on it, so that'skind of a secondary effect.
We really built a label to makeit easy to find data and use it
, but we were working with adoctor at Morrill Sloan
(29:31):
Kettering who builds data setsof skin imaging and is very
aware of the potential bias inthat and kind of the
distribution of skin colors isjust not very well
representative, as we all know.
And so she was talking to usand said since working with you
guys and building these labelsfor these large skin imaging
(29:51):
archives, the quality of my datasets has actually gone up,
because in building the label, Irealized that there are some
things I should have done a longway, because you're asking me
such questions about decisions Imade.
I should have made differentdecisions.
So the next time I make thedata set this is like a data set
they refresh once in a whilethe next time I make the data
set, I make different decisions.
I think that's the kind ofimpact that is really cool for
(30:13):
us, because it's not just aboutbringing transparency to data
sets, it's also about improvingdata set curation period.
Randy Silver (30:20):
And if people are interested in learning more
about this and getting involvedor applying it, where should
just Google the data, nutritionor yeah, datanutritionorg, but
if you Google data nutritionproject, it would probably come
up.
Kasia, this has been fantastic.
I have one last question.
You closed today with talking alittle bit about your mission,
(30:40):
and a personal mission statementis something not everyone has,
but I found it really inspiring.
So tell us a little bit moreabout your mission.
Kasia Chmielinski (30:48):
Oh, I don't know which part you saw.
Randy Silver (30:51):
I don't know that.
I have a very clear missionstatement.
Okay, the slide I saw, I'llread it directly.
Yeah, there will always bedangerous gaps in technology
that humans need to fill, but wealways have opportunities to
refuse, redirect, build betteror create new solutions to make
technology that works foreveryone.
Kasia Chmielinski (31:06):
I mean, I agree with that, so I'm happy to
I also wrote it.
No, I mean, I think that's itand it's maybe that feeling, not
always in those words, I think.
My own experience is that Ibuilt these systems and over
time I realized that not all thesystems actually worked for me,
Right.
So I'm non-binary, I use they,them pronouns.
(31:26):
I built systems that wouldclassify me in a binary and I
literally wrote these programs.
Or you know, during COVID, Iwas helping to automatically
assign race and ethnicity tomissing data fields so that we
could understand whether therollout of the vaccine was
equitable.
And when I put my owninformation into this, it came
out with the fact that I waswhite and that I was female and
(31:49):
all these things that I actuallydon't identify as right.
That could be dangerous.
Funny at first, and then it'sdangerous.
I use a hearing aid, right?
So I end up being the margin.
I end up being in the gaps ofmost of the technology that I
have built over my own careerand, as such, at some point I
decided no more.
I'm now going to focus more ofmy time and energy on filling
(32:11):
those gaps and helping othersmake better choices to make
technology that works better foreverybody.
Randy Silver (32:16):
Thank you, that was fantastic, thank you.
Lily Smith (32:30):
The product experience hosts are me, Lily
Smith, host by night and chiefproduct officer by day.
Randy Silver (32:36):
And me Randy Silver also host by night, and I
spend my days working withproduct and leadership teams,
helping their teams to doamazing work.
Lily Smith (32:47):
Luran Pratt is our producer and Luke Smith is our
editor.
Randy Silver (32:49):
And our theme music is from product community
legend Arnie Kittler's band Pow.
Thanks to them for letting ususe their track.
Thank you.
Hey, it's the Product Experience Podcast, and
I'm Randy and I'm Lily, and it'sbeen a while since we did an
intro together.
Lily Smith (00:08):
Yeah, we were lucky enough to join Pendemonium
earlier this year emceeing theMind the Product stage, while
the other person interviewedpeople for the podcast, and it's
been great to meet people faceto face, but we've missed being
together, so we promised to domore intros together.
Randy Silver (00:23):
We met a lot of amazing people in North Carolina
, but today's guest might havebeen my favorite.
Don't tell any of the others.
They're an advisor to the UN,they've worked with the White
House, they were part of MIT'sScratch which might be my
favorite part of the whole thingand they also helped scale the
US's COVID response.
Lily Smith (00:40):
Kasia Kermelinsky gave a great talk on stage, then
sat down with Randy to dig evendeeper on data nutrition.
In this interview, let's getright to it.
The Product Experience Podcastis brought to you by Mind, the
Product part of the Pendo family.
Every week we talk to inspiringproduct people from around the
(01:02):
globe.
Randy Silver (01:03):
Visit mindtheproductcom to catch up on
past episodes and discover freeresources to help you with your
product practice.
Learn about Mind, the Product'sconferences and their great
training opportunities.
Lily Smith (01:16):
Create a free account to get product
inspiration delivered weekly toyour inbox.
Mind, the Product supports over200 product type meetups from
New York to Barcelona.
There's probably one near you.
Randy Silver (01:33):
Hey, we're here in Raleigh at Pandemonium and I'm
here with Kasia Chmielinski andthey just got off stage and
gave this amazing talk and wewere hanging out yesterday and
had some really good things totalk about as well, and we're
going to get really into allthis stuff, but first, for
anyone who isn't here and didn'tget a chance to see you, can we
just do a quick introduction?
Tell us what are you doingthese days and how did you get
(01:55):
into this world in the firstplace?
Kasia Chmielinski (01:57):
Yeah, thanks for having me.
It's great.
The energy here is fantastic.
For those who can't see us,we're surrounded by small,
bright pink dinosaurs which Ilove.
Lily Smith (02:05):
It's a good vibe.
Kasia Chmielinski (02:06):
Yeah, so my name is Kasia.
I am a technologist.
I've been building products for20 years which is a scary thing
to say as a product manager,and I've worked in a number of
different places.
At this point, I'm reallyfocused on responsible AI and
data and ethics, and I do thisas a consultant.
So I work with a number oforganizations, including the
(02:27):
Diet Nations, and I also run asmall nonprofit called the Data
Nutrition Project.
We build nutrition labels fordata sets.
Randy Silver (02:33):
Which is fantastic , and we will definitely get
into all of that.
Let's start with the UN, though.
Tell us a little bit about thework you do with them.
Kasia Chmielinski (02:41):
So the UN is massive.
So, shorthand, I say I workwith the UN, but there's
actually so many differentcomponents and I'm still
learning the system.
I've been an advisor with themfor about three years.
I want to say two differentparts of the UN, both of which
are really thinking hard aboutdata quality, data standards and
(03:02):
then how to use data eitherinternal UN data or external
third-party data to build anddeploy algorithmic systems that
will benefit the sector.
So what that really means isI'm a team of one, but I work
with folks who are build teams,who are on the ground, and we
work together to try tointegrate some of the best
(03:23):
practices from academia andindustry into what they're doing
at the United Nations.
Randy Silver (03:27):
For those of us who struggle in just you know,
single corporate bureaucraciesor working in our own civil
services, we know how hard itcan be to have influence at that
kind of scale.
And you're trying to do it onan even larger scale and more
interconnected.
How do you make it work?
Kasia Chmielinski (03:44):
Oh, it's definitely not just me and I
would say, I'm still trying tofigure out how to have the kind
of impact I want to have.
It always comes down for me, itcomes down to people.
It comes down to meeting theright people at the right time
who are doing things inside theorganization and borrowing
heavily and being humbled bytheir deep knowledge of the
(04:05):
system.
So this happened as well.
When I worked in the federalgovernment, I realized that I
don't speak federal government.
Randy Silver (04:10):
I don't speak UN.
Kasia Chmielinski (04:12):
I mean, it's truly another language and I
don't think that I will everbecome fluent.
And so what I've done is I'vetried to find native speakers
and I go to them and I say, look, I don't understand what this
directive means.
Or can you boil this down forme into something that makes
sense?
Or what do you want me to do inthis situation and that's how I
actually end up having animpact is I find the amazing
(04:32):
teams doing things.
I convince them and charm theminto letting me work with them,
and then we build thingstogether.
Randy Silver (04:38):
That is a really smart way of making it work.
That's very cool.
Okay, I wanted to talk aboutsomething that you mentioned in
your talk.
You were talking aboutresponsible AI, or a responsible
approach to AI, and you kind ofput it straight out there that
the way that we approach productmanagement as a whole is kind
(04:58):
of antithetical to beingresponsible, and that's because
we work really hard to maximizewhat we consider value, so we're
trying to go for the biggestham and we're doing things which
inherently leaves thingsmissing.
Can you talk a little bit moreabout this dilemma?
Kasia Chmielinski (05:16):
Yeah, I think it's kind of at the core
of product management and theway we build things, and it's
both the process that allows usto make small innovative changes
quickly that then we can scale.
So it's definitely beneficialto building innovative products,
but it also gets in the way ofbeing thoughtful and moving
(05:37):
slowly and with intention, and Ithink that it really ends up
being a question that PMs haveto face is how do I balance
these things, how do I balancethese approaches?
And so what I mean is that, asa product manager, you know, I
have certain goals that I needto hit.
I need to reach my OKRs.
I have some you know indicatorsthat I'm tracked against.
(05:58):
I want to get my bonus, youknow, or?
if it's a really small company.
I need to make sure that welaunch on time so that we get
the next round of investment.
I mean, these are really bigproblems, right, and they're
non-trivial.
Randy Silver (06:08):
And it's not an academic exercise.
Kasia Chmielinski (06:10):
It's not academic and it's fun.
It's super fun.
I think PMs were kind of likeMacGyvers we're just in there
trying to make things work, youknow, and that's okay, because
I'll get to them one day, butthe decisions they make in the
(06:31):
very beginning they end up beingthe DNA of the product, and
that means that all those otherpeople that were left out of the
initial set of users willalways be secondary.
You know, maybe eventuallythey'll be brought into the fold
, but it's always going to bekind of updating or refactoring
the product as it currentlystood, and that's where a lot of
the gaps emerge that end upbeing identified later as bias
(06:52):
or discrimination, or even justyou know folks that end up using
your product off-label, right?
They kind of take it, they dosomething else for it and they
find a way to make it work,because people are geniuses,
people are really smart, but itwasn't meant for them to use the
product in that way, and that'sbecause it wasn't designed for
them, and that's the kind of anissue that I'm highlighting here
.
Randy Silver (07:10):
So how do we deal with it?
I mean again, in a perfectworld, we would build things
more deliberately, morecarefully, we would take as much
time as we need.
We have enough of a problemwith getting accessibility baked
in at the beginning and doingit right, but we're always in a
hurry to.
Let's just get it out, let'sbuild something that doesn't
scale, then let's add on thisother stuff on top of it to make
(07:31):
sure it scales and we'rehitting all the regulatory
compliance and everything elsethat we need to do.
How do we actually deal withthis?
Kasia Chmielinski (07:39):
There are a number of approaches.
I think if someone had solvedthis, I wouldn't be here.
I wouldn't be here, I wouldn'tbe talking, I'd just be
following what they say.
And I've done it a number ofways, sometimes more
successfully than others.
I think we can.
(08:00):
Obviously, if you're startingsomething from the beginning,
you have a great opportunity tocenter marginalized users, move
the center right and actuallybuild for folks who are at the
margins.
And you can say I'm going toactually build for folks that I
would usually cut out of thefirst round because I'll
actually end up building abetter product for everyone if I
build accessible first right,for example, and so you can do
that if you have the luxury ofstarting from scratch.
Even then you're still makingchoices.
And so I think that, regardlessof whether you're starting at
the beginning or you're comingin later, a really important
(08:22):
thing is to actually bemonitoring and tracking how
things are going and to makesure that you have real feedback
loops with people, not justkind of check box.
I asked somebody did some usertesting.
I built a product, I launchedit.
Well, did you go back to themand show them what you ended up
building?
Did you get feedback on whatyou actually launched or did you
just do it as part of thedesign thinking process?
Right, and I think if we buildin places where we can actually
(08:46):
get meaningful feedback andtrack the use of these products
and change them over time, wecan start to address the gaps in
retrospect.
It's not perfect, but I thinkthat that's another approach if
you're kind of jumping in in themiddle as opposed to starting a
redesign.
Randy Silver (08:59):
Okay.
So a lot of times we talk aboutI'm getting started with
something and let's thin slice,let's find one use case that
we're going to do, and usuallywe choose the easy one.
And what you're advocating foris don't choose the most obvious
easy one, choose something alittle more on the edge, whether
it's marginalized people ordiversity, or whether it's
(09:19):
special needs of some sort,special needs of some sort,
because if you solve that andone of the classic examples
there is, you know, the use ofclosed captioning and things has
made video better for everybodynot just for people who are
hearing impaired.
Kasia Chmielinski (09:36):
Yep, exactly , and it's a trade-off again.
So if you want to design forthe most complicated
accessibility case, you want todesign for, you know, users who
are using five differentlanguages instead of just one,
and all of a sudden you have tohave a multilingual interface.
I mean, these things can getmore and more and more
complicated.
You have to decide, kind ofwhere your line is going to be.
To your point, you want toactually launch something.
I get that, but I think thatthere are ways that we've just
(09:59):
done things and we assume thatit's the right way to do it.
We should start with a processof saying what are we actually
trying to solve and are thereways that we can increase the
footprint of who will be able touse this product and it'll make
it easier for us down the line,right?
So if I build something andit's just this is a dumb example
but single tenant, right, soit's just for one user, but I
(10:20):
know that I'm going to beonboarding multiple At some
point, I'm going to have to makeit multi-tenant.
That's painful.
Do you want to refactor lateror do you want to make some
really basic database choices inthe beginning to make it easier
down the road.
These are the kinds of things Ithink that we can dial up and
down when it comes to the typesof trade-offs we make from the
very beginning.
Randy Silver (10:37):
So being deliberate with your
architecture around things of wemay not be turning this on now.
We know we're going to have to,so we're going to leave space
for it, even if we're not doingit today.
Kasia Chmielinski (10:47):
Yes, where it makes sense.
Randy Silver (10:54):
Where it makes sense.
Yeah, of course.
Okay.
And then you also talked a lotabout when you're in these
situations.
You have to sometimes make somehard choices about how you're
going to approach existing as aproduct person in a company that
is trying to realize value atpace and be responsible.
So one of the things that yousaid is one of the options we
have is that we can refuse tobuild and sometimes redirect.
(11:15):
Yeah, Tell us a little bit moreabout that.
Kasia Chmielinski (11:17):
It's kind of the product manager version of
voting with your feet.
Randy Silver (11:20):
Yeah.
Kasia Chmielinski (11:21):
You know, you're presented with some kind
of a product plan that perhapsyou did not build.
It comes as an edict fromsomebody up in an office
somewhere and they say thatshall build the following thing,
and you go oh my God, that'sterrible.
For whatever reason, it's moralor otherwise, and one option
that you always do have is torefuse to build it.
(11:41):
Now, this can be a disastrousdecision for, for example, your
career.
Right, Also, it's a privilegeto be able to quit a job,
because that assumes you can getanother job.
Right, Maybe you need apaycheck and all these things,
and so those things are real.
But there is the conceptualoption, at least, of saying I
will not build that thing.
A way to soften that sometimesis to say I hear you, that's
(12:04):
really interesting, Let mecollect some information or do
some research.
And then to go back and say thething that you want to build is
not the right answer.
Assuming this is true, theright answer would be the
following right, and you canactually redirect and say I
won't do the thing.
Essentially, you're saying Iwon't build your thing, but I'll
build my thing and it willsolve the same problem, assuming
you agree with the problem, andthat, I think, is one kind of
(12:25):
initial approach that you cantake.
That is, I would say, it's abold choice but it's definitely
one that we have.
Randy Silver (12:37):
But we talk a lot about.
You know, people are alwayscoming and saying can I have
this?
And one of the core skills of aproduct manager that you have
to learn early in your career ishow to say no without using the
word no, and I think that'sessentially what you're saying.
Kasia Chmielinski (12:48):
Sometimes there is power in saying no, and
I think that's essentially whatyou're saying.
Sometimes there is power insaying no, yeah, like I won't do
it, but that I think should beused sparingly, otherwise you
get a reputation for being theone that yeah, you can only do
that so many times.
Exactly, and so I do think youcan take a stand sometimes and
say I do not think this is theright thing to build, I will not
build it.
But again, that is very bold.
I've done it a few times, but Iwouldn't overuse it.
Mostly it is this other thingyou're talking about, which is
(13:10):
let me show you what I would do.
That gets to the same problem.
Or let me explain to you whythe thing that you want to build
is maybe not the right answerright now.
There's all kinds of linguisticand communications tactics you
can take there to basically movethem off of the path that you
don't want to be on and movethem onto the path that you do
want to be on.
Randy Silver (13:28):
We did an episode a few months ago with a guy
named Steve Hearsom who's anorganizational design and a
development consultant, and hewas talking about.
One of the real challenges forus is that in product,
essentially you're middlemanagement let's not beat around
the bush but you're being askedto influence and act and work
with people multiple levelsabove you in an organization and
(13:49):
above your pay grade, and it isa real challenge.
Kasia Chmielinski (13:53):
Yeah, and not only that, but you're not
actually in charge of anything.
Yeah, but you're accountablefor everything.
You have no real power.
You can't make anybody doanything really, but you need
them all to do things right, andso I think that there's a lot
of nuance in the role.
Randy Silver (14:11):
Yeah, if it was easy, everyone would do it right
, okay, another thing that youidentified as things that we can
do is we can build better.
What do you mean by that?
Kasia Chmielinski (14:22):
So for that one, what I mean to say is that
let's say that you're in thecontext of organization or a
product or a roadmap and youkind of disagree with some of
the decisions that are beingmade.
Well, you can decide how tobuild that thing in a way that's
going to be less harmful ormaybe more beneficial, right,
and so it's kind of an extensionof the redirect, but maybe a
(14:45):
little bit less brutal.
The redirect is they're goingin one direction and you shift
them to another direction.
I think the build better is yousee the product, you see what
the gaps are and then you try tokind of shift so that you can
fill those gaps and you canaddress those errors.
You can buy yourself more time,you can get the resources you
need, you can get the expertiseyou need, whatever it is to
actually address those issues asyou go.
(15:06):
And I think you know whateverit is to actually address those
issues as you go.
And I think in the world of AI,if we want to specifically talk
about AI, there's a realopportunity here to not see AI
as a product but rather as aprocess and, if you broaden the
aperture, to think about AI as aprocess that involves use case
selection, training, dataselection, the training of the
model, the development, thedeployment, the updates and then
(15:29):
eventually the decommissiontraining of the model, the
development, the deployment, theupdates and then eventually the
decommissioning of the model.
And you think of this as many,many steps where things can go
right and things can go wrong.
Then you're not just at the endof the whole process saying, oh
God, we have a problem, right.
You're able to then jump in andbuild better by saying let's
now identify issues earlier inthe process, let's have ways to
check the data, let's actuallyknow what we're looking for,
let's have a series of metricsthat we're tracking over time.
(15:52):
When we launch something, let'sassume that's the beginning of
its life as opposed to the endof its life.
Right, and now it's live.
Now we have to train it, wehave to make sure it's okay,
it's out in the world, it'saffecting the world, is it
drifting?
Are things?
I don't mean to antipomorphizebut, you know and I think these
are all now places where we canbuild better, if we have a more
nuanced and a broaderperspective of what the product
(16:13):
is.
It's more of a process than aproduct.
So these are the ways I thinkthat we can build better.
Randy Silver (16:17):
So when we are building without AI, we have
developed lots of tools, verypractical approaches to doing
testing and QA along the way.
We've got BDD, behavior-drivendevelopment, we've got
test-driven development, we'vegot unit tests.
We've got all kinds of otheressentially tick-listy kind of
things and processes.
With AI it tends to be a blackbox without an audit trail.
(16:40):
Is there anything practical ifwe're taking a responsible
approach that we can do, as yousaid, if we're making a process
of things we should be doingalong the way?
Kasia Chmielinski (16:49):
Yeah Well, ai is a broad term.
So when you say it's kind ofblack boxy, it depends on what
kind of system, it depends onwhat algorithm, it depends on
the deployment method and alsohow it's contained.
So I agree, if you go and youget an AI system from a vendor,
that's really black boxy becauseyou might not even be, it might
be in a cloud environment youmight not even understand how
(17:10):
it's working.
If you're building somethinginternally and you're using a
statistical system, an AI system, machine learning system that's
actually interpretable in someway, then there are definitely
approaches that you can take tounderstand the inputs and the
outputs, or at least thedistributions of the inputs and
the outputs, against some kindof benchmark or against some
kind of test that you wouldexpect over time.
When you start moving into thekind of generative AI,
(17:32):
probabilistic, you're buildingsystems where you're, you know,
cobbling a lot of thingstogether.
I would say my best kind ofsuggestion right now is we need
to be thinking about evaluations, we need to be thinking about
red teaming and these kinds ofabout evaluations.
We need to be thinking about redteaming and these kinds of
newer approaches to basicallyprodding and testing the models
Another way to do it from or inaddition to that makes a lot of
(17:55):
sense to make sure that you'rebuilding in a componentized way
so that you can isolate eachcomponent and test it for
certain things.
So, for example, if you'regoing to use an LLM for
something, instead of having theLLM do the retrieval and all of
the analysis and also the inputand the output and send
everything back to you, so itmakes all these decisions
(18:15):
internally, split it up intopieces so that you can actually
test each piece for what.
The one thing is that it'ssupposed to be doing, and that
way you can kind of isolateproblems at least at that level
too.
So as the technology isevolving and we're finding new
ways to measure bias and issueswith the outputs, you can at
least limit the surface area orthe footprint of each component
(18:38):
and test them separately.
Randy Silver (18:48):
Product people.
Are you ready?
The word on the street is true.
Mtp Con London is back in 2025.
We're very excited for Mind theProduct's return to the
Barbican next March.
Lily Smith (19:01):
Whenever I hear people talking about the best
product conferences, Mind theProduct is always top of the
list.
If you've been before, you knowwhat's in store.
Oh, that rhymes.
New insights, strategies,hands-on learnings from the
absolute best in the field, plusgreat networking opportunities.
And if you're joining us forthe first time, I promise you
(19:21):
won't be disappointed.
Randy Silver (19:23):
We've got one speaker already announced.
That's Leah Tarrin, who you'veheard on this very podcast.
With more to come.
From the likes of WhatsApp, theFinancial Times and Google, you
know real people working in thefield who will share real
actionable insights to level upyour game as a product manager.
Lily Smith (19:40):
Whether you're coming to the Barbican in person
on March 10th and 11th ortuning in digitally, join us and
get inspired at MTP Con.
London Tickets are on sale now.
Check out mindtheproductcomforward slash MTP Con to find
out more, or just click onevents at the top of the page.
Randy Silver (20:07):
So you talked about it as a process and when
you are being responsible withthe process or deliberate with
the process, anyway, you havethe ability to control and put
responsibility in at each ofthese steps.
But a lot of the times nowwe've got people coming in
saying can we just add AI tothis?
Or we've got third-partyvendors coming in and people
(20:28):
frustrated with how long ittakes us to do things in product
development.
So they're just buyingsomething in and I'll pick on
marketing or sales, because,well, we always pick on
marketing and sales, but theyhave vendors coming to them
saying, yeah, we can do that foryou and it's a shadow IT
function that comes in.
How do we add responsibilityinto our companies when we're
competing with those pressures?
(20:48):
Or is that just outside what wecontrol?
What we control and don't worryabout the things that are
outside our scope?
Kasia Chmielinski (20:55):
We should definitely worry about them,
because we'll probably beresponsible for the things that
they do.
So, ultimately, if yourliability is sitting with you I
mean, if your customers areinteracting with your system and
your system as AI that youdon't understand, your customers
aren't going to yell at thethird party.
They're going to yell at you,right?
Because?
Yeah.
So I think there should, firstof all, be a kind of vested
interest in understanding howthis stuff works.
(21:17):
I also am not againstoutsourcing things as a
principle.
You know, if your real valueadd is one kind of feature set
and the other features you couldget by, you know, contracting
with some vendor who's going touse AI, like that's not
necessarily a bad answer, but Ido think that at this point, we
should be talking aboutprocurement and we should talk
(21:39):
about the types of questionsthat you would expect a vendor
to be able to answer.
So, how does their system work,right?
Well, I mean, first of all, doyou even need AI to solve the
thing that you're trying tosolve?
So is there a need for them?
Okay, let's assume that youactually do want to work with
them.
Great, what is the systemtrained on?
How do they train it?
How does it work?
How are they measuring success?
(22:01):
What's their ground truth thatthey're comparing it to to tell
you whether or not their modelis accurate?
How do they plan to update andmonitor this once it's in your
environment?
How often can you expect to bereceiving updates from them?
At what point would theyconsider decommissioning the
program?
Or would you considerdecommissioning or turning off
the contract and then makingsure those things are really
(22:21):
baked into the contracts and theprocurement process so that
there's some accountabilitythere between you and the vendor
?
I think these are the kinds ofthings that we have to consider
if we're going to be contractingwith third parties.
Randy Silver (22:32):
Yeah, sounds outside the scope, unfortunately
, of what most product managershave in their job.
Kasia Chmielinski (22:37):
I mean, a lot of things are outside the
scope of what's there.
What even is our jobdescription?
It's not very clear.
Randy Silver (22:43):
Fair enough, but yeah, as you were saying,
systems thinking is incrediblyimportant and the ability to
discuss this with other peopleand bring it to light is still
incredibly important regardless.
Kasia Chmielinski (22:53):
Yes, I mean, if you are a product manager
whose function in the companymight be replaced by some kind
of outsourcing, I think you dohave, then, a bit of a
prerogative to highlight.
Hey, before you decide to putme on a different project, let's
make sure the thing thatthey're doing is actually
similar to the problem thatwe're trying to solve and that
we're doing this in aresponsible way.
(23:14):
I think you have a little bitof power there, but you know
it's tough being a productmanager.
It really depends on thesituation.
Randy Silver (23:19):
Not for the timid.
Yes.
No but being a PM probably isnot for the timid, so you might
want to find a different job.
Okay, you also said there wasanother one about creating new
solutions, and this is whereyour other project comes in.
Really well, so talk about thata bit issues in the product
(23:47):
development process.
Kasia Chmielinski (23:47):
You can refuse to build things right If
someone shows you somethingthat's morally contentious.
You can redirect people, youcan decide to stay within the
organization and build better,but in some cases, the gap is
not even attempting to be filledby anybody, and so your only
option at that point is tocreate something new.
And the good news is that, asproduct managers, I think we're
really well positioned to buildstuff.
The good news is that, asproduct managers, I think we're
really well positioned to buildstuff, and we already understand
(24:11):
that we need to have, you know,a multi-stakeholder, kind of a
panel of folks and voices tohelp us build things.
We know how engineers think anddesigners, and we can bring
together, you know, the higherups and the folks who have
funding and all of this tobasically become little mini
CEOs of things, and so this iswhat I did with some folks in
2018.
My team's going to be so upsetI never remember.
But it was a while ago we allhad a fellowship at MIT and
(24:33):
Harvard.
It was a joint fellowshipcalled Assembly and we were
charged with thinking about theethics and the governance of AI.
And they said you've got fourmonths, here's a bunch of coffee
and some cookies, and come upwith something anything.
So we were puzzling over whatwe thought the kind of problem
space was, and at that timethere were a bunch of news
cycles about the problematicoutcomes of AI.
(24:56):
Bias and discrimination is kindof rising social consciousness
around this and what we noticedas practitioners was that these
media articles were reallyfocused on the output.
They're saying these systemsare deployed, they're in
production and they're hurtingpeople.
But all of us could kind of seethat most of these issues were
(25:18):
issues that were just carboncopies of issues in the training
data and nobody was focusing onwhat about the data?
Why is no one talking about thedata?
It's not public, it's nottransparent, no one knows what's
going into it.
But then they're shocked whenbad things come out the other
side.
Is anybody looking at the data.
And we started digging aroundand even from our own experience
, we knew that there are nostandardized data tools and no
(25:42):
standardized data approacheswhen it comes to identifying a
data set for use in AI.
And we were again in anacademic institution and we're
drinking a lot of coffee, we'reeating a lot of cookies, and
there was a box of cookies and Idon't remember who it was, and
they turn it over and they werelike there's a nutrition label
on this box of cookies andbefore I eat the cookies, I can
(26:02):
see what's inside.
Shouldn't we like have the samething for data?
Just like something like that?
And so we call ourselves theData Nutrition Project.
We emerged from that fellowship.
We make nutrition labels fordata sets and the idea is that
you have a standardized tool anddesign that allows you to
understand what's in a data setbefore you use it.
And for a practitioner whowants to do the right thing you
(26:24):
know we're not certificationbody, you know we don't validate
all the things but for somebodywho wants to do the right thing
, we give them an easy way tocommunicate what's in the data
and then an easy way to digestwhat's in that data.
Randy Silver (26:36):
What kind of things do you have on the label?
Kasia Chmielinski (26:38):
There are a bunch of different components
and we're actually in chats tosee whether we need to expand
this for different domains ofdata, like large language, model
data sets, you know.
But the kinds of things that wehave are actually more
qualitative, because it turnsout that qualitative information
is almost more important inthis kind of a scenario than
(26:58):
quantitative, quantitative stats.
People will run a datascientist will run those quickly
and say what's the distribution, show me this number of missing
fields, blah, blah, blah.
There's some kind of core setof things you're taught to do,
but qualitatively, I want tounderstand how did you fill in
that missing data?
How much was missing?
How did you clean it?
Who paid for this?
All of these kinds of things youwon't see in the data set.
(27:20):
So we surface things like that.
We also have a little modulethat explains to you what the
intended use is.
So this is what the data wasintended to be used for.
This is the way that it hasbeen used.
This is what you should not dowith the data.
These are known issues andareas for practitioners to
surface whether the data set hasundergone ethical assessment or
technical assessment, thesekinds of things.
Randy Silver (27:41):
And how are you seeing it being used?
Kasia Chmielinski (27:43):
There are a few prominent ways that people
use the label.
One is in kind of a industrydata set context, which is
honestly what we thought ofinitially, and we've partnered
with organizations like the UN,microsoft Research, various
places at Harvard to talk aboutdata and talk about labeling
that data and improving themetadata that already exists on
(28:06):
those data sets, and that'sreally cool.
So in some cases, folks haveactually used our schema In
other places.
We've taken components of oursand added it to theirs and those
are live and out there.
There's also an entirely secondapproach that we didn't really
consider, which is that thelabel is very, very useful for
(28:27):
education and for teaching andso there are a number of
curricula out there that use thelabel to basically teach data
science students how to thinkabout data and how to build data
sets and what a good data setwould look like.
And DNP Data Nutrition Projectdoes a lot of kind of tours
(28:47):
where we give lectures and wework with students and we kind
of jump in give workshops,because it ends up being a
really great teaching tool.
Randy Silver (28:55):
Fantastic, yeah, and have you seen it solve any
problems?
The difference in output andoutcomes in this one.
Kasia Chmielinski (29:04):
Yeah.
Randy Silver (29:05):
Usage is an indicator that it's being used,
but what for?
Kasia Chmielinski (29:09):
Well, here's an interesting one.
One kind of thing that wedidn't really expect was that
the quality of data output goesup when someone knows they have
to put a label on it, so that'skind of a secondary effect.
We really built a label to makeit easy to find data and use it
, but we were working with adoctor at Morrill Sloan
(29:31):
Kettering who builds data setsof skin imaging and is very
aware of the potential bias inthat and kind of the
distribution of skin colors isjust not very well
representative, as we all know.
And so she was talking to usand said since working with you
guys and building these labelsfor these large skin imaging
(29:51):
archives, the quality of my datasets has actually gone up,
because in building the label, Irealized that there are some
things I should have done a longway, because you're asking me
such questions about decisions Imade.
I should have made differentdecisions.
So the next time I make thedata set this is like a data set
they refresh once in a whilethe next time I make the data
set, I make different decisions.
I think that's the kind ofimpact that is really cool for
(30:13):
us, because it's not just aboutbringing transparency to data
sets, it's also about improvingdata set curation period.
Randy Silver (30:20):
And if people are interested in learning more
about this and getting involvedor applying it, where should
just Google the data, nutritionor yeah, datanutritionorg, but
if you Google data nutritionproject, it would probably come
up.
Kasia, this has been fantastic.
I have one last question.
You closed today with talking alittle bit about your mission,
(30:40):
and a personal mission statementis something not everyone has,
but I found it really inspiring.
So tell us a little bit moreabout your mission.
Kasia Chmielinski (30:48):
Oh, I don't know which part you saw.
Randy Silver (30:51):
I don't know that.
I have a very clear missionstatement.
Okay, the slide I saw, I'llread it directly.
Yeah, there will always bedangerous gaps in technology
that humans need to fill, but wealways have opportunities to
refuse, redirect, build betteror create new solutions to make
technology that works foreveryone.
Kasia Chmielinski (31:06):
I mean, I agree with that, so I'm happy to
I also wrote it.
No, I mean, I think that's itand it's maybe that feeling, not
always in those words, I think.
My own experience is that Ibuilt these systems and over
time I realized that not all thesystems actually worked for me,
Right.
So I'm non-binary, I use they,them pronouns.
(31:26):
I built systems that wouldclassify me in a binary and I
literally wrote these programs.
Or you know, during COVID, Iwas helping to automatically
assign race and ethnicity tomissing data fields so that we
could understand whether therollout of the vaccine was
equitable.
And when I put my owninformation into this, it came
out with the fact that I waswhite and that I was female and
(31:49):
all these things that I actuallydon't identify as right.
That could be dangerous.
Funny at first, and then it'sdangerous.
I use a hearing aid, right?
So I end up being the margin.
I end up being in the gaps ofmost of the technology that I
have built over my own careerand, as such, at some point I
decided no more.
I'm now going to focus more ofmy time and energy on filling
(32:11):
those gaps and helping othersmake better choices to make
technology that works better foreverybody.
Randy Silver (32:16):
Thank you, that was fantastic, thank you.
Lily Smith (32:30):
The product experience hosts are me, Lily
Smith, host by night and chiefproduct officer by day.
Randy Silver (32:36):
And me Randy Silver also host by night, and I
spend my days working withproduct and leadership teams,
helping their teams to doamazing work.
Lily Smith (32:47):
Luran Pratt is our producer and Luke Smith is our
editor.
Randy Silver (32:49):
And our theme music is from product community
legend Arnie Kittler's band Pow.
Thanks to them for letting ususe their track.
Thank you.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
My Favorite Murder with Karen Kilgariff and Georgia Hardstark
My Favorite Murder is a true crime comedy podcast hosted by Karen Kilgariff and Georgia Hardstark. Each week, Karen and Georgia share compelling true crimes and hometown stories from friends and listeners. Since MFM launched in January of 2016, Karen and Georgia have shared their lifelong interest in true crime and have covered stories of infamous serial killers like the Night Stalker, mysterious cold cases, captivating cults, incredible survivor stories and important events from history like the Tulsa race massacre of 1921. My Favorite Murder is part of the Exactly Right podcast network that provides a platform for bold, creative voices to bring to life provocative, entertaining and relatable stories for audiences everywhere. The Exactly Right roster of podcasts covers a variety of topics including historic true crime, comedic interviews and news, science, pop culture and more. Podcasts on the network include Buried Bones with Kate Winkler Dawson and Paul Holes, That's Messed Up: An SVU Podcast, This Podcast Will Kill You, Bananas and more.
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.