171 - Who Can Succeed in a Data or AI Product Management Role? - Experiencing Data w/ Brian T. O’Neill (UX for AI Data Products, SAAS Analytics, Data Product Management)

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:30):
Welcome back to Experiencing Data.
This is Brian T. O’Neill.
Today, I’m going to chat with youabout my vision for what a data and
AI product manager should look like.
This is actually in responseto a reader’s question.
I had someone on the mailinglist—as most of you know, I

(00:51):
have an insights mailing list.
I publish every other Tuesday betweenthese podcast episode drops, sometimes
a little more frequently—and thisreader was in a career change, you
know, not a drastic one, but wasparticularly thinking about moving,
I believe it was out of design andbusiness intelligence, data visualization

(01:12):
type role, and curious about possiblyexploring data product management.
And he had mentioned that he perceivedmost of my content was for people
that are already in management andexecutive leadership positions, and
he was wondering if I had ever donean episode about stepping into this
type of role with less experience,not coming from product management, or

(01:36):
not having been doing it for a while.
So, I thought that was probablyworth stepping into, whether that’s
you or you have a team and you’rethinking about hiring in this space.
I thought I would give you my veryopinionated take on this, about what
I hope to see as a product personworking in this space, like, if I

(01:56):
was to work with somebody, what theskills are, and really the mindset
that I would hope that that they wouldhave in the work that they’re doing.
And as with almost everything on thispodcast, I’m coming at this whole
thing from very much a business anduser experience perspective, and
not from a technical perspective.

(02:18):
And I’m going to talk a little bitabout the technology side here, the
requirements and that piece of it, butI believe there’s plenty of content
out there about this, and most ofthe people listening to show probably
already have most of that knowledge.
I know we have listeners here comingfrom more of the software product
management as well as user experiencedesign, so I’m going to talk about

(02:41):
those transitions, like, if you’removing across those roles, maybe what
some of the biases are, and things towatch out for, and all of that as well.
So, let’s talk about that.
Maybe we should talk about that first.
So, who can transition into thistype of role, and what is this role?
So, I’m kind of puttingthese two things together.
AI product management often, inmy perspective, seems to be very

(03:04):
much attached to software companiesthat are primarily leveraging AI or
trying to and so they tend to usethis AI product management role.
I’m seeing less of that in the internaldata teams where you tend to see data
product management more, which, forme, feels more like an umbrella term
that may include traditional analyticswork, data platforms, often, AI

(03:32):
and machine learning, that type of thing.
I’m probably going to talk more—I’mgoing to frame this more in the AI space,
primarily because I think AI tends tomore frequently capture the end-to-end
product than data product management does.
And the reason I say that is becausedata product management and data

(03:53):
products in general have manydifferent definitions, as we’ve
talked about on the show many times.
Data product management can still verymuch mean basically building reusable
data assets, data platforms, things thatrequire further refinement before they
are leveraged into actual applicationsand tools that provide end-to-end value.

(04:15):
And that’s not what I tend to seethe gold standard for the role.
I know those people exist, and I’m notsaying that you may not need someone
managing a data platform, for example,but in my vision and opinion of the
world, if your whole existence is toprovide the platform only, and you’re not

(04:38):
responsible for the actual value that’sgoing to be leveraged out of the platform,
then there’s kind of a gap there.
And I realize another PM might haveresponsibility for that, but I kind
of have this just purist mindset aboutit, I guess, and I think there’s just
so much work happening that happens inthese isolated buckets where one person

(05:01):
owns this piece of infrastructure,and another person owns this piece of
infrastructure, and this person owns themodel and the math—you know, as the Brits
would say, the maths behind it, and thenyet another person is supposed to refine
that into something, and on and on andon, and we end up just with a lot of
cost and not a lot of value a lot of thetime, so I’m going to still paint this

(05:23):
picture as I think it should be painted.
So, keep that in mind here.
So, who can transition into an AIand data product management role?
What I tend to see the most is the dataproduct management role tends to exist
mostly in the internal data crowd space.
So, this is for data teams that aretrying to leverage, usually working in

(05:45):
large enterprises, and trying to turnthat data into new forms of value.
I see that less in thesoftware company area.
That’s where I tend to see the AIproduct management roles, et cetera.
Regardless of where you see it,I would also say that I tend to
see more grooming of these peoplefrom different roles internally.

(06:11):
Particularly on the enterprise dataside, it’s going to usually be someone
that already was working with a datateam that has been groomed into, or
at least retitled into a data productmanagement, then they seem to throw
training at them, and they try towork with someone that they already
have and turn them into this role.

(06:33):
So, it’s kind of like change thejob title, add new skills, and then
I think hope that they will growinto this role and start operating
the producty way, whatever thatmeans inside of that organization.
On the AI PM side, I think, whilethere’s a lot of people, I think trying
to jump into this space because theysee it as being a hot place to be, I

(06:54):
think where you’re going to probably seemore successful people is tends to be
someone that’s come out of a traditionalsoftware product management role.
They understand product already, andthey’re really focusing now on AI
specifically in their product work.
So, who can transition here?

(07:14):
I tend to have a very open view thatanybody can transition into just about
any role if you’re willing to do thework and make the changes required to do
that work, and you’re willing—possibly—tohave to step back to do pro bono work,
for example, to build a portfolioor to do something that can show the

(07:35):
results that you know what you’re doing.
And there will always be plentyof jobs out there that just
list raw technical skills.
I’m still hearing this all the time,even at the chief data officer level—and
Kyle Winterbottom talks about this quitea bit—about how a lot of companies,
they keep hiring for the wrong stuff.
They all want to have the same valuecoming out of their chief data officer,

(07:58):
but then when you look at the jobdescriptions, the company clearly
doesn’t know how to hire for the role.
I think the same thing.
I don’t sit and look at job descriptionsregularly because I’m not in that space,
but when I do see them, I often will seelargely technical requirements for these
types of roles, and it tells me that thecompany really doesn’t understand what

(08:21):
product—and this is more true, again,in the enterprise data space—the company
really doesn’t understand softwareproduct management and this idea of
leveraging how to do that work internallyagainst their internal data products.
They really don’t getthe joke, so to speak.
And that’s probably understandableif they’ve never had this role there.
I think they’re trying to get all thebenefits because they hear that a lot

(08:44):
of people are working this way, and theywant to get those benefits, or maybe one
of the big four firms told them that thisis the way you should be doing stuff,
and so they’re trying to bring people in.
And I still applaud that work.
Even if they’re not sure, they’re takingthe first step and they’re saying,
“Hey, we’re not quite sure what thislooks like, but we’re trying to move
this direction.” So, I think that’sokay. So, who can move into that? Well,

(09:07):
again, I think the reality is that, onthe enterprise side, they’re looking for
people that have come out of data roles.
So, if you haven’t come out of adata-related role, it may be harder
to get a position in that space unlessyou have a very compelling portfolio or
resume that shows results doing this work.

(09:30):
And you might have to go out and do someprojects or work on some products or do
some pro bono work to build somethingthat will show the benefits that you have
created for users or for an organizationin order to step into that type of role.
There’s three disciplines I’m going tokind of talk about moving into this role.
So, coming into AI and data PM, fromdesign and UX, coming into it from

(09:56):
data engineering, or just broadlytechnical spaces, and then coming into
it from software product management.
I think software product management andmoving into the AI product management,
as long as you’re not someone that has20 years of experience, and you really
have two years of experience, and then18 years, repeating the second year
of experience over and over again, ifyou’ve actually really had a robust

(10:20):
product management background acrosssome different types of products, and
you can show that the domain doesn’tnecessarily stop you from producing
value, I think you will have theeasiest time, probably, moving into AI
product management because you’ve shownthat you can adapt across different
industries or whatever it may be.

(10:42):
And it also shows that, you know,arguably, what I think is probably
the trickiest space, which is not thetechnology, it’s all of the glue work
that product managers need to do withrelationship building, navigating
politics, especially when you get intolarger organizations, understanding
how to talk to sales and marketing,if that’s more on the commercial side.

(11:05):
But really this idea of you have allthese different stakeholders, you may
have all these competing differentneeds, and deciding what gets love
from your design and engineering anddata science teams, that’s often where
the hard work is because deliverywork is getting easier and easier
today, ironically because of AI.

(11:26):
So, the deciding what to do ishardest and PMs that have been
doing that kind of difficult work,I just think you’ve got most of the
skill sets that are more beneficial.
As long as you’re open torealizing that, particularly in
the AI side, we’re in a new time.
And I think that there’s a—thisis for the web, if you’ve been

(11:49):
through all the different stages ofthe web, like I have, it’s kind of
like these big forks in the road.
It’s like, Web 2.0 came out.
It’s like, are you going tokeep doing the old thing or are
you going to adopt to social.
And then I remember when, like, AJAXcame out, and you finally had dynamic
web interfaces where you could pull datawithout refreshing the entire screen,

(12:10):
and it’s like, basically, you haddesktop-type software experiences that
could finally come into the browser.
Are you going to keep doing it the oldway, or are you going to start thinking
more like we now have a platform to justbuild software right in the browser?
Right or left?
Which way will you go?
You can kind of keep going down thisroad and having these forks there.

(12:32):
And I think if you’ve kept understandingwhen you’re at a fork and then making
the correct turn at the fork, goingright or going left—if left means keep
going to doing the same thing; rightmeans, hey, I’m going to go down this
road because it seems like that’s wherethings are going—I kind of see this,
all of what’s happening with GenAIas one of these forks in the road.

(12:53):
So, you can either embrace it, or you cankind of keep doing what you did yesterday.
But back to these roles here.
So, product management probablyhas the easiest time here.
They’re going to need to obviously,really understand more of the tooling,
but I feel like it’s so much easier nowto get a grasp on all of this tooling,
to understand what’s possible out there.

(13:15):
It’s never been easier to actuallycatch up with this stuff, the amount
of content that’s available, usingAI to ramp up on these skills.
There’s just, there’s a lot here.
So, that’s my take.
Let’s just move into—these aren’tnecessarily ranked, but I think
that that one has the easiest time.
Let’s talk about designers next.
I’m going to include data visualization,user experience research, user experience

(13:39):
design, product design, all thosetypes of broad design, category roles.
So, moving into data and/or AI productmanagement, first of all, you don’t see
too many—I don’t hear about too manydesigners wanting to move into DPM roles
because oftentimes, I don’t think there’sa lot of heavy UI and UX all the time in
that space, or at least the teams thatare doing that work feel that’s somebody

(14:04):
else’s job because they’re not reallydoing end-to-end product thinking the
way I talk about it, so therefore, a lotof times they don’t see the application,
the user experience, the human adoption,the change management, they’re just
not looking at the world that way,even though I think they should be.
So, I think designers and user experiencepeople could definitely bring value

(14:25):
into this role, but I’m not convincedthat most people hiring for this role
will understand that mapping, so theyprobably will not see the value that you
can bring into the role unless you cantranslate your work into results that
directly address the challenges thatthe hiring company thinks they have.

(14:47):
So, even though you might think youhave the skill set to bear that’s going
to help them, they have to perceivethat as actually mapping on to their
problem space, and if you’re usingdifferent jargon and lingo to talk
about this, I just don’t think a lotof data and AI PM roles are going to
be able to map that work onto what theyperceive as being valuable for the role.

(15:10):
So, I think maybe this group will havethe hardest time moving into this.
The exception for this wouldprobably be more in the software
AI product management space.
If you have been a true product designer,which for me, means you’re really
looking at both user experience andthe human part as well as the business,

(15:32):
and so your design work has always beenbalancing the two of these, and you’ve
been, really, a true counterpart to aproduct triad, or however your product
groups were made, and that reallymeans to me, you’re almost like—you and
your product management counterpartsare essentially almost equals.
It’s like multiple brains all forminginto one product brain, and that’s how

(15:55):
you’ve been operating, as opposed to,I’m waiting for a product manager to give
me a list of work to go deliver against.
If you’re the former, I think you havea chance in this space, particularly
if you’ve been working on heavy dataapplications, analytics applications,
AI things, and you’re able to talk thatlanguage, and you’re able to attach

(16:15):
business value to the work that you’vedone and show that, I think you have more
of a chance here, but I think your uphillbattle is going to be your perceived lack
of technical skill here and not being ableto interface properly with data science
and engineering people in this space.

(16:36):
And there’s a lot of technicalthings to know about in this space.
And the technical teams arealso struggling with this.
I mean, I was just talking to one ofour data product management members the
other day—one of our founding members,actually—on one of our monthly team
times—so we have these—every month, wehave TEA[M]—T-E-A parentheses M—thank

(16:58):
you to [Marnix] for adding the M, whatused to be called TEA time; now it’s TEAM
time—but we have these, like, monthlyno agenda meetup opportunities online,
where it’s just a social time for 30minutes, drop in and it’s just a chance
for us to stay in touch with each other.
And so, I was catching up with thisleader, and she was telling me that,
you know, one of the challenges withthe—they’re deploying a GenAI-based

(17:22):
feature into—this is an accounting-relatedsoftware product, and they were launching
a GenAI feature in there, and justthis idea that how one small change
with, like, a pipeline or somethingthat may seem trivial, can have a
downstream effect—or even an unknowneffect—on what the model is generating.

(17:44):
In this experience, it was a chat,conversational-type interface inside of
the product that allowed you to do some,you know, some different tasks inside the
product through conversational interface.
And even the data teams are strugglingwith this because it’s not always
clear why this small change hereis having this downstream impact.

(18:05):
So, given that data and engineeringteams are having this challenge already,
if you’re coming from design and you’vereally not fully embraced the technology
part of doing product work, and you’vekind of, I mean, frankly, had your
head in the sand, or you’ve been partof that camp that says, “Designers
shouldn’t code. That’s not our job.
We don’t need to know about this, andI want to stay creative because I was

(18:28):
hired to be creative, and if I know moreabout how to build the products, then
therefore, I will not be as creative,”I am, just so not part of that camp.
I think that argumentdied 10 or 15 years ago.
I know some people still have it.
I’m very much of the camp, ifyou’re a designer, you need to know
your medium, whether it’s brushesbecause you’re a painter, or, you know,

(18:52):
if you’re a sculptor, to know the typesof material that you’re working with,
and if you’re doing software-relatedwork, whether it’s AI software or
not, you need to know how to code.
At a basic level, you need to understandhow APIs work, you need to understand
how models work, and if you don’t havethese skills and you don’t want to gain
them, I think operating in this roleis going to be very difficult for you.

(19:17):
It doesn’t mean you should be spendingyour time doing coding and all this,
or even doing code reviews or any ofthat, but you’re going to be spending
a lot of time, particularly with GenAI,on figuring out how to drive quality,
where quality—you’ll understand what userexperience outcomes look like, and what

(19:38):
quality looks like there, but there’salso the business side, and then there’s
the whole probabilistic side here, wherethis isn’t just a matter of defining all
the states that need to be managed withinthese applications and making sure that
we’re handling all those gracefully.
That’s no longer theworld we’re operating in.
So, you’re going to have this skill.

(19:58):
And I feel like a lot of user experienceprofessionals out there don’t have this.
This is still, like, a special 25%.
I mean, again, these are all just myperceptions about what’s out there,
but finding very technical designers,and I mean designers that understand
the tech that can hack togethertheir own—whether it’s vibe coding

(20:21):
or low-code, no-code, solutions,they just have a technical mindset.
They have a builder, hacker mindset.
They want to prototypetheir own solutions.
They’re willing to get in the weeds tounderstand how things work, just enough
such that they can be more productiveas a designer or a product manager.
I think it’s a rare skill, and a lotof UX people just don’t have it, or

(20:44):
they’re not working on it, frankly.
So, you’re going to need toreally understand this space.
I would probably be taking trainingon this if I was in that kind of
role and I didn’t have these skills.
Personally, I like to learn by doing, andthen I’ll dive into training or learning
through the process of doing something.
So, I’ll build my own application whenI get stuck, then I’ll try to go learn

(21:07):
about something conceptually, butalso through the practice of doing.
That’s more my style of liking to learn.
But I think an uphill challenge here,unless you really can show what you’ve
done here, you’ve done some volunteerwork or something to, again, have your
portfolio represent that you couldactually manage a technical team.

(21:28):
Even if you’re not directly managingthe engineering and the data science
and machine learning engineers anddata engineers and all of this,
you’re just going to be spending alot of time doing that, and not a lot
of time in the UI and the UX space.
So, that’s my take forthe designers out there.
Let’s talk about now maybethe largest, I’m guessing, the

(21:50):
largest part of this audience.
And it’s always funny, after all theseyears, you still don’t know exactly
who’s listening to this show, exceptwhen you all write into me, or if you
join the mailing list and you say,“Hey, that’s how I heard you,” or
you shoot me a message on LinkedIn.
I always appreciate getting those thatactually, whether it’s a compliment
or just a chain, like, I’d like tohear more of this kind of content.
All that stuff is super welcome.

(22:12):
I also just it’s really helpful for mewhen you drop me a line because it lets
me know, like, oh, who’s this person?
What kind of company are they at?
What kind of role do you have?
It tells me who’s listening tothe show and it’s really helpful,
so I always enjoy getting those.
But anyhow, so let’s talk about coming atthis from the data and engineering side.
This is the classic track for data productmanagement the way I tend to see it.

(22:36):
I think most companies wantto groom this role in-house.
My biggest concern there is thatyou end up with job title changes
and not necessarily the benefitsthat are supposed to come from this.
I do like learning by doing, but you know,and you’ll see—like, Marty Kagan talks
about this a lot, this idea of, like,having a coach and having someone senior
in there who can coach your other PMs,particularly because there’s all this

(22:59):
stuff that you’re not necessarily goingto get in a class or a course; it’s going
to come from experience doing the work.
Because this work is not delivery work.
It’s not a technical work wherehere’s a challenge, it’s technical
and there’s a puzzle to fill, andgenerally, there’s a right answer.
There might be a few different rightanswers, but there will be this moment

(23:19):
where you crack the code and—maybeliterally crack the code—and then
you’ll get through that gate, andyou’ll be on to the next thing.
I don’t think most productmanagement feels like that.
Product management just feels likethis ongoing thing that never ends.
You’re placing bets all the time.
A lot of times they have longer horizons.
It can take a while to get feedback on it.

(23:40):
It’s a very different space, and soI’m really curious to see what’s going
to happen with the teams that aretrying to implement this, that are only
doing this through grooming and lighttraining, especially if they’re primarily
taking training on things like projectmanagement and Agile training, and
like, using product management softwareand kind of all the hard skills stuff.

(24:06):
That’s pretty easy to get trainedon and to, kind of, act the part.
Creating the actual value, though, thisis really why are we doing all this?
Well, the product manager issupposed to be the buck stops
here on delivering benefits.
That’s really what this job is about,is not creating the outputs and

(24:27):
being a project manager for all thetechnical work that’s going to happen.
And this is often what happens whenyou—even PM, I hear people sometimes
interchange the word project managerand product manager and those are
entirely different things to me.
The project manager often is not in chargeof whether or not this project even makes

(24:48):
sense and whether this project is going toproduce value; their job is to manage the
assigned project that was given to them.
And I am not saying that that’s notimportant, but if you’re spending all
your time doing project management,you’re probably not a product manager
as the next company that’s going tohire you is going to see it because
that’s not what this role is about.

(25:09):
Most of the time, if there’s beenexecutive support for hiring product
managers, the assumption is that this isthe role that’s going to make sure all
this technical work and all the designand user experience work, all the delivery
work, is going to roll up, and it’sgoing to produce value for the company.

(25:30):
Change will happen based on yourdecisions about what we should do.
And if you can’t do that work, oryou’re not associating your work with
benefits delivery, outcome delivery,I think there’s a huge gap here.
And frankly, I’m really concerned aboutdata teams and when people come out of

(25:54):
data and engineering teams into thisrole, and they haven’t had the support
on all of the non-technical aspectshere, and instead, they’re kind of
looked as, like, a technical lead now,who’s going to manage all the parts
and then the delivery of some outputat some point, but with a new title.

(26:14):
That’s a tough space to be in because Ialways wonder, hmm, upstairs, somebody
probably sold this transition to doingdata products, and maybe they sold this
idea that we’re going to have these teams,and these product managers are going to
own the value delivery, and then they’regoing to have their technical counterparts
working, kind of, with them or underthem—whatever, however the management

(26:36):
structure looks—but they’re not actuallyenabled yet to really produce this value.
They don’t have the high-level supportthat they’re going to need to deal
with all the social, the politicalaspects inside the company, all the
blockers to getting access to customersand perceptions that you’re really
a cost center, you’re a deliveryteam, just give me what I asked for.

(26:58):
All that stuff there, it’s anuphill battle because you don’t
have the skills you may not havethe executive support for that role.
I think it’s a tough place to be.
Obviously, you’re going to have thissuper-strong asset on your side,
which is, you’re going to understandall the data parts and the data

(27:20):
science aspects and the engineering.
You’re probably going to be very wellwired and positioned to understand how to
predict where there’s going to be potholesand obstacles on the way, model quality
and drift and all the things that businessis going to need to look out for, you’re
going to have a much better sense of thatthan anyone else, and that’s really great.

(27:40):
I mean those skills are highly valuable.
But it’s really transitioning into thechange management piece and making sure
that you understand, well, what does itmean to ship a small increment of value?
And we often talk about, well, we got tobuild these models first, or build all
this plumbing and all this, and it tendsto be these giant monolithic projects all
the time, and that’s not really part ofwhat a product person’s mindset should be.

(28:04):
A product person’s mindset really,in my opinion, should be about how
regularly and quickly can we deploy ahypothesis into production, get feedback,
learn from it if it didn’t work.
If we did create some value, how dowe double down on that and continue
to create value whether it’s for thecustomer and the end-user or for the

(28:26):
stakeholder, or ideally both, sincereally the role of any PM is user
experience outcomes and business outcomes.
It needs to be both of those things,and sometimes those are at odds.
So, lots of benefits if you have adata and engineering background moving
into this role, but all of this softstuff that I’m talking about, all the

(28:46):
politics management, frankly, knowingabout design, user experience, change
management, user adoption, resistance.
We’re hearing all this again aboutresistance to adopting these tools,
there’s people that feel threatened bythese solutions, and if you’re brushing
all that off as, like, well, you’rejust behind the times and this and that,

(29:07):
you’re going to end up with technicallyright, effectively wrong solutions.
The same thing again, it’s just yourjob title might be different, but
if you keep doing yesterday’s workwith a new title, I wouldn’t expect
any new results as a result of that.
You’re really going to need to study upon those skills such that the technical

(29:28):
work your team that you’re now managingand not necessarily doing the delivery
and coding work and the model buildingand all of that, you need to really
start thinking about what is the humanand cultural change that’s going to
be required to get this thing adopted,particularly if the solution is perceived
as being disruptive and not necessarilysomething that people were asking for.

(29:50):
They’re eager to get their hands-on,the human dynamics here are
really important to understand.
As most of you know, I tend to see thischallenge of user adoption is often
the blocker to the business value,particularly—I mean, this is really true
for commercial and internal data products,but if you don’t have human adoption

(30:10):
of the work, of the applications, thetools, however, your solutions manifest
themselves, if users don’t care, theydon’t like it, they can’t use it, they
don’t trust it, it kind of doesn’t matter.
You’ve just been doing basicallyR&D work and cost center work.
Your bet did not pay off.
And so, how do we make surethe last mile isn’t where all

(30:32):
this great technical work died?
And so, this is what my training tendsto be focused on, you know, my Designing
Human-Centered Data Products courseand the seminar, the training that I do
with private teams, these are evergreenskills that I think anyone in a data or
AI product management job needs to haveif you want your work to create value.

(30:57):
I don’t think of my training as beingspecifically for, like, end-to-end
here’s how to become an AI or dataproduct manager; I’m focused really on
the blockers to adoption, the changemanagement being built into the process
of building these solutions, so it’snot something that happens downstream by
yet another part of the organization whotries to force some new solution down

(31:20):
people’s throats against their will.
I don’t think that’s howgreat data products are built.
I think we try to design the changemanagement into the solutions.
So, my training is really focused inthat particular space of all these
non-technical skills that are required.
So, it’s really oriented for, youknow, people in data engineering and/or

(31:42):
product management trying to move intothe data product space, and they’re
struggling with that, the value piece,the adoption piece and all of that.
So, this value piece is probablythe most important thing,
and I want to focus on that.
You know, this is something I talk aboutin my training in the seminar quite
frequently is, how do we attach financialvalue to the work that we’re doing?

(32:05):
And this is both art and science,but it’s very much a language that
anyone in a product management roleneeds to be comfortable with doing.
And if you’re finding it veryhard to figure out how your data
product contributes financial valuebecause it’s based on this, like,
waterfalling of well, we own themodel, and it’s deployed on a platform.

(32:26):
The platform then powers theseother things, which then power
an application, and how do wefigure out the value of our tool?
Yeah, these things are challenging.
And if it’s challenging for you,guess how hard it will be for
stakeholders downstream, if youhaven’t had the practice and the
skills required to understand howto estimate value, both before we

(32:48):
build something as well as after?
And this gets into being able to definesuccess metrics, both progress metrics
and success metrics for your initiatives.
So, I spend a lot of time on this whenI’m doing coaching and training, both
individuals coaching as well as inthe seminar, is really thinking about

(33:10):
the metrics that we will use for this.
And I’m not talking aboutanalytics necessarily.
Most of the time, analytics, which arecomputer-based metrics, those are not
the way projects or increments of productwork can be measured a lot of the time.
Or they’re the wrong stats, like,things like how many people used the
data product, and we can track that.
Unfortunately, what you’re doingis tracking product metrics.

(33:32):
You’re tracking metrics about the toolthat you created, and this is used
as a proxy for actual value created.
And that’s not actually telling youanything about whether or not benefits
were created for somebody or not,let alone the financial value there.
I don’t want to get rat-holed too farinto this, but this value topic in

(33:55):
general, if you don’t want to spendyour time talking business language
and you want to be either in thedesign space, the model-building
space, the engineering space and allof that, then this role is not for you.
If the dollars and pounds, or whatevercurrency you use, if this is not
language that you want, you don’twant to spend your time getting to
know how your business makes money,frankly, then this work is not for you.

(34:20):
It’s just not.
I would stay doing what you’re doingalready or find a different thing
because a lot of your time is goingto be spent managing up for half the
time, and then managing the productstuff down, and sitting in this middle
layer, trying to explain to the businesswhat’s going to come out and what’s
the impact going to be, in languagethat they care about and understand.

(34:43):
You cannot be talking aboutmodels and model accuracy and
data pipelines and all this stuff.
They’re not going to careabout any of that; they’re just
going to see AI is really cool.
It’s magic sauce.
When are we going toget some of that value?
What’s going on with this productthat we’re supposed to be rolling
out, or this feature we’re rollingout, or whatever it may be?

(35:05):
That’s the operating model when you’remanaging up as a data or AI product

manager (35:10):
when are we going to see some value here, as we define what value is?
So, you need to have this skill sethere, and broadly speaking, across all of
these roles, thinking about transitioninginto data and AI product management.
The most common thing that I see,like when I do training and if I have

(35:31):
individual contributor type roles in mytraining, is that, frankly, they don’t
have any incentive to focus on this.
They were primarily hired to do technicaldelivery work most of the time, and
so the mindset and attitude is that,well, my job, that’s someone else’s
responsibility to make sure that we’reactually building the right thing.

(35:54):
And I think if you—there’s nothingwrong with that, if that’s the
kind of work you want to do.
And I think most people kind of inherentlyknow, do I want to take the responsibility
for value creation, or do I want someoneelse to do that, and I want to kind
of help them realize their vision,but ultimately, I’m following them?

(36:14):
To me, product managers need to have alittle of that CEO-ness inside of them.
They need to be leaders, and allyou need to be a leader is to have
followers, but you need to be aleader that people want to follow.
And you need to champion this morethan anybody on the team does.
And a lot of ICs simply don’t havethis skill because they haven’t

(36:36):
been required to focus on value.
They haven’t had anyincentive to focus on value.
Financial incentive, whether it’sbonus structures or things like this.
It’s kind of like you get paid the samewhether or not the solutions work or not.
So, if you’re going to move someone intoa data and AI PM role with an expectation
of value creation, but there’s noincentive set up for that, I think you

(37:00):
should be really thinking about whetheror not simply changing job titles and
descriptions is going to work and actuallycreate the value inside your organization.
So, they need to have the desire tostart thinking about benefits and
outcomes—which is, kind of, synonymsfor value here—they need to be
enabled to spend the time on that.

(37:20):
They’re going to need todevelop skills in this space.
But most ICs don’t have them, andmost of the time they don’t want
to have them, in my experience.
I tend to feel like I canassess this pretty fast.
And it’s really someone thatwants to move up a career ladder.
If someone really wants to move up thecareer ladder inside an organization,

(37:40):
you have to start thinking aboutvalue and less about outputs delivery.
And it’s very clear, oftentimes,who these people are.
And I think if someone’s hungry todo that work, eager to do that work,
that’s probably the most importantskill, assuming that they have a
modicum of intelligence, and you canassess that intelligence, that desire
to do this work and to step into thatspace, again, I like to believe that,

(38:05):
especially for adult learners, yourdesire to learn and willingness to
change how you do your work, that’sthe most important thing to have.
There are people that do not believe this.
I [laugh] thinking of one of our membersin the DPLC, who’s a chief data officer.
He has very different opinions about whocan step into this role, and he doesn’t

(38:26):
believe that all these job titles,people that are coming out of these
different tracks, technical, design, orexisting product management, that all
people are created equal in this space.
So, I just want to share thatnot everybody believes that
all these skills are equal.
And this is casting abroad generalization.
I’m not saying this person says, “Ifyou have this job title, you’ll never

(38:47):
be able to do this work.” That was not—Idon’t think that’s the message this
person was sharing in one of our DPLCmonthly webinars that we had in the
community, but it was more the evidencehas shown, in his experience, that
certain types of personalities, roles,skill sets, tend to be more oriented

(39:07):
for moving into this track than others.
So, for what it’s worth, just knowthere’s different opinions, and I’m
just sharing my opinion, which islargely based on just my exposure time
and also my vision that I—my agenda.
I have an agenda.
I am very much trying toinfluence how you do your work.

(39:27):
I am pushing you as a listener, andthese teams, to build stuff that
matters and to stop making more outputs.
We don’t need more technology crudout in the world—I’m not talking about
Create, Read, Update, Delete, for youengineers, when I said ‘crud’—I just
think we have so much skill here.
We have limited time on the planet.

(39:49):
Work is a big part of a lotof people’s lives, in terms
of minutes spent in your life.
Let’s make that life—let’s put outmeaningful stuff into the world.
So, let’s design for humanimpact benefits for end-users.
Let’s make better software,better empowerment.
Let’s leverage AI forthe benefits of people.

(40:10):
And also, we need to feedthe beast for tomorrow.
So, we need to create financialvalue for the company as well.
But I do have an agenda, soI just want you to know that.
Take that with a grain ofsalt, this whole episode.
But anyhow, so this whole value spacehere, you got to live in this space.
You got to want to live in this space.
And ICs—Individual Contributors; sorryif that term is not used in your country.

(40:34):
It means individual contributors,non-management function—if you’re not
thinking this way, and you don’t want tothink this way, you’re probably dead in
the water trying to get a role like thisbecause you really need to step into that.
And frankly, you might be cominginto a space where the hiring people
know that this is important, but theydon’t know exactly how a data product

(40:56):
manager and AI product manager attachesthe financial value to their work.
They’re kind of learning, too.
They’re kind of switching into thisproduct way of working—and now I’m talking
back again to the internal enterprise datateams here—they’re moving into the data
product space, and for them, that meanssomething to do with adopting software

(41:18):
product management life cycle, the wayof working, the way of design, tech, and
product coming together to deliver value.
They don’t know exactly what that lookslike, and they don’t know exactly how
we move into a more outcome orientation.
So, even the hiring team may notfully see it and know how to do it,

(41:39):
which might be why they’re hiringfor this role in the first place.
So, you really need to have a compellingvision and be able to express that
if you want to move into this role,particularly from the outside.
I think there’s less pressure to do thisfrom the inside, if you’re transitioning
into this role from an existing datarole, and they’ve kind of—hey, you know,

(42:00):
John or Jane, please step into this role.
You know, we’d like to makeyou a data product management.
Are you interested in this course?
We need someone—you seem to beable to talk to the business.
You seem to know how to connect with, youknow, our designers and talk to customers
and all of this, and so they feel likeyou’re this, like, natural fit there.
But they may not have some ofthese value-estimating skills

(42:22):
and figuring out how to attachfinancial value to the problem space.
So, develop that skillset.
Do some pro bono work, do some projectwork if you don’t have it, but figure
out how to attach the benefits andthe value to the work that you’ve
done, such that somebody can see it,and they can understand how your work

(42:43):
contributed some form of value to theorganization that you were helping.
They really need to be able to seethat, otherwise, you’re just going
to be showing them all the mechanicsof product management, which might
get you a job in some places.
To me, you can go through all the ritualsand all of that and not create any value,
and that’s not what product is about.

(43:04):
It’s really this outcomes versus outputsmentality and being able to show that.
And on that, you know, just a finalcomment on this outcomes-outputs thing.
I really want to just keeppushing this idea down because
I don’t see this all the time.
For teams that do understand the financialvalue they’re trying to attach to their

(43:24):
data product work, they’re still oftennot talking at all about user experience
outcomes, and they’re not connectingthe dots that you can keep talking about
how your model is going to drive thisbusiness lift, for example, or something
like that, but if there are humans inthe loop here that have not been part of
the solution that we’re building here,or we’re building a tool and hoping

(43:48):
that the staff is going to—you know,you’re building your AI conversational
agent, or whatever it may be, insidethe business, and you don’t know what an
improvement to the end-user’s life lookslike, then you’re only doing half the
work because that’s where your roadblockis going to be to get to the estimated
value that you’ve been talking about.

(44:10):
And so, broadly speaking, the useradoption piece, which is tied to
the user experience piece, this isoften still a big skill-gap area,
but you will hear people talk aboutthis in terms of low adoption.
This is what I would call the presentingproblem that I hear about in the
industry, but a lot of teams do notassociate the practice of product

(44:33):
design, as an antidote to that.
So, I think this is a skill that youreally need to have, and frankly,
I kind of think that the directionwith AI product management is going
to be hybrid PMs and designers.
There will be fewer and fewer specificdesign-only roles here as we build
more applications that have lessreliance on heavy user interfaces.

(44:57):
The really powerful players, thepowerful product managers out there are
going to be these hybrid people thatunderstand the tech, they understand
user experience, and they understandthe business side of PM, and they
can bring all these skills to bear.
So, I’m probably very biased because,you know, I’m a designer, and I come
out of that space, and so I tend tothink design skills can help almost

(45:20):
anybody in anything, and that’s probablya bias that maybe isn’t always true.
But I haven’t seen it yet where not havingthat skill doesn’t make a product manager
much better, whether it’s because theynow understand the skill gap they have
and they know that they need help, sothey’re problem aware, or they’re just
literally enabled themselves becausethey’ve had training and they know what

(45:44):
to look for when it comes to design anduser experience problems, they know how
to address those problems themselves,particularly in an AI context because
they’ve had training in it, they’vedeveloped skills in this space, and I
really think that’s, like, the future.
If you want to be the top 10% ofpeople in this role, it’s going to

(46:06):
be having all three of those skillsets: the business PM side, the enough
technical knowledge to prototyperapidly, to understand how to talk to
data, the data science and engineeringcounterparts, and to understand that
user experience and adoption reallymatters, particularly in a probabilistic
environment where we cannot controleverything that people are going to do

(46:27):
and what the models are going to generateover time within our applications.
So, the last thing here I’m just goingto say to, kind of, wrap up this episode,
is that delivery work is getting faster.
Coding is getting easier.
This is the highest use of generative AIright now is for software engineering,
so—and I’m not talking about softwareengineering today, but my point here

(46:49):
is that all these tools right now areenabling us to build stuff faster.
It doesn’t necessarily mean we’re makingbetter things, so if you can connect the
dot—if you can leverage all the speedgains we’re getting, and figure out that
probably what you need to be doing isprototyping sooner, have a hypothesis,

(47:12):
get a working solution out into a user’shands, and getting feedback quickly, and
then tracking whether or not your progressmetrics are being met, if you can do
that, you’re going to be in a great place.
Because as delivery work gets faster,the most important part of this role is
deciding what should we actually make.

(47:32):
Making it will be fast, but there’s athousand different things we could make.
What’s the right thing to make?
And that’s often rooted in your customerexposure time, your understanding of the
problem space, you’re understanding thechange in adoption, your understanding
of how easily can we attach any actual,quantifiable business value to this thing?

(47:54):
That’s where the time isnow going to be spent.
It’s not going to be spent onthese giant, long delivery cycles.
The delivery cycles will be much faster.
So, this means placing morebets, running more hypothesis
more quickly, and hopefullyreacting to that feedback sooner.
So, the other thing I’ll just sayhere is also in the agent space, is

(48:16):
really understanding and thinking aboutagents almost as—and this isn’t my
[idea] ; I saw someone talk about this.
I can’t remember who it was aboutthinking about agents, designing
for machines as almost anotherclass of end-users to making your
services available to machines.
I think this is another skill set that,like, data and AI PMs, need to have here.

(48:37):
So, not just our human end-users,but also, how do we make our solution
useful in an AI ecosystem whenwe’re in the middle of deployment?
Because oftentimes, we probably havea very practical reason we’re building
this product to serve a specific classof problems that we’re trying to address,
but I think thinking ahead also to howcould other machines and agents leverage

(49:01):
this work that we’re doing now, this isanother skill—particularly for the AI
product manager, obviously—that I thinkis going to be important going forward.
So, that’s it.
That’s all I got for you today.
I hope this was helpful to you.
If you are looking fortraining and/or coaching help.
I am considering relaunching agroup coaching offering here.

(49:22):
So, if you hop over to mylist, you can get on that from
designingforanalytics.com/podcast.
You should get a welcome email when youget that, when you join the list there,
and you could just reply to that and tellme if you’re interested, and I can give
you some more information about that.
So, for now, that’s all.
Thank you for listening,and we’ll see you soon.

All Episodes

171 - Who Can Succeed in a Data or AI Product Management Role?

Episode Transcript

Popular Podcasts

Stuff You Should Know

24/7 News: The Latest

The Joe Rogan Experience

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}171 - Who Can Succeed in a Data or AI Product Management Role?