November 18, 2025 • 42 mins
Noa Ginsberg, SVP, Product Management Excellence at FactSet, faced a challenge that would make most product leaders nervous: transform how a 40-year-old financial data company builds products. Not a single team or feature, but the entire product organization.
In this episode of Hard Calls, Noa and host Trisha Price dig into what it truly takes to introduce modern product thinking to build a Product Management Center of Excellence. They discuss the best practices for rolling out new tooling and analytics-driven processes across thousands of employees without losing touch with the customer and what made FactSet successful in the first place.
"If we hadn’t positioned for product excellence, we wouldn’t be ready for the AI wave.” - Noa Ginsberg
Here's what you'll discover:
How to build a Product Center of Excellence that scales. Noa breaks down the framework she used to reduce duplication, create clarity, and align product teams around shared outcomes—without adding bureaucracy.
Why gut instinct still matters in a data-driven world. Experience teaches you to recognize patterns faster than dashboards can. Noa explains how to trust your intuition while building the discipline to validate it with data.
The metrics that actually drive outcomes. Noa describes that aligning product metrics to business outcomes is not as easy as it sounds. Using the FACT Model, Noa shares how her team moved beyond vanity metrics to metrics that connect product decisions to business results.
How AI is changing the product development lifecycle. From ideation to prototyping, AI is compressing timelines and allowing anyone to bring an idea to life. Noa reveals where her team is seeing the biggest impact and where human judgment still wins.
Episode Chapters
- (00:00) Introduction and Noa Ginsberg's Role at FactSet
- (03:00) The Hard Call of Transforming a 40-Year-Old Company
- (06:39) Gut Feel as Data: Trust Your Instinct But Verify
- (12:07) When to Build a Product Center of Excellence
- (16:34) Stakeholders, Supportability, and Voice of Customer
- (20:57) How to Get Started Building Your Own COE
- (25:07) Aligning Product Metrics to Business Outcomes
- (29:00) Keeping Teams Customer-Connected in the Messy Middle
- (32:09) How AI and Agents Are Accelerating Product Development
- (39:13) Closing Thoughts on Decision-Making and Product Excellence
Love the episode?
Drop us a ⭐⭐⭐⭐⭐ review and share it with a teammate navigating their own hard call. Every subscription helps more product leaders find Hard Calls.
Presented by Pendo. Explore more insights at pendo.io or connect with Trisha Price on LinkedIn.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Trisha Price (00:00):
How do you do that?
How do you help your product teamsas they move through that maturity
matrix to make sure that what they aremeasuring aligns to sort of the bigger
OKR business outcomes of the company?
Noa Ginsberg (00:13):
Yeah, this is such a tough one because at the end
of the day, it's top line andmargin and EPS growth that matter.
I would say that our headline metricsapproach has been a, our best stab
at that, which is providing thisportfolio of metrics that you could
view for any product to see howit's impacting on the user side.
(00:34):
So it doesn't really directlyimpact but it helps you to gauge
the quality of your revenue.
And I think that's a big piece.
Trisha Price (00:43):
If you build software or lead people who do,
then you're in the right place.
This.
Is hard calls, real decisions,real leaders, real outcomes.
Hi everyone.
I am Trisha Price, and welcome back toHard Calls, the podcast where we bring on
the best product leaders from across theglobe to talk about those moments, the
(01:05):
decisions that matter, the hard calls.
On today's show, we have Noa Ginsberg,who leads the Product Management
Center of Excellence at FactSet.
Noa and I have known each other for aboutfour years now, since I joined Pendo.
When I met Noa.
She was starting up a productanalytics and product center of
excellence function for FactSet.
(01:26):
I really, right from the start, admiredNoa's approach to the role, her commitment
to excellence, and her ability tobuild strong teams and relationships.
In addition to Noa sharing a hard callthat's made a big impact on her career,
we're also gonna dive into what doesit take to create a product center of
excellence and the benefits that theteam has on being a product-led company.
(01:50):
Noa's gonna share her insights on whyproduct operations is the glue that binds
all of the pieces of the org together.
Her learnings on how to keep productteams close to the customer and how
AI is changing the product landscape.
This is a truly knowledge richconversation, and I can't wait for
you to hear it and to jump into it.
Welcome to the show, Noa.
(02:11):
Thank you so much, Trisha.
It's great to be here.
So, Noa, you know that theshow is called Hard Calls, so
looking back over your career.
Or even just in the past year, can youshare a hard call that you've had to make?
What made it so challenging?
What considerations or process didyou follow to make the decision
and what'd you learn from it?
Noa Ginsberg (02:33):
I'll start off by saying, first of all, I'm thrilled to be here.
I saw the list of other guestsand I was super flattered.
So really appreciate the opportunityand I love chatting with you and talking
shop and the opportunity to do that.
It's awesome.
So thanks for having me.
Hard calls are pretty muchwhat we do in product.
I think there are very few easy calls,especially as you know, mature in your
(02:55):
career, you start seeing only hard calls.
and I've noticed that for me, thehardest calls tend to be at the
intersection of people and priorities.
And each of those are hard.
And those are kind ofwhat we do in product.
Again, people, the people on ourteam, the people we hire, the people
we work with as our stakeholders,our clients, importantly.
and then the priorities we set as aproduct team are pretty much our bread
(03:17):
and butter, and that's how we master allour resources together and deliver value
for our clients to, to be successful.
and.
The intersection of those are whenI think it's the hardest call.
So I can think back to a time aboutfour years ago when we really wanted
to double down on our ability to runa real true product organization, as
(03:38):
you might see at any tech company,and we've been around a long time.
So we've been around since the lateseventies before the internet, before ai.
so we've had a lot of different ways ofdoing things, but we had the opportunity
to really double down on product thinking,a product mindset, the product toolkit
that you need, the skillset that you need.
(03:59):
but that was a major transformationand a huge undertaking, and we did
it so that we could deliver on ourstrategy, and we did it so that we
could deliver for our customers.
and make them more successful,which is what we're here to do.
but that was a tough thing to go throughbecause it was a real shift for a lot
of people in their, their daily roles.
(04:20):
mostly very positively received,and we've come a long way.
But we had Pendo was a part of thatas we stood up product analytics, to
give a, a single platform for peopleto be able to monitor things rather
than having to stand up their own.
It was, it was, it was a hard call tomove forward with that versus being
(04:40):
successful as we are for so long.
Making that shift, getting everyone onboard, getting the tools and processes
in place to support that from a productexcellence perspective is definitely.
A very hard call, one that I think weare now seeing the fruit of because
we have positioned ourselves so wellto take advantage of this new wave of
(05:01):
technology with AI in the last few years.
if we hadn't had the foresight toreally position ourselves for that,
I think we wouldn't be where we are.
And we are pretty far along andahead of the curve, especially in,
in a space that can tend to lag thatadoption of, of these types of things.
So it was.
Risk.
But, it's always goodto have a balanced risk.
(05:23):
So I think that's a keypart of hard, a hard call.
And we use Pendo, we use a lot of othertools to help us make like de-risked
decisions, but there's always some elementof risk in any call, or we wouldn't be
making it, it wouldn't be a hard call.
So, yeah,
Trisha Price (05:39):
I I love that you brought that up because hard calls
are a little bit of our experience.
And our gut feel combined with dataand tools and bringing our stakeholders
together, all of those have to cometogether, to make those hard calls.
But I always like to tell my teamtoo, no decision is also a decision.
(06:02):
And so like, while a hard call might bedifficult, not making a decision is just
as impactful and a lot of times worse.
Noa Ginsberg (06:12):
Yes, absolutely.
We try to bring more speedto decision making here.
But yeah, you have to pull it forward.
You have to make the hard call,you have to make it clear to
people too, what the call is.
And I think recognizing where you are,whether it's a people call or if it's a
priorities call, I think knowing whereyou are is always important in order to.
(06:33):
Get where you wanna go, you haveto know where you're starting.
And that's something thatwe've, we think about a lot.
We have a lot of good, tooling in place.
And you mentioned instinct versus data.
I really think our gut feel as yougain experience is a bunch of data
that's just sort of in your own LLM,in your mind of what you should do,
(06:56):
what you've seen this before, youstart to have some pattern recognition.
Always relying on gut checking thatnot only with the data, but with
other experts, other stakeholderswith our customers within our own
teams, always validating that.
So like, trust your gut, but verifyI think is very important and using
all those tools at your disposal.
(07:16):
So you do make the best decision.
Trisha Price (07:19):
That is great advice.
I wanna dive into some of the thingsyou mentioned in your hard call a bit
in the Product Center of Excellence.
But before we do, Noa, maybe you couldjust tell us a little bit about FactSet
and what you do, and particularly yourrole and your team before we jump.
Noa Ginsberg (07:37):
Yeah, definitely.
So FactSet is a financialdata and analytics provider.
So we have workstation tools thatyou might use, to monitor the
markets, to make trades, to researchcompanies, to catch up on earnings.
And it's used by over 8,000 of thelargest financial institutions over
200,000 users, and with 95% retention.
(07:58):
So we've been around for 46 years.
I've been public for 30.
I actually used FactSet inmy first job out of school.
I was a user of FactSet for acouple years when I started out in
investment banking, and I'd always beeninterested in finance and technology,
so I went to a bank and worked inthe technology group at the bank.
Used FactSet, loved the tool,loved the people, loved the
(08:21):
client focus that they had.
The real obsession with the userwas clear to me as a client.
So I came to work for FactSet andI've been here for 23 years now in
a bunch of different roles, but Istarted out supporting our clients
as a consultant on the help desk.
We would answer the phones herein New York, from the bankers
calling at 2:00 AM who needed help.
(08:42):
So you gain a lot of empathy forthe role of our clients, and that's
inherent in everyone who works here.
A lot of people come up through,right outta college, answering
phones and things like that.
So, FactSet's a pretty indispensabletool to the investing community.
So anyone you know who works at a wealthadvisor or a hedge fund or an asset
(09:02):
manager or an investment bank definitelyuses FactSet or has in the past.
So it's really part of the fabric of.
Of how the investment communityoperates and we are known for our
service and our client obsession.
Really, that's a big differentiatorfor us and something that I think
is only continuing as we're seeingthe pace of change increase so much.
(09:24):
So it wasn't surprising to me whenwe made this shift to have a product
management excellence group because itreally does help enable the whole company
to have that client obsession, which wealready inherently have as fact setters,
but it really helps to bring scale andconsistency to that so that everybody
can do things and maybe there are evenfewer hard calls for others to make.
(09:47):
If we in the center of excellenceare kind of providing what's,
what tooling should we use?
How do I do user research?
What design systems should I use?
Trying to solve a lot of thoseissues at scale, for people so they
don't have to make those hard calls.
We've already made them.
That's, I think, a hugebenefit to the team as well.
Trisha Price (10:04):
Wow.
You know, first of all, incredible,longstanding successful company with
a incredible reputation and what.
An interesting career for you tocome, have joined them so long ago and
done so many different roles there.
And now to be leading this productcenter of excellence, which I know
(10:26):
the entire product team at Factsetreally leans on you for best
practices, for insights to really makesure that you can execute on that.
Client obsession or as we callit at Pendo, our core values,
maniacal focus on the customer.
So we have that in common.
And I think that that's I think a bigpart of not just our love of product,
(10:49):
which I know we'll talk about today,but our obsession with customers has
definitely brought you and I togetherover the four years as we've talked.
So, really excited to dive in.
You know, I get the question a lot, Noa.
When I'm out talking to customers or justproduct managers and product teams and
product leaders is product operations,center of excellence for product overall.
(11:15):
How do I do it?
What does it look like?
So I'm really excited to spend this timetoday with you who is truly a leader in
setting up a center of excellence andhas done just an incredible job with
it for, for people to learn from you.
So maybe like, tell me, do you think,because I get asked this a lot, do you
think every company should have a productmanagement center of excellence or if not,
(11:40):
is there a time or a level of maturitywhen you think it's time to establish one?
Noa Ginsberg (11:47):
Yeah, great question.
I wish I had a book to read, to tellme about every specific situation.
There are a few out there on thistopic and I've, but I've found that
we, nothing really it's great to takethose as inputs, but you kind of have
to do your own instinct, combinationwith data to decide what works for you.
So I think certainly a large companyFactSet has 13,000 employees globally.
(12:11):
It's a big company, hard tokeep everything coordinated.
So sometimes that can be sort of a warningor like a, maybe a light going off on the
dashboard to say, "Hey, maybe you shouldthink about this?" So there are a few
things that I think you could think about.
One is probably coordination.
So if you're starting to feellike things might not be.
As smooth as they could be.
(12:33):
Or you're seeing a lot of Conway's law,maybe if you're big enough where you're
seeing multiple kind of conveyor beltscoming outta the company and to the
consumer or the end user, it's like,looks like a bunch of different companies.
That's obviously a red flag, thatwould say like, Hey, maybe we should
benefit some from some less duplicationof effort, from more coordination.
That's one thing that I think any kindof centralized team can help with.
(12:55):
And another one is relatedto that probably stakeholder.
Confusion.
So does product marketing know how towork with each team in the same way?
Is the process in one team totallydifferent and then when another central
team tries to work with that productteam, it takes too long because it's
learning a new way of doing things.
same thing with support or sales.
(13:16):
Like, do we have a well-oiled machine?
Does everybody know what to do?
You don't always needpeople to set that up.
You can just be clear of how things shouldgo with existing personnel and staff.
But beyond a certain scale, I think it'svery helpful to have a central team like
this that operates almost like a platformteam would, like a platform product
team, but for the product of buildingproducts, which is a little bit meta,
(13:39):
but we do think about it as a product.
What we do, what we put out fromour team is a product and what.
The people using it internallyat the company are our users,
and we are very careful to alwaysstay connected to our users.
It's very important for what we're doing.
And then I think the last thing I wouldsay is if you find that the language
(13:59):
you're speaking is not the same asanother product team, there might be
some differences that are necessarydepending on the kind of product you have.
But if you're all talking aboutadoption and you have a different
denominator for adoption.
It's very hard to compare the successof one product versus another.
And when you have a large productportfolio, it's very important to
(14:20):
be able to have an apples to applescomparison on the success of that product.
So those are kind of three areas, Iguess I would say where you might wanna
think about it if you don't already.
And I would say even if you can see alittle bit ahead and anticipate those
things, you'd be even better off.
Because once you have a certainscale, I think it really helps to
have some central coordination.
Trisha Price (14:41):
I am so glad you brought up the stakeholders piece.
I am such a firm believer, and I knowyou are too, that the only reason we
do product investments or technologyinvestments, depending on the kind
of kind of company you are, is todrive business outcomes, right?
It's either to drive revenue, tocut costs, to reduce risk, and the.
(15:03):
Leaders of our companies, teams,investors, they expect a return for
this investment that we're making.
And you know, I think it's easy in productto get too focused on the feature you're
building or the product you're shippingand not the outcome you're driving.
(15:23):
And without that really strong stakeholdercommunication and coordination, you
won't get those business outcomesthat you need from your product.
So it's great that you talked about howyou operate and the processes with your
stakeholders being such an importantpart of A COE and when you need a COE
(15:44):
is to standardize that because withoutthose relationships and without the
processes to partner, if sales isn'tselling it and support doesn't know how
to support the new features and productsthat you're putting out, and you don't
have a convenient way to work with thosepeople 'cause they have day jobs too.
It's kind of like, why are youbuilding what you're building?
Noa Ginsberg (16:05):
Totally.
We've tried to reallyprovide data around that too.
So we, on our agile teams, we havestakeholder assessments that some of
the teams can do to see assess how wellthe team is meeting the stakeholder's
expectations on things like communicationsor value delivery, or timing.
So those things are allreally important to monitor.
(16:26):
And then we also have a portfolioof headline metrics that is
visible to everyone at the company,across the product portfolio.
And those are the headlines thatwe want people to be talking about.
And that can be used tocompare across products.
So a key one for supportwould be supportability,
some of these sort of hidden.
(16:48):
Costs, trying to bring them tolight so people really understand
the full implication of a product.
It's not just who paid for it andhow much they paid, but how much
it costs to support the product.
And that's something we'vetried to bring forward.
So stakeholders have morerealization of that too.
Trisha Price (17:05):
Yeah.
Noa Ginsberg (17:05):
And even within the product teams to be able to have that visibility.
Trisha Price (17:09):
Yeah.
It's good that you brought upsupportability, because we often, as
product managers get pretty good atVoice of Customer and whether that
comes through our sales teams or throughcustomers directly or customer success
(17:31):
teams really understanding what it isthat our customers are looking for, but
indirectly it does impact our customers.
And if we're gonna be customerobsessed, if a product's difficult
to support and support's not educatedon it, not only does that drive
costs up of your overall product, butit does impact customer happiness.
(17:52):
And you know, it may be okay for aperiod of time where your support team
can cover up for you, but in the end,if your product's not easy to support
or there's a lot of support ticketscoming out of a particular part of your
product, that's equally important tofix than like direct voice of customer.
Noa Ginsberg (18:09):
We try to monitor, some buddy metrics along with supportability,
so we're not just looking at doessupport go down And then because people
don't like it and stopped using it,that's one reason it could go down.
So we always pair it up with NPS, whichwe do get from Pendo, and we track
NPS versus year over year change insupportability for cases per active user.
(18:34):
And we have tracked that for yearsnow since we've had Pendo and NPS in
place, and it's been really helpfulto see that validation that we are
improving and we're not not at theexpense of our customer satisfaction.
So that's something that's reallyimportant to us in terms of
(18:54):
supportability and voice of the customer.
And we've also gotten so muchmore data in the last four years
since standing up this team.
We really didn't have a centralized place.
We have a lot, again, a lot ofconnectivity with our customers,
super focused on them and really atthe core of everything we do, but a
systematic way of sharing that, makingsure it's available to everyone.
(19:16):
You know, it didn't exist.
You had to know someone or knowwho had it or find it somewhere.
but we've.
We've kind of brought that togetherinto an inventory so we know what we're
getting, what streams of VOC data we have,who owns it, what the follow up is that's
expected and that's a work in progress.
Like we still wanna expand that.
We wanna do more there, but we'vecome a really long way, and I think it
(19:39):
shows in how we develop our productsand ultimately what we deliver.
Trisha Price (19:44):
So we kind of dove in a little bit into some of the things
your teams, your team does and focuseson around analytics and voice of
customer and stakeholder management.
But for those just getting startedwith a product center of excellence
and maybe just establishing them.
(20:07):
How do you get started?
What are the things to consider?
Like do you jump right to some ofthe mature processes you're talking
about with NPS and data collectionand standard analytics and adoption?
Or are there more simpleplaces for people to start?
What are the considerationsfor folks to think about?
Noa Ginsberg (20:27):
Yeah, I think it's important to know what good looks like.
As I said, sort of where you wannago and where you are and try a,
try to chart a path to get there.
But obviously you can't start witheverything and like good product
people we know we have to prioritize.
So I would look at what is thebiggest pain point for your product
managers or for the business?
Like what visibility is missing?
(20:49):
In the KPIs we're using to trackour investments or to track how
the company's doing as a whole.
What are the blind spots?
Maybe you don't have them.
Maybe that's really well oiledmachine and you don't need to
focus on the metrics piece.
I think there are some companiesthat probably are there just.
Starting off.
But if, if you have that, great,you can move on to something else.
(21:11):
But I think, having a centralizedqualitative and quantitative
view is very helpful and thosethings inform each other.
So that's another one.
So if you don't have a user researchfunction that is central, I think
that's another place that could be akind of high leverage place to start.
As well as a system to store theuser research you're doing so you
(21:32):
can share that knowledge broadly.
that's really important becauseproduct people can conduct user
research and do it very well, butthey're not laser focused on that.
They don't know all of the methodologiesthat are available necessarily, right?
So having a centralized, really skilleduser research team can be very helpful.
And again, bring leverageto the product teams.
(21:54):
I think there's also some, somegoodness in centralizing how
we communicate with clients.
So in app, whether it's a homegrown thingor Pendo, we use a combination, how are we
communicating to clients with our voice?
And then how do we get informationfrom customers in the app or
in email or in other channels.
(22:16):
So those are some sortof good ideas to start.
I think even when you have started,we actually, we work on this
maturity assessment for ourselves.
We made an acronym out of itso that we could remember it.
'cause we like acronymsin finance and at FactSet.
So it's FACT is our own internal maturitymodel specifically for product analytics.
(22:42):
But it's foundational, aspirational,competitive, and then transformative.
And we use that to assess.
Ourselves, but also how well ourproduct teams are using and leveraging
the data that we are providing.
And that helps us see, okay,let's think about where we are.
We've come back to it when we'resetting goals for the year.
You know, what do we wanna doto level up the whole function?
(23:04):
And look what's, well, if we'rein column A, we wanna move to
column C, what do we need to do?
Or what are the things we couldtry or consider doing next
year to just move us forward?
So I think that's, whether you're at azero or you're on your way, I think that's
just a good outlook and way to frame it.
Trisha Price (23:22):
Yeah.
It forces us all to constantly getbetter and mature and see a path
to that no matter where we are.
Noa, when you're, when you're movingthrough your maturity matrix or
aligning on which metrics matterfor a particular product group.
(23:45):
How do you align what your productteams are measuring to what your
leadership is accountable too, right?
Because I see this is really hard.
I think in product, I think this is oneof the hardest things that's hard for me.
And I mean, we at Pendo are an analyticscompany and it's still really hard to go
(24:07):
from, "Hey, we're trying to drive revenue.
Hey, we're trying to improve cross sell.
Hey, we're trying to reduce supportcost," to this is the particular
part of the product I'm gonnafocus on and I'm gonna measure.
You know, time to value orretention, like how do you do that?
How do you help your product teamsas they move through that maturity
(24:29):
matrix to make sure that what they aremeasuring aligns to sort of the bigger
OKR business outcomes of the company?
Noa Ginsberg (24:37):
Yeah, this is such a tough one because at the end
of the day, it's top line andmargin and EPS growth that matter.
so I would say that ourheadline metrics approach has.
Been our best stab at that, which isproviding this portfolio of metrics that
you could view for any product to seehow it's impacting on the user side.
(24:58):
So it doesn't really directlyimpact, but it, it helps you to
gauge the quality of your revenue.
And I think that's a big piece.
So whereas a SV, what we callannual subscription value,
because we're a subscriptionbusiness, is a lagging indicator.
We want more leading indicators.
We wanna be able to assess thehealth of our customers better.
(25:20):
So the more data we have on how the systemis being used, what tools they're using,
what data they're using, the better wecan understand the quality, maybe better
address at risk, and build that successprofile based on the data we have.
So I think without a basic set of metrics,and we only have a handful, right?
It's five metrics working on six,but, Adoption sentiment, which is from
(25:46):
Pendo NPS, engagement supportability.
Those are the main areasthat we are providing for
everyone else to monitor that.
So I think then beyond that, beyond sortof equipping people with the basics, it's
also fostering some experimentation andbeing okay to experiment because, we
(26:09):
might say that we think this is a leadingindicator, but it may not turn out to be.
You can't argue thatadoption is a good thing.
So you want higher adoption.
Yes, you wanna build something.
It's like everyone's worst nightmare.
Every engineer's worst nightmare.
Every product person, you buildsomething and no one uses it.
No one uses it.
And it could be a marketingproblem, it could be a, a good
market problem or a sales problem.
Like it doesn't it radiatesout from the product though.
(26:31):
If it's not a good product, it doesn'tmatter if you have a world class.
Trisha Price (26:35):
I always say that, you know, if you have an adoption
problem, it's the product.
Now, those other groups could bemaking it better or can cover up
really well to make it less bad.
But at the end of the day, if we havean adoption problem, we have to look
ourselves in the eye as product teamand ask ourselves what's going wrong?
Noa Ginsberg (26:56):
Yes, a hundred percent.
And if, and so we wannaalways in increase adoption.
And we work very closelywith go to market.
Like we've actually, we as theCenter for Product Excellence, we
redefined our product lifecycle.
So that, again, everyone'sspeaking the same language.
Everyone knows what we mean when we saythe ideate or validate or build phases.
(27:19):
And we work with product marketing tobe very close throughout the lifecycle,
especially at the earliest stages.
Yeah.
So that they're involved.
And it isn't a handoff where it'slike finger pointing happening.
But we've got, we've gotten alot of runway out of that and
it's, it's really improved,especially over the last few years.
Trisha Price (27:42):
Well, Noa, your team is so important at setting these standards
of facts, your maturity matrix andhow to work with stakeholders and
really ensuring you get businessoutcomes from your investment.
But they're not the ones.
Accountable or owning the actualindividual products, right?
(28:04):
They're that messy, middle, supportingall of the other teams around them.
With that I know you've talkedabout being customer obsessed
is so important at FactSet.
How do you motivate your team, makethem feel connected to the customer
and accountable to outcomes when youare sort of the messy middle and one
(28:26):
step removed from the end product.
Noa Ginsberg (28:30):
Yeah.
Yeah.
I mean, we do work really directly andas one with these teams, especially
the pillars in my team are UX clienthelp and communication enterprise,
business agility, and product analytics.
So those teams are workingclosely with the product teams.
So they're part of the teamand they're invested and they.
Understand as well asthe others on the team.
(28:52):
What we're trying to drive andunderstanding the user is always
something we are seeking to do.
But we also have another set ofusers, which is our internal users.
So I've tried, and I think most of theteam has always seen it as what we do
as a product for internal consumers andwe are very closely connected to them.
(29:13):
We run cabs, we do annual surveys.
We check engagement with the toolsthat we build and the products,
quote unquote, that we release.
And some of those products are,are events, things like we run this
annual idea, hon, which is a prettyunique innovation tournament that
precedes our hackathon and it seedsthe hackathon with business validated
(29:34):
ideas coming from, it can come fromengineering, but it often comes from
outside of product engineering design.
It comes from sales, it comes fromfinance, it comes from hr. So it's
really the whole company putting forthinnovation ideas and then people can
take those forward to the hackathon.
And so we try to foster that innovationand that feeling of a product
(29:56):
mindset, even within our internalclients and our internal teams.
And I think that's really important.
But when you're settinggoals, it is hard to.
Especially OKRs, right?
They're supposed to be within yourline of influence and something
too far beyond it can be difficult.
So the we, we are challenged by that, butI think we try to focus more on the direct
(30:17):
impact we're having and then sort of thesecond order things we do follow also.
Trisha Price (30:22):
Yeah, that's great.
It is such a critical role, and it iscritical to keep people feeling connected
and tied to the business outcomes,and I know you do a great job of that.
A few minutes ago you mentioned yourproduct development lifecycle and
even changing it up and standardizingit to make sure that everyone,
even your stakeholders knew it.
You also just mentioned your ideaon, and your hackathon right now.
(30:47):
All of us, our product developmentlifecycle is in flux and the tools that
we're using and the pace at which we'reideating and validating is changing.
And even things likeideathons to hackathon.
It used to be probably two years agoor a year ago, if you did an ideathon,
(31:10):
they probably ended up in some sort ofslides or maybe a document outlining
the idea, and now I'm guessingyou have a full blown prototype
even without engineering involved.
So tell me about how you think aboutthat and how the roles are blurring
and just how are you respondingand changing in this sort of a
(31:33):
new era of product development.
Noa Ginsberg (31:36):
Yeah.
I mean, everyone's head is spinning,but in a really exciting way.
I mean, I first entered the workforcearound the.com time and that was also
super exciting, that feels the same to me.
We are really forward thinking onthis and from the very early stages
we had our own internal AI tooling.
(31:58):
So everyone at FactSet has beenable to play around and get good at
these technologies for years now.
Of course that has differentimplications, but we are a
pretty creative group of people.
We're like high agency.
It's a very entrepreneurial kindof environment for a large company.
And so we've seen a lotof that happening already.
We've got the builders comingin no matter what their role is.
(32:20):
Everyone's now a builder it sounds like.
So whether you're a productperson or a designer.
Or an engineer who didn't know frontend development, like now everyone can
do that, of course, not in production,but it really has facilitated the
speed because instead of talking aboutdoing something, we're just doing it.
(32:41):
So the early stages of the lifecycle,I think, are the most exciting and
ripe for disruption and accelerationby these tools, even though it sort
of started off in the development.
Side with the code completion or codeautomation and we're seeing great, great
results there, but it's more incremental.
(33:02):
I would say, when you think aboutthe build phase, like obviously
execution's very important and, butwe're really good at it already and
so it's hard to sort of 10 x that.
But I think when you think about allthe communication that happens, the
long documents that we would write like.
I just don't think that's the future.
And that cuts not hours off of developingsomething, but months off of deciding
(33:27):
whether something's a good idea or not.
Trisha Price (33:29):
Yeah.
Noa Ginsberg (33:29):
And that's super high leverage.
Trisha Price (33:31):
I totally agree with you.
It's the confidence factorwhen we actually put.
Engineering effort at something is wildlydifferent now than it was a year ago.
you know, it used to be, "Hey, we'vegot this idea. We've talked to a few
customers." We've done some discovery.
(33:53):
Let's get V zero into somesort of design partner's hands.
And then iterate, iterate, iterate,and kind of constantly examine
whether we're gonna kill something.
But now what we can, what our builders,as you said, can build even without
engineering support, can get us to apoint where we can iterate, iterate
(34:16):
so much faster that by the time we're.
Building for real at the confidence level,we've gone through so much iteration and
levels of discovery and it's so fast.
It's mind boggling.
Noa Ginsberg (34:28):
Yeah.
We're hearing teams cancelingmeetings because they don't need
to sync up on something like theprototype kind of helps them do that.
We're seeing people show a prototypeto a client in the morning and then
15 minutes later show it again, andso the number of iterations is just
skyrocketing because anyone can iterate.
You don't need to go have a meeting,you don't need to ask the designer
(34:49):
to redo the Figma prototype.
You know, there's some downside tothat and we we're gonna work through
a lot of the issues it raises oflike, who's, whose job is whose.
And you know, but I think we're anopen-minded place and so I think we'll
work through those things, but there arereal things to consider, like how do our
(35:10):
processes get impacted by these changes.
But overall, it's super excitingand most people are really excited
about it because again, we're aplace where people just wanna get
stuff done and want the best outcome.
So I think that's gonnahave a huge impact.
We're piloting a few differenttools here for product and design.
For the AI prototyping piece, we'reworking on an agent backlog for
(35:34):
accelerating the product lifecycle.
And obviously our developershave been at this for a while.
When tools like GitHub, copilotand others came out in cursor.
So there's a lot of explorationhappening and we don't want people to
have to go do it on the side of thedesk or you know, in their spare time.
Like, we wanna provide these tools,we wanna change the way we're working
(35:56):
and be at the cutting edge here.
Because, it's a huge opportunity.
Trisha Price (36:00):
So you talked about utilizing all kinds of different
tools that give you productivity.
AI really supercharging andgiving you productivity.
You just mentioned an agent.
Is that an agent?
You guys are actually thinking aboutbuilding that's particular and has context
about your processes and your products.
(36:23):
That helps your productmanagers be more efficient.
Noa Ginsberg (36:26):
Yeah, we're not sure yet what it looks like, but we've
put the call out to build up ourbacklog by the end of the year.
And then on our internal agent platform,which we're developing, we will be
building out those tools that aregonna help us to get that productivity.
So it could be something morerules-based, maybe it's not an agent.
There's a lot of opportunityto improve and get speed out
(36:48):
of a process that isn't AI.
So we're open to that too.
But there, the agentic piece is.
I think going to be very important.
Imagine you have an agent that updatesyour documentation for you automatically
based on the release notes or justconstantly trolls your documentation
to say, Hey, like these pages, and thenour writers can then document so much
more or they can work on other things.
(37:10):
It's a really exciting area.
Our quality would be better, because it'sjust hard for us to manage everything,
even though we have great people.
It's just a lot to cover.
Trisha Price (37:21):
No, I know we haven't talked about this before,
but imagine it'll be really cool.
And, and this is what we'replanning for at Pendo too.
If, when you're building that agent, thatagent would have access to your Pendo data
and you could ask it even this is youragent, not the one we're building, but one
you have configured and built on your own.
But through our MCP server, you'dbe able to ask adoption numbers,
(37:44):
et cetera, through your own agent.
And that's, I think, anexciting thing because I do
think everyone Yeah, of course.
All software companies, you're doing it.
We're doing it, are building theseagentic interfaces and agentic automation.
But we also know you're gonna haveyour own and your own's gonna have
special context about things thatis only relevant to FactSet and
(38:06):
FactSet group of product managers.
And we wanna make sure that kind of datais available to those agents as well.
Noa Ginsberg (38:13):
It's totally, exactly the same for us with our clients.
So we definitely would want to be involvedin that design partner program for MCP.
And it's the same way we're thinkingabout both internally for ourselves,
enabling that with the protocol andthe agent platform, and agent re
registry so that we know what's outthere and can permission appropriately.
(38:33):
There's so much to consider, but weare doing that for our clients as well.
So we're totally on the same page.
Although that page keepsmoving, keeps moving.
Trisha Price (38:41):
Keeps turning.
Well, I think there's no, nobetter time in my career of
building software than now.
And I think it allows us to focus onthe things that are important, like
business outcomes and customer happinessand customer outcomes, versus a lot of
(39:01):
the more mundane tasks that have to getdone 'cause they're a part of the job.
But allowing these tools toautomate more and more of those.
And I think that's just such anexciting time and I think it'll
help us get back to the part ofour job that is the most rewarding,
but also the most difficult, which.
Is really the hard call and youknow, how we opened it and how we can
(39:26):
finish today, which is all around.
What makes product hard is thateverybody thinks they know the right
answer and has an opinion, right?
One particular customer.
One particular sales person, but atthe end of the day, we are the ones
who have to make the hard calls in ourjobs and you know, is this interesting?
(39:50):
Sort of balance between gutinstincts that we talked about
earlier, which I agree with you.
I love your analogy that gut instincts.
Is really data.
It's just our own LLM, humanprocessing, all of that data and
pattern recognition to make decisions.
(40:11):
And you can, you can use otherpeople and you can use systems
and other pieces of data, but yourgut instincts is really data too.
And I love that.
and I love.
You know, learning from you todayabout your product center of excellence
and how that enables not just youto make decisions, but all of your
product managers to make more informedand faster and better decisions
(40:34):
and how you have been able to.
Continue to evolve with AI and new toolsthat are out there, but yet still remained
business outcome, analytics focused, andmost of all, customer success focused.
So, I just really appreciate you,Noa, been a long time partner for us
(40:55):
at Pendo, but also for me, and I alwayslearn something from you every time
we have a conversation and impressedwith you and what Factset has done.
So truly appreciate you coming on todayand sharing your knowledge and wisdom
with, our listeners and with me, andlook forward to future conversations.
Noa Ginsberg (41:13):
Likewise, thank you so much and I learned so much from you
as well and from the podcast, so it'sa great thing you've started here.
Trisha Price (41:21):
Thank you.
Appreciate it Noa.
Thank you for listening to HardCalls, the Product podcast, where
we share best practices and allthe things you need to succeed.
If you enjoyed the show today, sharewith your friends and come back for more.
How do you do that?
How do you help your product teamsas they move through that maturity
matrix to make sure that what they aremeasuring aligns to sort of the bigger
OKR business outcomes of the company?
Noa Ginsberg (00:13):
Yeah, this is such a tough one because at the end
of the day, it's top line andmargin and EPS growth that matter.
I would say that our headline metricsapproach has been a, our best stab
at that, which is providing thisportfolio of metrics that you could
view for any product to see howit's impacting on the user side.
(00:34):
So it doesn't really directlyimpact but it helps you to gauge
the quality of your revenue.
And I think that's a big piece.
Trisha Price (00:43):
If you build software or lead people who do,
then you're in the right place.
This.
Is hard calls, real decisions,real leaders, real outcomes.
Hi everyone.
I am Trisha Price, and welcome back toHard Calls, the podcast where we bring on
the best product leaders from across theglobe to talk about those moments, the
(01:05):
decisions that matter, the hard calls.
On today's show, we have Noa Ginsberg,who leads the Product Management
Center of Excellence at FactSet.
Noa and I have known each other for aboutfour years now, since I joined Pendo.
When I met Noa.
She was starting up a productanalytics and product center of
excellence function for FactSet.
(01:26):
I really, right from the start, admiredNoa's approach to the role, her commitment
to excellence, and her ability tobuild strong teams and relationships.
In addition to Noa sharing a hard callthat's made a big impact on her career,
we're also gonna dive into what doesit take to create a product center of
excellence and the benefits that theteam has on being a product-led company.
(01:50):
Noa's gonna share her insights on whyproduct operations is the glue that binds
all of the pieces of the org together.
Her learnings on how to keep productteams close to the customer and how
AI is changing the product landscape.
This is a truly knowledge richconversation, and I can't wait for
you to hear it and to jump into it.
Welcome to the show, Noa.
(02:11):
Thank you so much, Trisha.
It's great to be here.
So, Noa, you know that theshow is called Hard Calls, so
looking back over your career.
Or even just in the past year, can youshare a hard call that you've had to make?
What made it so challenging?
What considerations or process didyou follow to make the decision
and what'd you learn from it?
Noa Ginsberg (02:33):
I'll start off by saying, first of all, I'm thrilled to be here.
I saw the list of other guestsand I was super flattered.
So really appreciate the opportunityand I love chatting with you and talking
shop and the opportunity to do that.
It's awesome.
So thanks for having me.
Hard calls are pretty muchwhat we do in product.
I think there are very few easy calls,especially as you know, mature in your
(02:55):
career, you start seeing only hard calls.
and I've noticed that for me, thehardest calls tend to be at the
intersection of people and priorities.
And each of those are hard.
And those are kind ofwhat we do in product.
Again, people, the people on ourteam, the people we hire, the people
we work with as our stakeholders,our clients, importantly.
and then the priorities we set as aproduct team are pretty much our bread
(03:17):
and butter, and that's how we master allour resources together and deliver value
for our clients to, to be successful.
and.
The intersection of those are whenI think it's the hardest call.
So I can think back to a time aboutfour years ago when we really wanted
to double down on our ability to runa real true product organization, as
(03:38):
you might see at any tech company,and we've been around a long time.
So we've been around since the lateseventies before the internet, before ai.
so we've had a lot of different ways ofdoing things, but we had the opportunity
to really double down on product thinking,a product mindset, the product toolkit
that you need, the skillset that you need.
(03:59):
but that was a major transformationand a huge undertaking, and we did
it so that we could deliver on ourstrategy, and we did it so that we
could deliver for our customers.
and make them more successful,which is what we're here to do.
but that was a tough thing to go throughbecause it was a real shift for a lot
of people in their, their daily roles.
(04:20):
mostly very positively received,and we've come a long way.
But we had Pendo was a part of thatas we stood up product analytics, to
give a, a single platform for peopleto be able to monitor things rather
than having to stand up their own.
It was, it was, it was a hard call tomove forward with that versus being
(04:40):
successful as we are for so long.
Making that shift, getting everyone onboard, getting the tools and processes
in place to support that from a productexcellence perspective is definitely.
A very hard call, one that I think weare now seeing the fruit of because
we have positioned ourselves so wellto take advantage of this new wave of
(05:01):
technology with AI in the last few years.
if we hadn't had the foresight toreally position ourselves for that,
I think we wouldn't be where we are.
And we are pretty far along andahead of the curve, especially in,
in a space that can tend to lag thatadoption of, of these types of things.
So it was.
Risk.
But, it's always goodto have a balanced risk.
(05:23):
So I think that's a keypart of hard, a hard call.
And we use Pendo, we use a lot of othertools to help us make like de-risked
decisions, but there's always some elementof risk in any call, or we wouldn't be
making it, it wouldn't be a hard call.
So, yeah,
Trisha Price (05:39):
I I love that you brought that up because hard calls
are a little bit of our experience.
And our gut feel combined with dataand tools and bringing our stakeholders
together, all of those have to cometogether, to make those hard calls.
But I always like to tell my teamtoo, no decision is also a decision.
(06:02):
And so like, while a hard call might bedifficult, not making a decision is just
as impactful and a lot of times worse.
Noa Ginsberg (06:12):
Yes, absolutely.
We try to bring more speedto decision making here.
But yeah, you have to pull it forward.
You have to make the hard call,you have to make it clear to
people too, what the call is.
And I think recognizing where you are,whether it's a people call or if it's a
priorities call, I think knowing whereyou are is always important in order to.
(06:33):
Get where you wanna go, you haveto know where you're starting.
And that's something thatwe've, we think about a lot.
We have a lot of good, tooling in place.
And you mentioned instinct versus data.
I really think our gut feel as yougain experience is a bunch of data
that's just sort of in your own LLM,in your mind of what you should do,
(06:56):
what you've seen this before, youstart to have some pattern recognition.
Always relying on gut checking thatnot only with the data, but with
other experts, other stakeholderswith our customers within our own
teams, always validating that.
So like, trust your gut, but verifyI think is very important and using
all those tools at your disposal.
(07:16):
So you do make the best decision.
Trisha Price (07:19):
That is great advice.
I wanna dive into some of the thingsyou mentioned in your hard call a bit
in the Product Center of Excellence.
But before we do, Noa, maybe you couldjust tell us a little bit about FactSet
and what you do, and particularly yourrole and your team before we jump.
Noa Ginsberg (07:37):
Yeah, definitely.
So FactSet is a financialdata and analytics provider.
So we have workstation tools thatyou might use, to monitor the
markets, to make trades, to researchcompanies, to catch up on earnings.
And it's used by over 8,000 of thelargest financial institutions over
200,000 users, and with 95% retention.
(07:58):
So we've been around for 46 years.
I've been public for 30.
I actually used FactSet inmy first job out of school.
I was a user of FactSet for acouple years when I started out in
investment banking, and I'd always beeninterested in finance and technology,
so I went to a bank and worked inthe technology group at the bank.
Used FactSet, loved the tool,loved the people, loved the
(08:21):
client focus that they had.
The real obsession with the userwas clear to me as a client.
So I came to work for FactSet andI've been here for 23 years now in
a bunch of different roles, but Istarted out supporting our clients
as a consultant on the help desk.
We would answer the phones herein New York, from the bankers
calling at 2:00 AM who needed help.
(08:42):
So you gain a lot of empathy forthe role of our clients, and that's
inherent in everyone who works here.
A lot of people come up through,right outta college, answering
phones and things like that.
So, FactSet's a pretty indispensabletool to the investing community.
So anyone you know who works at a wealthadvisor or a hedge fund or an asset
(09:02):
manager or an investment bank definitelyuses FactSet or has in the past.
So it's really part of the fabric of.
Of how the investment communityoperates and we are known for our
service and our client obsession.
Really, that's a big differentiatorfor us and something that I think
is only continuing as we're seeingthe pace of change increase so much.
(09:24):
So it wasn't surprising to me whenwe made this shift to have a product
management excellence group because itreally does help enable the whole company
to have that client obsession, which wealready inherently have as fact setters,
but it really helps to bring scale andconsistency to that so that everybody
can do things and maybe there are evenfewer hard calls for others to make.
(09:47):
If we in the center of excellenceare kind of providing what's,
what tooling should we use?
How do I do user research?
What design systems should I use?
Trying to solve a lot of thoseissues at scale, for people so they
don't have to make those hard calls.
We've already made them.
That's, I think, a hugebenefit to the team as well.
Trisha Price (10:04):
Wow.
You know, first of all, incredible,longstanding successful company with
a incredible reputation and what.
An interesting career for you tocome, have joined them so long ago and
done so many different roles there.
And now to be leading this productcenter of excellence, which I know
(10:26):
the entire product team at Factsetreally leans on you for best
practices, for insights to really makesure that you can execute on that.
Client obsession or as we callit at Pendo, our core values,
maniacal focus on the customer.
So we have that in common.
And I think that that's I think a bigpart of not just our love of product,
(10:49):
which I know we'll talk about today,but our obsession with customers has
definitely brought you and I togetherover the four years as we've talked.
So, really excited to dive in.
You know, I get the question a lot, Noa.
When I'm out talking to customers or justproduct managers and product teams and
product leaders is product operations,center of excellence for product overall.
(11:15):
How do I do it?
What does it look like?
So I'm really excited to spend this timetoday with you who is truly a leader in
setting up a center of excellence andhas done just an incredible job with
it for, for people to learn from you.
So maybe like, tell me, do you think,because I get asked this a lot, do you
think every company should have a productmanagement center of excellence or if not,
(11:40):
is there a time or a level of maturitywhen you think it's time to establish one?
Noa Ginsberg (11:47):
Yeah, great question.
I wish I had a book to read, to tellme about every specific situation.
There are a few out there on thistopic and I've, but I've found that
we, nothing really it's great to takethose as inputs, but you kind of have
to do your own instinct, combinationwith data to decide what works for you.
So I think certainly a large companyFactSet has 13,000 employees globally.
(12:11):
It's a big company, hard tokeep everything coordinated.
So sometimes that can be sort of a warningor like a, maybe a light going off on the
dashboard to say, "Hey, maybe you shouldthink about this?" So there are a few
things that I think you could think about.
One is probably coordination.
So if you're starting to feellike things might not be.
As smooth as they could be.
(12:33):
Or you're seeing a lot of Conway's law,maybe if you're big enough where you're
seeing multiple kind of conveyor beltscoming outta the company and to the
consumer or the end user, it's like,looks like a bunch of different companies.
That's obviously a red flag, thatwould say like, Hey, maybe we should
benefit some from some less duplicationof effort, from more coordination.
That's one thing that I think any kindof centralized team can help with.
(12:55):
And another one is relatedto that probably stakeholder.
Confusion.
So does product marketing know how towork with each team in the same way?
Is the process in one team totallydifferent and then when another central
team tries to work with that productteam, it takes too long because it's
learning a new way of doing things.
same thing with support or sales.
(13:16):
Like, do we have a well-oiled machine?
Does everybody know what to do?
You don't always needpeople to set that up.
You can just be clear of how things shouldgo with existing personnel and staff.
But beyond a certain scale, I think it'svery helpful to have a central team like
this that operates almost like a platformteam would, like a platform product
team, but for the product of buildingproducts, which is a little bit meta,
(13:39):
but we do think about it as a product.
What we do, what we put out fromour team is a product and what.
The people using it internallyat the company are our users,
and we are very careful to alwaysstay connected to our users.
It's very important for what we're doing.
And then I think the last thing I wouldsay is if you find that the language
(13:59):
you're speaking is not the same asanother product team, there might be
some differences that are necessarydepending on the kind of product you have.
But if you're all talking aboutadoption and you have a different
denominator for adoption.
It's very hard to compare the successof one product versus another.
And when you have a large productportfolio, it's very important to
(14:20):
be able to have an apples to applescomparison on the success of that product.
So those are kind of three areas, Iguess I would say where you might wanna
think about it if you don't already.
And I would say even if you can see alittle bit ahead and anticipate those
things, you'd be even better off.
Because once you have a certainscale, I think it really helps to
have some central coordination.
Trisha Price (14:41):
I am so glad you brought up the stakeholders piece.
I am such a firm believer, and I knowyou are too, that the only reason we
do product investments or technologyinvestments, depending on the kind
of kind of company you are, is todrive business outcomes, right?
It's either to drive revenue, tocut costs, to reduce risk, and the.
(15:03):
Leaders of our companies, teams,investors, they expect a return for
this investment that we're making.
And you know, I think it's easy in productto get too focused on the feature you're
building or the product you're shippingand not the outcome you're driving.
(15:23):
And without that really strong stakeholdercommunication and coordination, you
won't get those business outcomesthat you need from your product.
So it's great that you talked about howyou operate and the processes with your
stakeholders being such an importantpart of A COE and when you need a COE
(15:44):
is to standardize that because withoutthose relationships and without the
processes to partner, if sales isn'tselling it and support doesn't know how
to support the new features and productsthat you're putting out, and you don't
have a convenient way to work with thosepeople 'cause they have day jobs too.
It's kind of like, why are youbuilding what you're building?
Noa Ginsberg (16:05):
Totally.
We've tried to reallyprovide data around that too.
So we, on our agile teams, we havestakeholder assessments that some of
the teams can do to see assess how wellthe team is meeting the stakeholder's
expectations on things like communicationsor value delivery, or timing.
So those things are allreally important to monitor.
(16:26):
And then we also have a portfolioof headline metrics that is
visible to everyone at the company,across the product portfolio.
And those are the headlines thatwe want people to be talking about.
And that can be used tocompare across products.
So a key one for supportwould be supportability,
some of these sort of hidden.
(16:48):
Costs, trying to bring them tolight so people really understand
the full implication of a product.
It's not just who paid for it andhow much they paid, but how much
it costs to support the product.
And that's something we'vetried to bring forward.
So stakeholders have morerealization of that too.
Trisha Price (17:05):
Yeah.
Noa Ginsberg (17:05):
And even within the product teams to be able to have that visibility.
Trisha Price (17:09):
Yeah.
It's good that you brought upsupportability, because we often, as
product managers get pretty good atVoice of Customer and whether that
comes through our sales teams or throughcustomers directly or customer success
(17:31):
teams really understanding what it isthat our customers are looking for, but
indirectly it does impact our customers.
And if we're gonna be customerobsessed, if a product's difficult
to support and support's not educatedon it, not only does that drive
costs up of your overall product, butit does impact customer happiness.
(17:52):
And you know, it may be okay for aperiod of time where your support team
can cover up for you, but in the end,if your product's not easy to support
or there's a lot of support ticketscoming out of a particular part of your
product, that's equally important tofix than like direct voice of customer.
Noa Ginsberg (18:09):
We try to monitor, some buddy metrics along with supportability,
so we're not just looking at doessupport go down And then because people
don't like it and stopped using it,that's one reason it could go down.
So we always pair it up with NPS, whichwe do get from Pendo, and we track
NPS versus year over year change insupportability for cases per active user.
(18:34):
And we have tracked that for yearsnow since we've had Pendo and NPS in
place, and it's been really helpfulto see that validation that we are
improving and we're not not at theexpense of our customer satisfaction.
So that's something that's reallyimportant to us in terms of
(18:54):
supportability and voice of the customer.
And we've also gotten so muchmore data in the last four years
since standing up this team.
We really didn't have a centralized place.
We have a lot, again, a lot ofconnectivity with our customers,
super focused on them and really atthe core of everything we do, but a
systematic way of sharing that, makingsure it's available to everyone.
(19:16):
You know, it didn't exist.
You had to know someone or knowwho had it or find it somewhere.
but we've.
We've kind of brought that togetherinto an inventory so we know what we're
getting, what streams of VOC data we have,who owns it, what the follow up is that's
expected and that's a work in progress.
Like we still wanna expand that.
We wanna do more there, but we'vecome a really long way, and I think it
(19:39):
shows in how we develop our productsand ultimately what we deliver.
Trisha Price (19:44):
So we kind of dove in a little bit into some of the things
your teams, your team does and focuseson around analytics and voice of
customer and stakeholder management.
But for those just getting startedwith a product center of excellence
and maybe just establishing them.
(20:07):
How do you get started?
What are the things to consider?
Like do you jump right to some ofthe mature processes you're talking
about with NPS and data collectionand standard analytics and adoption?
Or are there more simpleplaces for people to start?
What are the considerationsfor folks to think about?
Noa Ginsberg (20:27):
Yeah, I think it's important to know what good looks like.
As I said, sort of where you wannago and where you are and try a,
try to chart a path to get there.
But obviously you can't start witheverything and like good product
people we know we have to prioritize.
So I would look at what is thebiggest pain point for your product
managers or for the business?
Like what visibility is missing?
(20:49):
In the KPIs we're using to trackour investments or to track how
the company's doing as a whole.
What are the blind spots?
Maybe you don't have them.
Maybe that's really well oiledmachine and you don't need to
focus on the metrics piece.
I think there are some companiesthat probably are there just.
Starting off.
But if, if you have that, great,you can move on to something else.
(21:11):
But I think, having a centralizedqualitative and quantitative
view is very helpful and thosethings inform each other.
So that's another one.
So if you don't have a user researchfunction that is central, I think
that's another place that could be akind of high leverage place to start.
As well as a system to store theuser research you're doing so you
(21:32):
can share that knowledge broadly.
that's really important becauseproduct people can conduct user
research and do it very well, butthey're not laser focused on that.
They don't know all of the methodologiesthat are available necessarily, right?
So having a centralized, really skilleduser research team can be very helpful.
And again, bring leverageto the product teams.
(21:54):
I think there's also some, somegoodness in centralizing how
we communicate with clients.
So in app, whether it's a homegrown thingor Pendo, we use a combination, how are we
communicating to clients with our voice?
And then how do we get informationfrom customers in the app or
in email or in other channels.
(22:16):
So those are some sortof good ideas to start.
I think even when you have started,we actually, we work on this
maturity assessment for ourselves.
We made an acronym out of itso that we could remember it.
'cause we like acronymsin finance and at FactSet.
So it's FACT is our own internal maturitymodel specifically for product analytics.
(22:42):
But it's foundational, aspirational,competitive, and then transformative.
And we use that to assess.
Ourselves, but also how well ourproduct teams are using and leveraging
the data that we are providing.
And that helps us see, okay,let's think about where we are.
We've come back to it when we'resetting goals for the year.
You know, what do we wanna doto level up the whole function?
(23:04):
And look what's, well, if we'rein column A, we wanna move to
column C, what do we need to do?
Or what are the things we couldtry or consider doing next
year to just move us forward?
So I think that's, whether you're at azero or you're on your way, I think that's
just a good outlook and way to frame it.
Trisha Price (23:22):
Yeah.
It forces us all to constantly getbetter and mature and see a path
to that no matter where we are.
Noa, when you're, when you're movingthrough your maturity matrix or
aligning on which metrics matterfor a particular product group.
(23:45):
How do you align what your productteams are measuring to what your
leadership is accountable too, right?
Because I see this is really hard.
I think in product, I think this is oneof the hardest things that's hard for me.
And I mean, we at Pendo are an analyticscompany and it's still really hard to go
(24:07):
from, "Hey, we're trying to drive revenue.
Hey, we're trying to improve cross sell.
Hey, we're trying to reduce supportcost," to this is the particular
part of the product I'm gonnafocus on and I'm gonna measure.
You know, time to value orretention, like how do you do that?
How do you help your product teamsas they move through that maturity
(24:29):
matrix to make sure that what they aremeasuring aligns to sort of the bigger
OKR business outcomes of the company?
Noa Ginsberg (24:37):
Yeah, this is such a tough one because at the end
of the day, it's top line andmargin and EPS growth that matter.
so I would say that ourheadline metrics approach has.
Been our best stab at that, which isproviding this portfolio of metrics that
you could view for any product to seehow it's impacting on the user side.
(24:58):
So it doesn't really directlyimpact, but it, it helps you to
gauge the quality of your revenue.
And I think that's a big piece.
So whereas a SV, what we callannual subscription value,
because we're a subscriptionbusiness, is a lagging indicator.
We want more leading indicators.
We wanna be able to assess thehealth of our customers better.
(25:20):
So the more data we have on how the systemis being used, what tools they're using,
what data they're using, the better wecan understand the quality, maybe better
address at risk, and build that successprofile based on the data we have.
So I think without a basic set of metrics,and we only have a handful, right?
It's five metrics working on six,but, Adoption sentiment, which is from
(25:46):
Pendo NPS, engagement supportability.
Those are the main areasthat we are providing for
everyone else to monitor that.
So I think then beyond that, beyond sortof equipping people with the basics, it's
also fostering some experimentation andbeing okay to experiment because, we
(26:09):
might say that we think this is a leadingindicator, but it may not turn out to be.
You can't argue thatadoption is a good thing.
So you want higher adoption.
Yes, you wanna build something.
It's like everyone's worst nightmare.
Every engineer's worst nightmare.
Every product person, you buildsomething and no one uses it.
No one uses it.
And it could be a marketingproblem, it could be a, a good
market problem or a sales problem.
Like it doesn't it radiatesout from the product though.
(26:31):
If it's not a good product, it doesn'tmatter if you have a world class.
Trisha Price (26:35):
I always say that, you know, if you have an adoption
problem, it's the product.
Now, those other groups could bemaking it better or can cover up
really well to make it less bad.
But at the end of the day, if we havean adoption problem, we have to look
ourselves in the eye as product teamand ask ourselves what's going wrong?
Noa Ginsberg (26:56):
Yes, a hundred percent.
And if, and so we wannaalways in increase adoption.
And we work very closelywith go to market.
Like we've actually, we as theCenter for Product Excellence, we
redefined our product lifecycle.
So that, again, everyone'sspeaking the same language.
Everyone knows what we mean when we saythe ideate or validate or build phases.
(27:19):
And we work with product marketing tobe very close throughout the lifecycle,
especially at the earliest stages.
Yeah.
So that they're involved.
And it isn't a handoff where it'slike finger pointing happening.
But we've got, we've gotten alot of runway out of that and
it's, it's really improved,especially over the last few years.
Trisha Price (27:42):
Well, Noa, your team is so important at setting these standards
of facts, your maturity matrix andhow to work with stakeholders and
really ensuring you get businessoutcomes from your investment.
But they're not the ones.
Accountable or owning the actualindividual products, right?
(28:04):
They're that messy, middle, supportingall of the other teams around them.
With that I know you've talkedabout being customer obsessed
is so important at FactSet.
How do you motivate your team, makethem feel connected to the customer
and accountable to outcomes when youare sort of the messy middle and one
(28:26):
step removed from the end product.
Noa Ginsberg (28:30):
Yeah.
Yeah.
I mean, we do work really directly andas one with these teams, especially
the pillars in my team are UX clienthelp and communication enterprise,
business agility, and product analytics.
So those teams are workingclosely with the product teams.
So they're part of the teamand they're invested and they.
Understand as well asthe others on the team.
(28:52):
What we're trying to drive andunderstanding the user is always
something we are seeking to do.
But we also have another set ofusers, which is our internal users.
So I've tried, and I think most of theteam has always seen it as what we do
as a product for internal consumers andwe are very closely connected to them.
(29:13):
We run cabs, we do annual surveys.
We check engagement with the toolsthat we build and the products,
quote unquote, that we release.
And some of those products are,are events, things like we run this
annual idea, hon, which is a prettyunique innovation tournament that
precedes our hackathon and it seedsthe hackathon with business validated
(29:34):
ideas coming from, it can come fromengineering, but it often comes from
outside of product engineering design.
It comes from sales, it comes fromfinance, it comes from hr. So it's
really the whole company putting forthinnovation ideas and then people can
take those forward to the hackathon.
And so we try to foster that innovationand that feeling of a product
(29:56):
mindset, even within our internalclients and our internal teams.
And I think that's really important.
But when you're settinggoals, it is hard to.
Especially OKRs, right?
They're supposed to be within yourline of influence and something
too far beyond it can be difficult.
So the we, we are challenged by that, butI think we try to focus more on the direct
(30:17):
impact we're having and then sort of thesecond order things we do follow also.
Trisha Price (30:22):
Yeah, that's great.
It is such a critical role, and it iscritical to keep people feeling connected
and tied to the business outcomes,and I know you do a great job of that.
A few minutes ago you mentioned yourproduct development lifecycle and
even changing it up and standardizingit to make sure that everyone,
even your stakeholders knew it.
You also just mentioned your ideaon, and your hackathon right now.
(30:47):
All of us, our product developmentlifecycle is in flux and the tools that
we're using and the pace at which we'reideating and validating is changing.
And even things likeideathons to hackathon.
It used to be probably two years agoor a year ago, if you did an ideathon,
(31:10):
they probably ended up in some sort ofslides or maybe a document outlining
the idea, and now I'm guessingyou have a full blown prototype
even without engineering involved.
So tell me about how you think aboutthat and how the roles are blurring
and just how are you respondingand changing in this sort of a
(31:33):
new era of product development.
Noa Ginsberg (31:36):
Yeah.
I mean, everyone's head is spinning,but in a really exciting way.
I mean, I first entered the workforcearound the.com time and that was also
super exciting, that feels the same to me.
We are really forward thinking onthis and from the very early stages
we had our own internal AI tooling.
(31:58):
So everyone at FactSet has beenable to play around and get good at
these technologies for years now.
Of course that has differentimplications, but we are a
pretty creative group of people.
We're like high agency.
It's a very entrepreneurial kindof environment for a large company.
And so we've seen a lotof that happening already.
We've got the builders comingin no matter what their role is.
(32:20):
Everyone's now a builder it sounds like.
So whether you're a productperson or a designer.
Or an engineer who didn't know frontend development, like now everyone can
do that, of course, not in production,but it really has facilitated the
speed because instead of talking aboutdoing something, we're just doing it.
(32:41):
So the early stages of the lifecycle,I think, are the most exciting and
ripe for disruption and accelerationby these tools, even though it sort
of started off in the development.
Side with the code completion or codeautomation and we're seeing great, great
results there, but it's more incremental.
(33:02):
I would say, when you think aboutthe build phase, like obviously
execution's very important and, butwe're really good at it already and
so it's hard to sort of 10 x that.
But I think when you think about allthe communication that happens, the
long documents that we would write like.
I just don't think that's the future.
And that cuts not hours off of developingsomething, but months off of deciding
(33:27):
whether something's a good idea or not.
Trisha Price (33:29):
Yeah.
Noa Ginsberg (33:29):
And that's super high leverage.
Trisha Price (33:31):
I totally agree with you.
It's the confidence factorwhen we actually put.
Engineering effort at something is wildlydifferent now than it was a year ago.
you know, it used to be, "Hey, we'vegot this idea. We've talked to a few
customers." We've done some discovery.
(33:53):
Let's get V zero into somesort of design partner's hands.
And then iterate, iterate, iterate,and kind of constantly examine
whether we're gonna kill something.
But now what we can, what our builders,as you said, can build even without
engineering support, can get us to apoint where we can iterate, iterate
(34:16):
so much faster that by the time we're.
Building for real at the confidence level,we've gone through so much iteration and
levels of discovery and it's so fast.
It's mind boggling.
Noa Ginsberg (34:28):
Yeah.
We're hearing teams cancelingmeetings because they don't need
to sync up on something like theprototype kind of helps them do that.
We're seeing people show a prototypeto a client in the morning and then
15 minutes later show it again, andso the number of iterations is just
skyrocketing because anyone can iterate.
You don't need to go have a meeting,you don't need to ask the designer
(34:49):
to redo the Figma prototype.
You know, there's some downside tothat and we we're gonna work through
a lot of the issues it raises oflike, who's, whose job is whose.
And you know, but I think we're anopen-minded place and so I think we'll
work through those things, but there arereal things to consider, like how do our
(35:10):
processes get impacted by these changes.
But overall, it's super excitingand most people are really excited
about it because again, we're aplace where people just wanna get
stuff done and want the best outcome.
So I think that's gonnahave a huge impact.
We're piloting a few differenttools here for product and design.
For the AI prototyping piece, we'reworking on an agent backlog for
(35:34):
accelerating the product lifecycle.
And obviously our developershave been at this for a while.
When tools like GitHub, copilotand others came out in cursor.
So there's a lot of explorationhappening and we don't want people to
have to go do it on the side of thedesk or you know, in their spare time.
Like, we wanna provide these tools,we wanna change the way we're working
(35:56):
and be at the cutting edge here.
Because, it's a huge opportunity.
Trisha Price (36:00):
So you talked about utilizing all kinds of different
tools that give you productivity.
AI really supercharging andgiving you productivity.
You just mentioned an agent.
Is that an agent?
You guys are actually thinking aboutbuilding that's particular and has context
about your processes and your products.
(36:23):
That helps your productmanagers be more efficient.
Noa Ginsberg (36:26):
Yeah, we're not sure yet what it looks like, but we've
put the call out to build up ourbacklog by the end of the year.
And then on our internal agent platform,which we're developing, we will be
building out those tools that aregonna help us to get that productivity.
So it could be something morerules-based, maybe it's not an agent.
There's a lot of opportunityto improve and get speed out
(36:48):
of a process that isn't AI.
So we're open to that too.
But there, the agentic piece is.
I think going to be very important.
Imagine you have an agent that updatesyour documentation for you automatically
based on the release notes or justconstantly trolls your documentation
to say, Hey, like these pages, and thenour writers can then document so much
more or they can work on other things.
(37:10):
It's a really exciting area.
Our quality would be better, because it'sjust hard for us to manage everything,
even though we have great people.
It's just a lot to cover.
Trisha Price (37:21):
No, I know we haven't talked about this before,
but imagine it'll be really cool.
And, and this is what we'replanning for at Pendo too.
If, when you're building that agent, thatagent would have access to your Pendo data
and you could ask it even this is youragent, not the one we're building, but one
you have configured and built on your own.
But through our MCP server, you'dbe able to ask adoption numbers,
(37:44):
et cetera, through your own agent.
And that's, I think, anexciting thing because I do
think everyone Yeah, of course.
All software companies, you're doing it.
We're doing it, are building theseagentic interfaces and agentic automation.
But we also know you're gonna haveyour own and your own's gonna have
special context about things thatis only relevant to FactSet and
(38:06):
FactSet group of product managers.
And we wanna make sure that kind of datais available to those agents as well.
Noa Ginsberg (38:13):
It's totally, exactly the same for us with our clients.
So we definitely would want to be involvedin that design partner program for MCP.
And it's the same way we're thinkingabout both internally for ourselves,
enabling that with the protocol andthe agent platform, and agent re
registry so that we know what's outthere and can permission appropriately.
(38:33):
There's so much to consider, but weare doing that for our clients as well.
So we're totally on the same page.
Although that page keepsmoving, keeps moving.
Trisha Price (38:41):
Keeps turning.
Well, I think there's no, nobetter time in my career of
building software than now.
And I think it allows us to focus onthe things that are important, like
business outcomes and customer happinessand customer outcomes, versus a lot of
(39:01):
the more mundane tasks that have to getdone 'cause they're a part of the job.
But allowing these tools toautomate more and more of those.
And I think that's just such anexciting time and I think it'll
help us get back to the part ofour job that is the most rewarding,
but also the most difficult, which.
Is really the hard call and youknow, how we opened it and how we can
(39:26):
finish today, which is all around.
What makes product hard is thateverybody thinks they know the right
answer and has an opinion, right?
One particular customer.
One particular sales person, but atthe end of the day, we are the ones
who have to make the hard calls in ourjobs and you know, is this interesting?
(39:50):
Sort of balance between gutinstincts that we talked about
earlier, which I agree with you.
I love your analogy that gut instincts.
Is really data.
It's just our own LLM, humanprocessing, all of that data and
pattern recognition to make decisions.
(40:11):
And you can, you can use otherpeople and you can use systems
and other pieces of data, but yourgut instincts is really data too.
And I love that.
and I love.
You know, learning from you todayabout your product center of excellence
and how that enables not just youto make decisions, but all of your
product managers to make more informedand faster and better decisions
(40:34):
and how you have been able to.
Continue to evolve with AI and new toolsthat are out there, but yet still remained
business outcome, analytics focused, andmost of all, customer success focused.
So, I just really appreciate you,Noa, been a long time partner for us
(40:55):
at Pendo, but also for me, and I alwayslearn something from you every time
we have a conversation and impressedwith you and what Factset has done.
So truly appreciate you coming on todayand sharing your knowledge and wisdom
with, our listeners and with me, andlook forward to future conversations.
Noa Ginsberg (41:13):
Likewise, thank you so much and I learned so much from you
as well and from the podcast, so it'sa great thing you've started here.
Trisha Price (41:21):
Thank you.
Appreciate it Noa.
Thank you for listening to HardCalls, the Product podcast, where
we share best practices and allthe things you need to succeed.
If you enjoyed the show today, sharewith your friends and come back for more.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Are You A Charlotte?
In 1997, actress Kristin Davis’ life was forever changed when she took on the role of Charlotte York in Sex and the City. As we watched Carrie, Samantha, Miranda and Charlotte navigate relationships in NYC, the show helped push once unacceptable conversation topics out of the shadows and altered the narrative around women and sex. We all saw ourselves in them as they searched for fulfillment in life, sex and friendships. Now, Kristin Davis wants to connect with you, the fans, and share untold stories and all the behind the scenes. Together, with Kristin and special guests, what will begin with Sex and the City will evolve into talks about themes that are still so relevant today. "Are you a Charlotte?" is much more than just rewatching this beloved show, it brings the past and the present together as we talk with heart, humor and of course some optimism.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.