August 8, 2023 • 16 mins
Kaaren Hanson, PhD discusses her book chapter "Observed Behavior is the Gold Standard" and offers insight on effectively building UX teams.
Sponsored by Watch City ResearchWatch City Research is your trusted UX research partner
Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Dan Berlin (00:14):
Hi everyone, and welcome to another episode of
the 97 UX Things podcast. DanBerlin here, your host and book
editor. I'm joined this week byKaaren Hanson, who wrote the
chapter Observed Behavior is theGold Standard. Welcome, Kaaren.
Kaaren Hanson (00:27):
Thank you. I'm so happy to be here. Dan.
Dan Berlin (00:30):
Thanks for joining the podcast. Can you please tell
us a little bit about yourself.
Kaaren Hanson (00:34):
So I'm the Chief Design Officer for Consumer and
Community Banking, also known asChase, which is a part of
JPMorgan Chase. And in thatrole, I get to lead a wonderful
team of individuals who aretrying to make sure that
customers are able to gettremendous value in a very easy
to use manner. What I would sayis professionally I grew up at
(00:55):
Intuit, where I spent about 12years, and one of the things I'm
most proud of is really helpingchange that company to become
basically design driven ordesign lead. It took a lot of
time and a lot of people. Afterthat I spent some time at a
startup. And I went to Meta,where I learned a lot about how
the power of metrics in terms ofaligning teams and helping
(01:15):
people to truly achieve morethan perhaps they had expected.
And then I moved back into thepersonal finance space where I
went to Wells Fargo beforecoming here to Chase.
Dan Berlin (01:25):
Nice. Thanks for telling us your journey. Can you
tell us about the beginning ofthat journey? How did you
discover UX?
Kaaren Hanson (01:32):
Yeah, so, one of my first jobs, I was actually
working at a company that was anational jury project where we
really worked on certain sidesof cases. So I worked for
victims of sexual discriminationand victims of sexual
harassment, or racialharassment, or sexual
orientation, harassment, ordiscrimination or whatnot. And
when I was in that role, Ireally was doing some exhibit
(01:52):
design, I was reallyunderstanding people and how
they thought, and I realized thepower of helping people to
understand the bigger story andseeing the forest for the trees,
and then also what it took toreally drive that change. That
wasn't actually the rightindustry for me, I actually
found it both very stressful,but also very boring. And so at
(02:13):
the time, there were a lot ofstartups in San Francisco that
were hiring essentially anybodywho had some knowledge of design
and research. And so I hoppedonto one of those startups, and
I learned a ton from my peers.
And then from then, I just keptmoving to company to company
taking on different roles,learning more, and eventually
realized that one of the areas Iam particularly effective at is
(02:34):
really helping to championdesign within an organization.
And so that has been what I'vereally gravitated towards is how
do I create conditions in whichteams can thrive and do their
best work? Not just design, butthe whole community with whom
they work those engineers, datascientists and product managers?
Dan Berlin (02:54):
Yep.Tell me a little bit more about that. What's your
philosophy or strategy when youcome in to build a great culture
for a design team?
Kaaren Hanson (03:02):
Yeah, I think a lot about do we have the right
people in the right places? Somepeople perform really well in
one job, but not necessarily inanother? Right? So within a
company, there are certain rolesthat where you need to do
different types of work. So dowe have the right people in the
right places? Is the broaderteam that quad is what we refer
to at Chase? Is that quad setuparound the customer's purpose,
(03:25):
the customer problem they'retrying to solve? Is there a
clear metric for them to know ifthey've succeeded? Not from a
business perspective, but from acustomer's perspective that then
ladders to that businessoutcome. And are they aligned on
the quality of the experiencethey want to provide? And how
are they going to measure that?
And once you've got a teamthat's aligned on this is the
problem we're solving, here'show we're going to measure
(03:47):
success, and this is the qualityof the experience we expect to
create, you're in pretty darngood shape. But it's a matter of
getting there and then makingsure that the processes around
that company are reinforcingthat customer centricity, and
reinforcing empowering the teamsas opposed to getting in the
way. Once you start to buildthat team, when you start to
(04:08):
build the relationships, thealignment, then is about really
tackling the broader companyoperating mechanisms or
structures to ensure thatthey're reinforcing conditions
in which teams can thrive.
Dan Berlin (04:20):
That's wonderful. I love how you describe the quad
as being based from a customerperspective, because often
enough, in enterprise, we seeteams or however we structure
our teams, based on businessgoals and the business way of
looking at it. And it'sheartening to hear you describe
it as a customer focused way offorming your teams.
Kaaren Hanson (04:42):
And I do have a very deep belief that if you do
the right thing by the customer,in most cases, you will do the
right thing by the business. Andpart of our job as designers and
product managers and data andtechnologists is to really make
sure that we're solving thecustomer problem in a way that
works for the business, becausewe do operate in a business. And
(05:03):
there are many ways to solvethat customer problem. So our
job is solving well for thecustomer. And in a way that
works for the business.
Dan Berlin (05:11):
I love it. That's what we do in UX. It's that
trifecta of business goals, userneeds, and the technological
ability to support these two,the way you just described that
was wonderful, thank you forthat.
Kaaren Hanson (05:22):
And what I think is really important is sometimes
where I see people and UXgetting stuck is because they
think of it as customer orbusiness. It's not, it's never
customer or business. It'scustomer and business.
Dan Berlin (05:33):
100%. Thanks for all of that. Very interesting, it's
always nice to hear how folksare looking at forming their
teams from a high level. Let'sturn to your chapter, Observe
Behavior is the Gold Standard.
Tell us about that, please.
Kaaren Hanson (05:48):
Yeah. So, I think that it can be very seductive to
listen to what people say. Andwe're very used to having
conversations with people. Andwe tend to believe what they
tell us because most people arefairly honest. Now, the downside
is, we also tend to believe whatthey tell us about how they will
act in the future. And so it canbe very easy for us to take what
(06:09):
people say, whether verbally orin writing, and assume that's
true. And when we do that, weare often wrong. We're not
always wrong, but we're oftenwrong. And the problem is, it's
hard to predict when are yougoing to be wrong? And when are
you not going to be wrong whenyou simply listen to what people
say, right? And I'm going togive you a couple of examples
(06:29):
from Intuit. And then I alsohave a couple examples from
Chase. But in one case, we wereworking on a payroll product.
And there was a question of,hey, when people come and
they're just trying to pay a newindividual on their payroll, do
they want to pay a person or dothey want to actually set up the
payroll, and then pay a person.
And when we ask people, someenormous majority, like maybe
(06:51):
90% of people said, I would wantto set up my payroll right away.
Now, fortunately, the teamdidn't necessarily believe what
they said, because often what wesay and do are different. So
they set up a painted doors testwhere they had people on the
site, you could either go paysomeone right away, or you could
save your payroll, what we foundwas the majority of customers
(07:12):
went to pay an employee rightaway, because it was a time of
need. Because of that, the teamwas able to get the funding they
needed in order to build asolution that solved this
customer problem of needing topay my employee. And they were
able to get that funding becausethey had that behavioral
evidence. And they didn't justlisten to what people say. Very
similarly, when I was at intoIntuit, another product, we were
(07:35):
looking at the experience of it,and we were talking to our
customers, after they did sometasks about what their
experience was, they were sayingthings like, I love it, it's so
easy, it's amazing. And then welooked at what they had actually
done. And what we realized isthey had made so darn many
errors, there are going to be inhell with their accountant in
(07:56):
about, you know, maybe three tofour months, right. So again, if
we had simply listened to whatthey had said, we would have
thought this company we boughthad a great product. But because
we also observed what they didand had very clear metrics for
success, we knew that it wasn'tactually that easy for them to
be accurate in what they weretrying to do.
Dan Berlin (08:18):
Right. And accuracy in the finance world is
critical.
Kaaren Hanson (08:23):
Yeah.
Dan Berlin (08:23):
Tell me about the painted door. You mentioned
doing a painted door research tofigure out what they were
actually doing. Can you expandon that, please?
Kaaren Hanson (08:31):
Yeah, well, essentially, what was done at
the time is there was simply twooptions on this particular page
that you would land on, right.
And this was actually on the webis before the app was strong as
it is now. And you could eitherset up my payroll, or you could
pay an employee. And it turnsout, majority of people want to
pay an employee, because whathad happened is maybe they had
(08:51):
just hired somebody, maybe theyhad forgotten that they even
needed to set up a payroll maybewas their first employee, they
didn't know what was happening.
And all of a sudden, it wasFriday, and they had to pay
somebody, and they didn't havetime to find their EIN, and you
know, other documentation, theyneeded to figure out all of the
information that had to be hadin order set this up correctly,
(09:15):
they just needed to get Joe apaycheck, right. And so they did
that. And then later on, theywould come back and set it up.
So let me give you anotherexample of the difference
between what people say and whatthey do at Chase. So we recently
conducted some research onbehaviors and needs of
consumers. And we were reallylooking at what people said
initially, and we asked them torate the importance of like
(09:38):
personalized shopping versusother drivers of customer value
like price or return policy,etc, etc, etc. And so when we
had this conversation, it turnsout when people talk,
personalization is ranked verylow, likely because it's a
fairly abstract concept, andit's hard to imagine what
(09:59):
personalization might mean,right? Or everybody might be
imagining it means somethingdifferent in their head. And so
to get around that, then what wereally looked at was how to
create examples ofpersonalization live, that would
be meaningful to individuals.
And by doing that, we start tohave very different
conversation. Now, the reason Ibring this up as an example is
because it goes to show thatthere are limitations sometimes,
(10:21):
and how people understand wordsand understand questions, which
is another weakness of simplylistening to what people say, as
opposed to really going further,and helping them more deeply
understand whether by showingthem something different, or
having them go after doing atask themselves.
Dan Berlin (10:44):
Yep, getting more of that reaction than just their
their words.
Kaaren Hanson (10:48):
Yeah, then just like what they're imagining your
thinking and what they'reimagining their future self is
going to do, right, that's a lotof imagination, which may be
accurate, or may not beaccurate.
Dan Berlin (11:00):
Right. So how can we get at these behaviors? What are
some of the ways that you UXersshould be trying to get at these
Kaaren Hanson (11:08):
Yeah, I mean, I think instrumentation and
behaviors?
knowing what are the core flows,the core behaviors are looking
for, and doing theinstrumentation for something
that is live, I think a lot ofit is the anthropology and going
into people's houses andobserving what they do today.
Because what they do today isprobably what they're going to
do tomorrow. Again, if you goback to the Intuit days, and
(11:31):
TurboTax, we would go intopeople's houses, and we would
see all the different places,they kept their tax forms,
right, and you know, all thedifferent materials they had,
and then somebody on thefrigerator. And so would be
like, you know, practically inthe garbage can pull it all
together. And they wouldprocrastinate then and now that
everything's electronic, well,let's not assume that everybody
has their act together. It canbe just as complicated and messy
(11:54):
and as much procrastination hasever said the best day and
actually, there's a great quote,I can't remember who made it.
But the best predictor of futurebehavior is past behavior, even
if the modalities and thetechnology is different.
Dan Berlin (12:06):
Yeah. And what about doing it in an artificial
setting versus in someone'shome? Right, bringing someone to
someone into a lab or doing aremote usability study versus
going to their place? You know,what are some advantages or
disadvantages there?
Kaaren Hanson (12:23):
Yeah, I mean, I'm a fan of all of that, I think
about it as a portfolio view ofresearch methods, every research
method has a benefit and adrawback, just like your stock
has a benefit, your bond has abenefit, and they both have
drawbacks you want both in yourportfolio. And so what I would
say is true about the usabilitystudies, whether in person or
remote is you are able to lookat behavior, if you set it up to
(12:44):
look at behavior, as long asyou're making sure is, what they
gave you accurate was thepayroll they made did have the
right taxes taken out, so theywon't be caught in the end,
right? It's very easy to dothese studies in a more rigorous
manner, which is probably goingto be more valuable to you or a
less rigorous manner. Now, ifthe goal is simply to have
everybody have some experienceof seeing what the customer goes
(13:07):
through, as they're trying toattempt something, fine, don't
worry about being rigorous,because then the goal is really
to make sure everybody has acustomer voice in their head or
has seen the customer and isaligned around that problem, and
aligned around the quality ofthe experience. But if you want
to get deeper, be more rigorous,and be very clear on what
success looks like and does not.
Dan Berlin (13:29):
Tell me a little bit more about that please. How did
you define rigor in the wayyou're describing it here?
What's the difference betweennon rigorous and rigorous.
Kaaren Hanson (13:37):
So I'll give you an example of a consulting
company we worked with at onepoint, and they were they were
fine. We just coached them towhat we needed. But initially,
we were going to do somebenchmarking on a particular set
of products and theircompetitors. And this was at a
previous company. And soinitially, what they had set up
was, you know, having people dothings and then asking
(13:58):
questions, like were yousuccessful and how easy was it
and you have the usualquestions. They didn't actually
have any behavioral evidence.
And so it was really pushingthem back and saying, Wait, now
we're in this flow, are we goingto say that they're successful?
What page do they need to getto? What is the screenshot that
they have to have and send in orwhat is the screenshot we have
to see, for us to know that thiswas a win, what needs to be on
(14:19):
that screenshot? And so gettingreally rigorous about what it is
gave us very different dataresults. But that wasn't
necessarily how that particularset of practitioners was
thinking to begin with. Again,going back to what are we going
to do tomorrow is probably whatwe did yesterday. They were
coming at it with the sameprotocol they'd used in many
(14:41):
other places.
Dan Berlin (14:43):
Gotcha. So was there anything else you were hoping to
convey here today?
Kaaren Hanson (14:48):
I guess I would just say that there's nothing
wrong with talking to people. Idon't want anyone to get the
wrong idea, right. Talking topeople is incredibly useful and
can help you understand thecontext they're in and it can
help you understand whatmotivates them. And some
struggles they might have had orsome dreams and wishes they
have. But don't assume thattalking is enough.
Dan Berlin (15:10):
Well put, don't assume that talking is enough.
And this comes back to be mixedmethods and always triangulating
on your users using mixedmethods, whether it's a survey
and interviews or usability andinterviews or however you're
doing your triangulation. It'sso important to get that that
quant and qual almost.
Kaaren Hanson (15:30):
It is and it's also okay if your goal is simply
to make sure the quad like theproduct managers and the
technologists and the datascientists and the designers and
whatnot. If your goal is to makesure everybody understands who
this person is, talking may befine. But if you're trying to
make decisions around whatproducts to go after, is their
product market fit. You know, isthis a good enough experience
(15:52):
that they're able to besuccessful and confident?
Probably talking will not beenough in many cases.
Dan Berlin (16:00):
Well Kaaren, you gave us a lot of great
information here today. Thankyou so much for joining the
podcast.
Kaaren Hanson (16:05):
Oh, thank you so much. It was a pleasure.
Dan Berlin (16:07):
My guest today has been Kaaren Hanson, who wrote
the chapter Observed Behavior isthe Gold Standard. Thanks for
listening, everyone. You've beenlistening to the 97 UX Things
podcast, a companion to the book97 Things Every UX Practitioner
Should Know published byO'Reilly and available at your
local bookshop. All bookroyalties go to UX nonprofits as
(16:27):
well any funds raised by thispodcast. The theme music is
Moisturize the Situation byConsider the Source and I'm your
host and book editor Dan Berlin.
Please remember to find theneeds in your community and fill
them with your best work. Thanksfor listening
Hi everyone, and welcome to another episode of
the 97 UX Things podcast. DanBerlin here, your host and book
editor. I'm joined this week byKaaren Hanson, who wrote the
chapter Observed Behavior is theGold Standard. Welcome, Kaaren.
Kaaren Hanson (00:27):
Thank you. I'm so happy to be here. Dan.
Dan Berlin (00:30):
Thanks for joining the podcast. Can you please tell
us a little bit about yourself.
Kaaren Hanson (00:34):
So I'm the Chief Design Officer for Consumer and
Community Banking, also known asChase, which is a part of
JPMorgan Chase. And in thatrole, I get to lead a wonderful
team of individuals who aretrying to make sure that
customers are able to gettremendous value in a very easy
to use manner. What I would sayis professionally I grew up at
(00:55):
Intuit, where I spent about 12years, and one of the things I'm
most proud of is really helpingchange that company to become
basically design driven ordesign lead. It took a lot of
time and a lot of people. Afterthat I spent some time at a
startup. And I went to Meta,where I learned a lot about how
the power of metrics in terms ofaligning teams and helping
(01:15):
people to truly achieve morethan perhaps they had expected.
And then I moved back into thepersonal finance space where I
went to Wells Fargo beforecoming here to Chase.
Dan Berlin (01:25):
Nice. Thanks for telling us your journey. Can you
tell us about the beginning ofthat journey? How did you
discover UX?
Kaaren Hanson (01:32):
Yeah, so, one of my first jobs, I was actually
working at a company that was anational jury project where we
really worked on certain sidesof cases. So I worked for
victims of sexual discriminationand victims of sexual
harassment, or racialharassment, or sexual
orientation, harassment, ordiscrimination or whatnot. And
when I was in that role, Ireally was doing some exhibit
(01:52):
design, I was reallyunderstanding people and how
they thought, and I realized thepower of helping people to
understand the bigger story andseeing the forest for the trees,
and then also what it took toreally drive that change. That
wasn't actually the rightindustry for me, I actually
found it both very stressful,but also very boring. And so at
(02:13):
the time, there were a lot ofstartups in San Francisco that
were hiring essentially anybodywho had some knowledge of design
and research. And so I hoppedonto one of those startups, and
I learned a ton from my peers.
And then from then, I just keptmoving to company to company
taking on different roles,learning more, and eventually
realized that one of the areas Iam particularly effective at is
(02:34):
really helping to championdesign within an organization.
And so that has been what I'vereally gravitated towards is how
do I create conditions in whichteams can thrive and do their
best work? Not just design, butthe whole community with whom
they work those engineers, datascientists and product managers?
Dan Berlin (02:54):
Yep.Tell me a little bit more about that. What's your
philosophy or strategy when youcome in to build a great culture
for a design team?
Kaaren Hanson (03:02):
Yeah, I think a lot about do we have the right
people in the right places? Somepeople perform really well in
one job, but not necessarily inanother? Right? So within a
company, there are certain rolesthat where you need to do
different types of work. So dowe have the right people in the
right places? Is the broaderteam that quad is what we refer
to at Chase? Is that quad setuparound the customer's purpose,
(03:25):
the customer problem they'retrying to solve? Is there a
clear metric for them to know ifthey've succeeded? Not from a
business perspective, but from acustomer's perspective that then
ladders to that businessoutcome. And are they aligned on
the quality of the experiencethey want to provide? And how
are they going to measure that?
And once you've got a teamthat's aligned on this is the
problem we're solving, here'show we're going to measure
(03:47):
success, and this is the qualityof the experience we expect to
create, you're in pretty darngood shape. But it's a matter of
getting there and then makingsure that the processes around
that company are reinforcingthat customer centricity, and
reinforcing empowering the teamsas opposed to getting in the
way. Once you start to buildthat team, when you start to
(04:08):
build the relationships, thealignment, then is about really
tackling the broader companyoperating mechanisms or
structures to ensure thatthey're reinforcing conditions
in which teams can thrive.
Dan Berlin (04:20):
That's wonderful. I love how you describe the quad
as being based from a customerperspective, because often
enough, in enterprise, we seeteams or however we structure
our teams, based on businessgoals and the business way of
looking at it. And it'sheartening to hear you describe
it as a customer focused way offorming your teams.
Kaaren Hanson (04:42):
And I do have a very deep belief that if you do
the right thing by the customer,in most cases, you will do the
right thing by the business. Andpart of our job as designers and
product managers and data andtechnologists is to really make
sure that we're solving thecustomer problem in a way that
works for the business, becausewe do operate in a business. And
(05:03):
there are many ways to solvethat customer problem. So our
job is solving well for thecustomer. And in a way that
works for the business.
Dan Berlin (05:11):
I love it. That's what we do in UX. It's that
trifecta of business goals, userneeds, and the technological
ability to support these two,the way you just described that
was wonderful, thank you forthat.
Kaaren Hanson (05:22):
And what I think is really important is sometimes
where I see people and UXgetting stuck is because they
think of it as customer orbusiness. It's not, it's never
customer or business. It'scustomer and business.
Dan Berlin (05:33):
100%. Thanks for all of that. Very interesting, it's
always nice to hear how folksare looking at forming their
teams from a high level. Let'sturn to your chapter, Observe
Behavior is the Gold Standard.
Tell us about that, please.
Kaaren Hanson (05:48):
Yeah. So, I think that it can be very seductive to
listen to what people say. Andwe're very used to having
conversations with people. Andwe tend to believe what they
tell us because most people arefairly honest. Now, the downside
is, we also tend to believe whatthey tell us about how they will
act in the future. And so it canbe very easy for us to take what
(06:09):
people say, whether verbally orin writing, and assume that's
true. And when we do that, weare often wrong. We're not
always wrong, but we're oftenwrong. And the problem is, it's
hard to predict when are yougoing to be wrong? And when are
you not going to be wrong whenyou simply listen to what people
say, right? And I'm going togive you a couple of examples
(06:29):
from Intuit. And then I alsohave a couple examples from
Chase. But in one case, we wereworking on a payroll product.
And there was a question of,hey, when people come and
they're just trying to pay a newindividual on their payroll, do
they want to pay a person or dothey want to actually set up the
payroll, and then pay a person.
And when we ask people, someenormous majority, like maybe
(06:51):
90% of people said, I would wantto set up my payroll right away.
Now, fortunately, the teamdidn't necessarily believe what
they said, because often what wesay and do are different. So
they set up a painted doors testwhere they had people on the
site, you could either go paysomeone right away, or you could
save your payroll, what we foundwas the majority of customers
(07:12):
went to pay an employee rightaway, because it was a time of
need. Because of that, the teamwas able to get the funding they
needed in order to build asolution that solved this
customer problem of needing topay my employee. And they were
able to get that funding becausethey had that behavioral
evidence. And they didn't justlisten to what people say. Very
similarly, when I was at intoIntuit, another product, we were
(07:35):
looking at the experience of it,and we were talking to our
customers, after they did sometasks about what their
experience was, they were sayingthings like, I love it, it's so
easy, it's amazing. And then welooked at what they had actually
done. And what we realized isthey had made so darn many
errors, there are going to be inhell with their accountant in
(07:56):
about, you know, maybe three tofour months, right. So again, if
we had simply listened to whatthey had said, we would have
thought this company we boughthad a great product. But because
we also observed what they didand had very clear metrics for
success, we knew that it wasn'tactually that easy for them to
be accurate in what they weretrying to do.
Dan Berlin (08:18):
Right. And accuracy in the finance world is
critical.
Kaaren Hanson (08:23):
Yeah.
Dan Berlin (08:23):
Tell me about the painted door. You mentioned
doing a painted door research tofigure out what they were
actually doing. Can you expandon that, please?
Kaaren Hanson (08:31):
Yeah, well, essentially, what was done at
the time is there was simply twooptions on this particular page
that you would land on, right.
And this was actually on the webis before the app was strong as
it is now. And you could eitherset up my payroll, or you could
pay an employee. And it turnsout, majority of people want to
pay an employee, because whathad happened is maybe they had
(08:51):
just hired somebody, maybe theyhad forgotten that they even
needed to set up a payroll maybewas their first employee, they
didn't know what was happening.
And all of a sudden, it wasFriday, and they had to pay
somebody, and they didn't havetime to find their EIN, and you
know, other documentation, theyneeded to figure out all of the
information that had to be hadin order set this up correctly,
(09:15):
they just needed to get Joe apaycheck, right. And so they did
that. And then later on, theywould come back and set it up.
So let me give you anotherexample of the difference
between what people say and whatthey do at Chase. So we recently
conducted some research onbehaviors and needs of
consumers. And we were reallylooking at what people said
initially, and we asked them torate the importance of like
(09:38):
personalized shopping versusother drivers of customer value
like price or return policy,etc, etc, etc. And so when we
had this conversation, it turnsout when people talk,
personalization is ranked verylow, likely because it's a
fairly abstract concept, andit's hard to imagine what
(09:59):
personalization might mean,right? Or everybody might be
imagining it means somethingdifferent in their head. And so
to get around that, then what wereally looked at was how to
create examples ofpersonalization live, that would
be meaningful to individuals.
And by doing that, we start tohave very different
conversation. Now, the reason Ibring this up as an example is
because it goes to show thatthere are limitations sometimes,
(10:21):
and how people understand wordsand understand questions, which
is another weakness of simplylistening to what people say, as
opposed to really going further,and helping them more deeply
understand whether by showingthem something different, or
having them go after doing atask themselves.
Dan Berlin (10:44):
Yep, getting more of that reaction than just their
their words.
Kaaren Hanson (10:48):
Yeah, then just like what they're imagining your
thinking and what they'reimagining their future self is
going to do, right, that's a lotof imagination, which may be
accurate, or may not beaccurate.
Dan Berlin (11:00):
Right. So how can we get at these behaviors? What are
some of the ways that you UXersshould be trying to get at these
Kaaren Hanson (11:08):
Yeah, I mean, I think instrumentation and
behaviors?
knowing what are the core flows,the core behaviors are looking
for, and doing theinstrumentation for something
that is live, I think a lot ofit is the anthropology and going
into people's houses andobserving what they do today.
Because what they do today isprobably what they're going to
do tomorrow. Again, if you goback to the Intuit days, and
(11:31):
TurboTax, we would go intopeople's houses, and we would
see all the different places,they kept their tax forms,
right, and you know, all thedifferent materials they had,
and then somebody on thefrigerator. And so would be
like, you know, practically inthe garbage can pull it all
together. And they wouldprocrastinate then and now that
everything's electronic, well,let's not assume that everybody
has their act together. It canbe just as complicated and messy
(11:54):
and as much procrastination hasever said the best day and
actually, there's a great quote,I can't remember who made it.
But the best predictor of futurebehavior is past behavior, even
if the modalities and thetechnology is different.
Dan Berlin (12:06):
Yeah. And what about doing it in an artificial
setting versus in someone'shome? Right, bringing someone to
someone into a lab or doing aremote usability study versus
going to their place? You know,what are some advantages or
disadvantages there?
Kaaren Hanson (12:23):
Yeah, I mean, I'm a fan of all of that, I think
about it as a portfolio view ofresearch methods, every research
method has a benefit and adrawback, just like your stock
has a benefit, your bond has abenefit, and they both have
drawbacks you want both in yourportfolio. And so what I would
say is true about the usabilitystudies, whether in person or
remote is you are able to lookat behavior, if you set it up to
(12:44):
look at behavior, as long asyou're making sure is, what they
gave you accurate was thepayroll they made did have the
right taxes taken out, so theywon't be caught in the end,
right? It's very easy to dothese studies in a more rigorous
manner, which is probably goingto be more valuable to you or a
less rigorous manner. Now, ifthe goal is simply to have
everybody have some experienceof seeing what the customer goes
(13:07):
through, as they're trying toattempt something, fine, don't
worry about being rigorous,because then the goal is really
to make sure everybody has acustomer voice in their head or
has seen the customer and isaligned around that problem, and
aligned around the quality ofthe experience. But if you want
to get deeper, be more rigorous,and be very clear on what
success looks like and does not.
Dan Berlin (13:29):
Tell me a little bit more about that please. How did
you define rigor in the wayyou're describing it here?
What's the difference betweennon rigorous and rigorous.
Kaaren Hanson (13:37):
So I'll give you an example of a consulting
company we worked with at onepoint, and they were they were
fine. We just coached them towhat we needed. But initially,
we were going to do somebenchmarking on a particular set
of products and theircompetitors. And this was at a
previous company. And soinitially, what they had set up
was, you know, having people dothings and then asking
(13:58):
questions, like were yousuccessful and how easy was it
and you have the usualquestions. They didn't actually
have any behavioral evidence.
And so it was really pushingthem back and saying, Wait, now
we're in this flow, are we goingto say that they're successful?
What page do they need to getto? What is the screenshot that
they have to have and send in orwhat is the screenshot we have
to see, for us to know that thiswas a win, what needs to be on
(14:19):
that screenshot? And so gettingreally rigorous about what it is
gave us very different dataresults. But that wasn't
necessarily how that particularset of practitioners was
thinking to begin with. Again,going back to what are we going
to do tomorrow is probably whatwe did yesterday. They were
coming at it with the sameprotocol they'd used in many
(14:41):
other places.
Dan Berlin (14:43):
Gotcha. So was there anything else you were hoping to
convey here today?
Kaaren Hanson (14:48):
I guess I would just say that there's nothing
wrong with talking to people. Idon't want anyone to get the
wrong idea, right. Talking topeople is incredibly useful and
can help you understand thecontext they're in and it can
help you understand whatmotivates them. And some
struggles they might have had orsome dreams and wishes they
have. But don't assume thattalking is enough.
Dan Berlin (15:10):
Well put, don't assume that talking is enough.
And this comes back to be mixedmethods and always triangulating
on your users using mixedmethods, whether it's a survey
and interviews or usability andinterviews or however you're
doing your triangulation. It'sso important to get that that
quant and qual almost.
Kaaren Hanson (15:30):
It is and it's also okay if your goal is simply
to make sure the quad like theproduct managers and the
technologists and the datascientists and the designers and
whatnot. If your goal is to makesure everybody understands who
this person is, talking may befine. But if you're trying to
make decisions around whatproducts to go after, is their
product market fit. You know, isthis a good enough experience
(15:52):
that they're able to besuccessful and confident?
Probably talking will not beenough in many cases.
Dan Berlin (16:00):
Well Kaaren, you gave us a lot of great
information here today. Thankyou so much for joining the
podcast.
Kaaren Hanson (16:05):
Oh, thank you so much. It was a pleasure.
Dan Berlin (16:07):
My guest today has been Kaaren Hanson, who wrote
the chapter Observed Behavior isthe Gold Standard. Thanks for
listening, everyone. You've beenlistening to the 97 UX Things
podcast, a companion to the book97 Things Every UX Practitioner
Should Know published byO'Reilly and available at your
local bookshop. All bookroyalties go to UX nonprofits as
(16:27):
well any funds raised by thispodcast. The theme music is
Moisturize the Situation byConsider the Source and I'm your
host and book editor Dan Berlin.
Please remember to find theneeds in your community and fill
them with your best work. Thanksfor listening
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Therapy Gecko
An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.