Ask Participants to Tell You What You Don't Know to Ask (feat. Amanda Rotondo)

Amanda Rotondo, PhD discusses her book chapter "Ask Participants to Tell You What You Don't Know to Ask" and provides some insight about building team empathy in a remote world.

Sponsored by Watch City Research
Watch City Research is your trusted UX research partner

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Episode Transcript

Dan Berlin (00:14):
Hi, everyone, and welcome to another episode of
the 97 UX Things podcast. DanBerlin here, your host and book
editor. I'm joined this week byAmanda Rotondo, who wrote the
chapter "Ask Participants toTell You What You Don't Know to
Ask." Welcome, Amanda.

Amanda Rotondo (00:28):
Hi, there.

Dan Berlin (00:30):
Thanks for joining the podcast. Can you please tell
us about yourself?

Amanda Rotondo (00:34):
Sure. I am a user experience researcher in
the Boston area. I've been doingthis for over 20 years. And
yeah, I'm excited to be here.

Dan Berlin (00:43):
Excellent. Yeah.
Tell us about your UX journey,how you discovered UX and how
you went up where you are today.

Amanda Rotondo (00:49):
I kind of discovered UX as UX discovered
itself back in the 90s. I was inundergrad and I came across the
concept of hypertext fiction,which was brand new at the time.
And it led me to realize whatcould be when online experiences
were user driven. And how thatwas so different from... it was

(01:11):
essentially brochureware andsort of lit the path for me
that, wow, there's so much morewe can do with this technology.
So when I graduated, I got ajob, information architecture,
of course, and did that for awhile and wanted to learn more
about the way that what I wasdoing and the media I was
putting out there affectedpeople. So I ended up getting a

(01:33):
master's in media effects fromPenn State, and then went back
to to UX research independently,and was doing UX research and
design. But really, during gradschool, I fell in love with
research. And I did that for awhile. And I kind of started
teaching a bit to see if Iwanted to maybe go the professor
route. And I loved it. So Iended up going back for a PhD in

(01:55):
human computer interaction. Andwhen I finished that, I did love
teaching, but I also really missjust being in the field. So I
continued with consulting.

Dan Berlin (02:07):
Great, are there any highlights from working on your
on your PhD that are worthsharing here?

Amanda Rotondo (02:13):
For my, for my dissertation, I studied the role
of empathy in the success ofdesign teams working at a
distance. And this was instarted in 2009. So this was way
before the pandemic before, youknow, non co located working was
big, really, at all. And it wasalso before people were really

(02:33):
talking about empathy. Now, it'ssuch a buzzword. And this was at
a time where I had to explain topeople what that meant. So it
was really new and really fun tobe working. One of the things
that my research... my researchdidn't come up with this, but it
did substantiate it further,is... So when you have a group
of people working remotely, solike now we'll just say by Zoom,

(02:56):
it can be very hard toestablish, you know, team
cohesion. And the one thing thatwas discovered at this point
that has the biggest effect onteam's success working together
and enjoyability of each otherbeing on the team is having had
met in person one time. So yeah,and it's so funny, because it's

(03:18):
so simple, right? Like, youknow, people are trying to think
of all these very convolutedthings of ways that, that they
can build teams from afar andhigh tech, this, et cetera. And
really, you just have to meet inperson one time, sort of prove
the other is real. And then, andthen the relationship changes
dramatically. I thought that wasso powerful.

Dan Berlin (03:39):
Yep. Makes sense, you know, putting a real face to
the name of the people thatyou're interacting with. And
even just that one time, I guessit makes a big deal.

Amanda Rotondo (03:48):
Exactly.

Dan Berlin (03:49):
Yeah. Great. Thanks for that. Your chapter, "Ask
Participants to Tell You WhatYou Don't Know to Ask." Can you
tell us about that, please?

Amanda Rotondo (03:57):
Sure. So this all boils down to asking one
question, and that question is,is there anything you'd like to
tell me that I haven't alreadyasked? And it's kind of a
strange question to ask. But Ifound it to be extremely
powerful. And I sort ofdiscovered it by accident. I was

(04:18):
working on this project. I wasin house at the time. And it was
for a company that was workingto conglomerate various data
across multiple onlineretailers. So we had an
attribution system where we hada big team in India that was
going through all of theseindividual pages of items for
sale and giving, I think it wasover 100 attributes about each

(04:42):
one so that they could then befiltered and sorted. So it's a
very intensive, very expensiveoperation. And what was
happening is the data from thesystem was coming back an
absolute disaster. It was a messthere. There was a ton of
incorrect data that was soweirdly incorrect that it didn't
quite make sense that it wouldbe, you know, a

(05:04):
misunderstanding. So this fellon me to try to figure out. So
we tested the system from justa, you know, is it is it
functional perspective, and itwas working fine. We did cross
cultural tests, because werecognize that we developed the
system for a different culture.
So we did that, it was fine. Wescheduled times to watch

(05:26):
employees while they were doingthese attribution exercises, and
they were doing a perfectly goodjob, no problem. So, we're just
baffled, like, what couldpossibly be so dramatically
wrong? So in the end, we decidedto, you know, well, it wasn't
really in the end, it was sortof, as we went along, we were

(05:47):
interviewing people and talkingto various people and trying to
figure out where to go next. Andone of the interviews I did,
toward the end was with somebodywho was working the attribution
system, just, you know, one ofthe people on the floor, and I
was so frustrated at this point,because we had tried so much,

(06:07):
and I couldn't figure this out.
And I just sort of blurt it out,I said, you know, is there
anything I'm not asking you, youthink I need to know. And it was
such a weird thing, it just sortof came flying out of my mouth.
And the participant just sort offroze for a second. And, you
know, kind of looked at me overZoom. Like, "Well, you just went

(06:28):
off script." And then she said,"Well, maybe about the volume
bonus." And I was just what areyou talking about? And so it
turned out the floor managerswere incentivized based on how

(06:48):
many products had beenattributed during their during
their shift. So they were thenprompted to further incentivize
their teams to produce highervolume. So they had sort of this
very informal deal, where it'slike, hey, whoever, you know,
whoever attributes the mostproduct on this shift, I'll give

(07:09):
you 50 bucks. And it was thisvery casual, very, you know,
sort of informal thing. And whatit did, was it totally upended
the system and made it so thethe people inputting the data
were far more driven to just flythrough and get as many as they
could versus any concern forquality. So once we discovered

(07:33):
that, we realized, well, theproblem isn't the system, the
problem isn't, you know, themechanics, the problem is the
motivation. And we were able toget in and tweak that system, or
tweak their the informal systemthat had developed and solve the
problem.

Dan Berlin (07:50):
That's super interesting. And it speaks to a
number of things. First off, howthere are happy accidents when
we are doing research. And whenyou have a question that just
comes out like that, and that'sa wonderful question. But it
speaks to the fact that we don'tnecessarily know all of the
things to ask when we go into aninterview. And it allows the

(08:13):
participant to guide us to themost important things.

Amanda Rotondo (08:17):
Right? And you know, one of the things I always
always think about with, youknow, what makes somebody a good
UX researcher, what sort of atthe pith of that, and I think
one of the most crucialcharacteristics is humbleness.
You have to recognize that,yeah, you might be the one who
knows how to ask questions. Andyou might be the one who knows
how to analyze data, but youdon't know anything about the

(08:38):
product that you're working on.
And you're talking to people whoknow everything about it. So you
know, being able to just justsay, "Look, I don't even know
what to ask you," sometimes cancan result in some some really
great observations.

Dan Berlin (08:51):
Yep. And we can learn things that go beyond the
things that we're thinkingabout. In your case, you were
concentrating on the userinterface itself. But it
uncovered this business processproblem, which, you know, user
experience and business processare often inter intertwined. But

(09:12):
it's interesting how that onequestion allowed you to uncover
something that was... not beyondthe scope of what you were
investigating, but an importanttangent.

Amanda Rotondo (09:23):
Absolutely. And I think a lot of the time we're
set up to go into these projectsa little bit skewed, because you
know, more often than not, whenwe go in to do usability
research, user experienceresearch, were guided and sort
of introduced to the project bythe stakeholders who are most
often, you know, higher up inthe company. Maybe they don't

(09:46):
even use the system. I mean, Ican't tell you how many times
I've had project owners who havenever seen a screen. And so
they're the ones really kind offraming the project up for us
and telling us who to talk toand, and being the gatekeepers
And it's it's extremely commonthat the problem is something
that they don't know to directus to.

Dan Berlin (10:08):
Yep. No, that's a great point. It allows us to...
not eliminate, but check ourbiases in terms of how we're
approaching the research andrely on the participant to guide
us.

Amanda Rotondo (10:19):
Exactly. And even when the participants are
stakeholders, right. So whenyou're doing stakeholder
research at the beginning, a lotof times, the stakeholders are,
you know, they're, they'reexisting within a power
structure, they have a boss, theboss has a an agenda, they have
people who work for them thatthey need to protect in certain
ways. There's so many politicsand they might not be empowered

(10:42):
to speak out on certain issuesor to, you know, really kind of
say, "Hey, my boss is great, butyou know, she's not really
guiding you in the right way."So by giving this opportunity in
a, you know, one on oneinterview, in a very sort of
comfortable setting, it oftenlets that power dynamic kind of

(11:05):
shimmy to the back a little bit,and lets people speak a little
more openly.

Dan Berlin (11:09):
Yeah. So I was really happy when you wanted to
write this chapter. Because thatquestion is my favorite part of
the research. Right? Is thereanything I didn't ask you today
that you think is important forme to know? And it's so
important, or so useful outsideof research too. I mean, we're
always researching in our world,right. But when we're doing, as

(11:32):
you said, when we're talkingwith stakeholders, or we are
chatting with colleagues, wedon't know all the questions to
ask, and I think peopleappreciate when we look to their
perspective for what we shouldbe thinking about.

Amanda Rotondo (11:44):
Exactly. And there's two times when I'm out
in the wild, you know, in my ownpersonal life, where I realized,
like, "Oh, God, I'm aresearcher." The first is when I
ask a question, and like, youknow, unbiased, I'll say to my
kids, like, "Do you want to goout for ice cream or would you
prefer not to go out for icecream?" And then the second is
when I bust this question out atthe end of, you know, just

(12:05):
regular interactions and I doit, because like you said,
there's so much value in it, youknow, just in so many different
situations.

Dan Berlin (12:13):
Yep. So I'm a UX consultant, always trying to
sell business. And honestly, Ialways ask that question when
scoping a project. It's becausescoping a project is its own
research project. And so look tothe potential client to help
guide me on the things I may notbe thinking about.

Amanda Rotondo (12:33):
Sure. I think that's a great question to ask
at that point. And also, I thinkit clues your potential client
into the fact that you are awarethat you don't know everything.
And you may stand out withintheir their process as somebody
who's who's truly curious. Andlike I said, humble, and, you
know, really going to try tofind the answers and not just

(12:55):
presented a shiny researchreport.

Dan Berlin (12:57):
Yep. And I love what you said about keeping humble
because that's what we have todo as researchers because we
don't know everything by alongshot. And humbleness is what
allows us to be curious andbeing curious allows us to
uncover deep insights.

Amanda Rotondo (13:12):
Exactly.

Dan Berlin (13:13):
So it was there anything else from your chapter
you were hoping to convey heretoday?

Amanda Rotondo (13:16):
Are you asking me if there's anything that you
didn't ask me that I want peopleto know?

Dan Berlin (13:21):
Exactly. Well done, well done.

Amanda Rotondo (13:25):
You know, when this question kind of came
flying out of my mouth, I wassurprised at myself because it
felt very uncontrolled. Right?
As a researcher, I think you'resupposed to sort of... there's
this idea that you're supposedto be very formal, very, you
know, bullet pointy, kind of,and this was not an instance of
me doing that. That was this wasnot me playing that role. This

(13:47):
was me being myself. And I thinkthat to me, was a very powerful
observation to that. You have tobe yourself when you're doing
your research. First of all,because, you know, you kind of
have to have faith in yourselfthat you, you're doing a good
job, you know, you know whatyou're doing, you've got the
right instinct for this. Andthen second of all, just the

(14:09):
extent to which participantsrecognize when you're being
yourself and being genuine. Andthat makes a huge difference in
how much they're willing todisclose to you.

Dan Berlin (14:19):
Very well put. You have to be yourself. People and
participants will sniff outacting pretty quickly and clam
up. If you're genuine, it'sdefinitely gonna go a long way
to keep them comfortable.

Amanda Rotondo (14:31):
Exactly. And every time I've tried, you know,
early in my career when I triedto be the formal fancy
researcher, it's been adisaster,

Dan Berlin (14:38):
Agreed 100% And it's a balance between you know,
regular, everyday you versuswhat's in who you are in the
lab. But you have to let thatthat personality come through,
that genuineness come through.

Amanda Rotondo (14:52):
Exactly.

Dan Berlin (14:52):
Great. Well, Amanda, this has been a wonderful
conversation. Thank you so muchfor joining me here today.

Amanda Rotondo (14:56):
Absolutely. This has been fun. Thank you again.

Dan Berlin (14:58):
My guest today has been Amanda Rotondo, who wrote
the chapter "Ask Participants toTell You What You Don't Know to
Ask." You've been listening tothe 97 UX things podcast. Thanks
for listening, everyone.
You've been listening to the 97UX Things podcast companion to
the book 97 Things Every UXPractitioner Should Know
published by O'Reilly andavailable at your local

(15:19):
bookshop. All book royalties goto UX nonprofits as well any
funds raised by this podcast.
The theme music is Moisturizethe Situation by Consider the
Source and I'm your host andbook editor Dan Berlin. Please
remember to find the needs inyour community and fill them
with your best work. Thanks forlistening