STARTUPS PULSE - Why Your User Research Is Lying to You (And the 3-Question Fix)

Wix Acquires Base44 for $80 Million to Enhance AI-Driven No-Code Platform Meta Intensifies AI Talent Race with New Hires After Failed Safe Superintelligence Bid Why Your User Research Is Lying to You (And the 3-Question Fix) #startups, #userresearch, #AI, #nocode, #acquisitions, #podcast, #innovation

Episode Transcript

(00:00):
Welcome to Innovation Pulse, your quick no-nonsense update covering the latest in startups and

(00:09):
entrepreneurship news.
Israeli startup Base 44, founded by Maurer Shlomo, has been acquired by Wix for $80 million
in cash, while Meta strengthens its AI initiatives with strategic investments and key hires.
After this, we will dive deep into improving user research practices to enhance product

(00:32):
development.
Israeli developer Maurer Shlomo's startup Base 44 was sold to Wix for $80 million in
cash just six months after its inception.
Base 44, a vibe coding platform, allows users to build software applications without coding
by using text prompts.

(00:54):
The platform takes care of the entire application infrastructure, including database, storage,
authentication and analytics, and supports email, texting and maps integration.
This makes it valuable and unique as it democratizes software development non-programmers.
The company rapidly grew to 250,000 users and was profitable, earning $189,000 in profit

(01:21):
in just one month.
Great high costs.
Shlomo, previously known for founding the data analytics company Explorium, leveraged
his network to form partnerships with major Israeli tech companies like eToro and similar
Web.
His decision to use Anthropics' Claude LLM for cost efficiency caught Amazon's attention,

(01:42):
leading to a demo at an AWS event.
Wix's acquisition of Base 44 adds a profitable and innovative vibe coding platform to its
no-code offerings, enhancing its product suite.
Up next, we're exploring Meta's strategic AI investments.
Meta is making waves in the AI industry with significant investments and high-profile hires,

(02:07):
aiming to bolster its capabilities in developing advanced AI technologies.
Recently, Meta CEO Mark Zuckerberg attracted scale AI founder Alexander Wang through a $14,300,000,000
investment, securing a 49% stake in the startup.
This move is part of a broader strategy to compete in the AI talent war against major

(02:31):
players like Google and OpenAI.
Meta also attempted to acquire Safe Superintelligence, co-founded by AI pioneer Ilya Tsutskev, but
was unsuccessful.
Instead, Daniel Gross, Safe Superintelligence's SCEO, and Nat Friedman, its venture partner,
have joined Meta.

(02:52):
Both will work under Wang, bringing their extensive experience and networks, including
connections through their venture firm, NFDG.
Meta aims to leverage these new hires to push towards advancements in artificial general
intelligence, which aims to create AI systems with capabilities equal to or surpassing human

(03:13):
intelligence.
This aggressive expansion highlights Meta's commitment to leading an AI development and
positioning itself as a formidable competitor in the AI landscape.
And now, pivot our discussion towards the main entrepreneurship topic.

(03:36):
All right, everybody.
Welcome back to Innovation Pulse.
I'm Dana, and as always, I've got my co-host, Yakov, with me today.
We're diving into something that's been bugging me for weeks now.
This whole idea that we might be asking the wrong questions when we do user research.
Oh, this is good, Dana.
So picture this scenario.
You're sitting in a focus group, and you ask a bunch of 20-somethings which dating app

(04:00):
they'd use next time they're single.
Not one person says Tinder, but then, same group, you ask if they've actually met someone
on Tinder in the past year, and nearly everyone raises their hand.
Wait, hold up.
So they're actively using the app they claim they'd never use again?
Exactly.
And that disconnect right there, that's the whole problem with how we approach user research.

(04:24):
We're asking questions that sound logical, but actually lead us completely astray.
This is making me rethink every user interview I've ever conducted.
So what's going on here?
Are people just lying to us?
Not lying, exactly, but there's this fundamental issue with the types of questions we ask.
Think about it, every user research question basically falls into one of three buckets.

(04:48):
We're either asking, what did you do?
What will you do?
Or what are you doing right now?
Okay, lay it on me.
Break this down for me.
So, what did you do?
Questions are like asking someone to walk you through the last time they booked a flight,
or what product they used to solve a problem before.

(05:08):
These are actually pretty reliable because people have decent memories about their actual
behaviors.
Makes sense, it's concrete, it already happened.
Right, and that leads us to the second type.
What will you do?
Questions, this is where we get into trouble.
We ask things like, would you use this feature?

(05:29):
Or how much would you pay for this service?
And here's the kicker.
Questions are terrible at predicting their own future behavior.
Oh, I can already see where this is going.
It's like when I tell myself I'll definitely go to the gym tomorrow, and then tomorrow comes,
and suddenly Netflix looks way more appealing.
Exactly, and it gets worse when virtue signaling kicks in.

(05:53):
People will say they'll invest significant money in their health because they see themselves
as healthy people.
They'll claim they'll buy local, even if it costs more.
Because that's the right answer.
So we're not just dealing with bad predictions, we're dealing with people giving us the answers
they think we want to hear, or the answers that make them look good.

(06:14):
Bingo, sometimes these future focused answers aren't just wrong, they're the complete
opposite of what people actually do.
But here's where it gets interesting.
The third type of question, what are you doing right now?
Can be pure gold.
How so?
When you ask someone to narrate their current actions while they're actually using your

(06:34):
product, like, talk me through what you're thinking as you look at this screen, you bypass
what I like to call their internal PR team.
Their internal PR team?
You know that voice in our heads that rationalizes our past decisions and makes our future selves
sound way more logical and disciplined than we actually are.

(06:54):
When someone is actively doing something and talking through it in real time, they don't
have time to filter their responses.
So you get the raw, unvarnished truth about their experience.
Exactly, when someone says, I'm confused about where to click next, or I assume this
button would do something else.
That's genuine feedback you can actually use to improve your product.

(07:17):
This is really shifting how I think about research design.
So if I'm understanding correctly, past and present questions are more reliable than
future questions?
That's the pattern, yeah.
What users have actually done, and what they're doing right now will consistently give you
better insights than what they imagine they might do.
But I'm guessing future focused questions aren't completely useless, right?

(07:40):
No, they have their place.
Human imagination has flaws, but it also has tremendous power.
I remember reading about a case where a teenager predicted that short form vertical videos
were the future.
This was before TikTok had even launched in the US.
He was obsessed with old Vine compilations on YouTube.

(08:01):
Even though Vine had already shut down.
And he was completely right about where social media was headed.
Exactly.
So these imagination questions can help you tap into your customer's vision of the future,
but you need to understand their limitations and use them strategically.
Right, and that leads me to wonder, how do we actually implement this in practice?

(08:22):
Like what should product teams be doing differently?
Great question.
First, start classifying your research questions.
Look at your last research protocol and figure out how many questions asked about past behavior,
how many about present behavior, and how many about future behavior.
I bet most teams will discover they've been leaning heavily on those unreliable future

(08:44):
questions without even realizing it.
So it's about being more intentional with question design.
Exactly.
Prioritize questions about what users have actually done and what they're doing in the
moment.
When you do ask future focused questions, dig deeper into the why behind the answers.
The motivations often reveal more truth than the stated preferences.

(09:08):
And I'm thinking about something else here.
Context matters a lot, right?
Like you wouldn't ask men to test a menstrual cycle tracking app.
Perfect example.
But teams make similar mistakes all the time.
They'll test with people who are way outside their ideal customer profile or ask people
to imagine scenarios that are completely unrealistic for them.

(09:29):
Like planning luxury trips that are way beyond their budget.
So even those present moment narration questions can become imagination questions if you're
not careful about who you're testing with.
Exactly.
The key is making sure your participants are actually in a realistic context for using
your product.
This all makes me think about how we approach product development more broadly.

(09:51):
We're essentially trying to predict the future.
We want to build something today that'll be useful tomorrow.
Right.
And the traditional approach is to just ask users about that future directly.
But what we're learning is that observing actual behavior, past and present, gives us
much better predictive power than asking people to imagine future scenarios.

(10:13):
It's like the difference between asking someone if they'd buy a product versus watching
them actually go through a purchase process.
Perfect analogy.
And here's the thing.
That gap between what people say they'll do and what they actually do.
That's not just a research curiosity.
That's exactly where your most valuable product opportunities are hiding.

(10:34):
Because if everyone says they want feature X, but their actual behavior suggests they
need feature Y.
feature Y is probably where the real opportunity lies.
You got it.
Great products are built on observations of real behavior, not on aspirations or what
people think they should want.
So for our listeners who are running product teams or doing user research, the big takeaway

(10:55):
is to shift from asking, what would you do?
To asking, what did you do?
And what are you doing?
That's the core message, yeah.
Treat past and present behavior as your foundation for making product decisions.
Use future focused questions sparingly and strategically.
And always trace your findings back to the type of question that generated them.

(11:20):
It'll help you understand how much weight to give different insights.
This has been eye-opening, Yakov.
I'm definitely going to be looking at user research protocols with a completely different
lens now.
Same here.
And for everyone listening, maybe take a step back and think about your own recent research.
How much of it was based on what users actually do versus what they say they'll do?

(11:44):
You might be surprised by what you find.
Absolutely.
And remember, the goal isn't to stop talking to users.
It's to ask them the right questions so we can build products that actually solve real
problems.
Thanks for diving deep with us today, everyone.
Until next time, keep innovating.
Catch you next episode on Innovation Pulse.

(12:09):
We've explored how Israeli startup Base44's rapid rise and acquisition by Wix highlights
the booming potential of no-code platforms, while Meta's bold investments in AI signal
its competitive aspirations.
Dana and Yakov also dissected the challenges in user research, urging teams to prioritize

(12:30):
real behavior insights over future predictions.
Don't forget to like, subscribe, and share this episode with your friends and colleagues
so they can also stay updated on the latest news and gain powerful insights.
Stay tuned for more updates.