From A to B

We are temporarily renaming this podcast From D to A - From Data to Action. Because data is almost useless unless you do something meaningful with it.

Josh "One of a Kind" Silverbauer hopped on the pod to dig deeper into the 'analytics' and 'data' part of product building. Identifying better ways to tell the 'data' story, and better ways to communicate that data so you actually can get shit done.

We got into:

- Josh's Pro Tip for co...

Are fake door tests ethical? We are super purposefully kinda misleading people to see if something is worth building.... (cue Wile E. Coyote reference).

Jon "Metal as Fuck" Crowder had a SUPER interesting post about the ethics of fake door testing, which I had to have him on the pod to discuss this topic. But we also got into a more broader conversation around ethics of A/B testing too!

We got into:

- Are fake door tests truly ethica...

Juliana "Run the Jewels" Jackson is someone who I consider a thought leader in the AI/digital marketing space, and I needed her POV on how to understand the impact of LLMs to user behavior.

So I had her on the pod. And she shared some insightful information into how LLMs actually work, how it's meaningfully (or maybe not so meaningfully) changing behavior, and what you should be doing about it.

(PS vote for me for a thought leaders...

Are manipulation and bribery the keys to building a culture of experimentation?

No.

(Well, maybe)

Jenni "Broadway" Bruckman (Director of Digital Product at YETI) joined this week to help broad the topic of building "cultures" which foster data-driven thinking and experimentation. It's different than focusing on explicitly building a culture of experimentation, in that your goal is to basically build a culture of fun, transparency, an...

A/B testing tools - all they do is randomize traffic. So why are we spending thousands of dollars for it?

Jonas "Elvis" Alves (of AB Smartly) joined the pod to discuss some of the major challenges in building A/B testing tools. It kinda started in a way I had no idea it would - that you're not buying a testing tool, but buying a... trust platform?

Jonas throws some shade at other testing tools, but I bleep it out, don't worry ;) . ...

Is 'ethics' something we've undervaluing when it comes to experimentation? Should we actually care about this nerd stuff of 'ethics'?!

Well, yes. Obviously. AJ "Does No Wrong" Long joined Shiva to chat about this specifically, as someone who works in the fantasy sports realm of things, ethics can be a contentious subject.

We got into:

- Why "users first" is really the only way to foolproof ethnical experimentation (and low-key, the o...

Emotion in data analysis, from what we've been told, should never happen. But the kicker - humans are humans. We use emotions in literally everything we do. So how do we channel it best to make sure we are positively impacting the business?

Gerda "The koala" Thomas joined Shiva to chat about this topic. Where does it make sense to channel emotion? In getting EXCITEMENT about testing, and ideas! But maybe we park it when actually ana...

Don't know what the hell CRO and experimentation is? Trying to understand the basic concepts and process?

We gotchu. Rommil "The Thrill" Santiago and Shiva Manjunath went into all the common topics, and we can even help you explain this to others who might not fully understand what you do in CRO.

Please passive aggressively share this episode to people who don't understand CRO. I will fall on my sword for you.

We got into:

- Why CRO...

Have you ever struggled to get buy in from senior leadership? Never in your entire life, right?

Well I've faced it. A lot. And I wanted to get in the mind of a super seasoned CMO to truly understand what makes them tick. What they care about. What they DON'T care about.

Who better than Kelly Hopping, CMO at Demandbase. And y'all, this one is CHOCK FULL of amazing insights from her to better understand how to communicate what you're ...

Finding clients is hard enough - finding GOOD ones? Feels damn near impossible.

I enlisted the help of Tas Bober to help navigate this complex field of getting client work, and explored topics related to CRO/client work as well!

We got into:

- Bringing confidence to the client relationship (you don’t always have to cave in to EVERY little freaking thing they want you to do)

- Running from bad clients when they don’t pass t...

People are finding websites more and more manipulative. And we don't mean this word lightly.

Why is this the case? In large part, it's because of the shitty tactics people are using on sites. But really, it's because websites care far more about short term sales and metrics instead of focusing on LTV. And that leads to hacks, not long term gains.

Made With Intent released a REALLY great report which went into detail on this, and I ...

We back out here with the amazing Emma Travis to dig into 'quality vs. quantity' when it comes to experimentation programs. The conversation Ben Labay keeps trying to force, but a bit more nuance to it.

There is a time and place for quality. But there is a time and place for velocity and scaling your program up in terms of number of tests run. How do you know when to do it? And what are triggers to get you .... ahem.... From A to B...

How many of you have seen a website teardown and thought “Dude you literally have 0 data, this is just a vibes analysis?”

I have that thought regularly. And it got me thinking about heuristic audits as a whole, and trying to understand what makes a good heuristic audit good, and a bad heuristic audit bad. I enlisted the help of my mortal enemy Rishi Rawat to explore this. 

Vibes were immaculate, and he actually said ...

Problematic stakeholders could be convinced if they... just got data. Right?

Nope. Sometimes it's deeper than that. You gotta understand what makes them tick. Is it data? Is it optics? Maybe you should be running heuristic audits on your stakeholders instead of your website...

In the second time EVER, someone else has changed my mind on something. Leave it to "Kingpin" Finn McKenty, who's apparently gone from Punk Rock MBA to Finn M...

Do you REALLY know what cookies are? Like really, REALLY know? What about GDPR? What about PII?

I know the words. But what do they REALLY mean? I enlisted the help of Eddie "The Techie" Aguilar to help me simplify some of these complex topics, and help me create meaningful next steps on how to address PII concerns and other marketing-related issues in data collection.

We got into:

- Simplified definitions of cookies, data collect...

It happens rarely, but somehow Linda was able to change Shiva's mind this podcast. And we got it ON THE RECORD.

What did she change his mind on? Linda "The Winna" Bustos brought some compelling conversation to the fact that company CAN differentiate in 'cookie cutter' websites (e.g. Shopify) in really, really creative ways.

We got into:

- The importance of copy when you're using a generic 'cookie cutter site' (e.g. Shopify tem...

Do product/CRO people REALLY know well? There may be some Dunning Kruger happening, and maybe we really don't get why they keep complaining about our initiatives.

Spoiler: Many times, they have a VERY legitimate point!

Who better than the freaking VP of Nielsen Norman Group "Checkmate" Kate Moran to come dish all the things we're getting wrong about UX, and how to better work together.

We got into:

- What product teams/CRO can...