Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Michael Hartmann (00:01):
Hello everyone
, welcome to another episode of
OpsCast brought to you byMarketingOpscom.
Powered by all the mo pros outthere, I am your solo host,
michael Hartman.
Joining me today is Andy Caronto talk about curiosity,
attribution psychology and howall of it connects to marketing
ops in surprising ways.
Andy is currently president ofRevenue Pulse, where she brings
(00:22):
nearly two decades of experienceacross marketing operations,
sales ops, platform strategy andconsulting.
Her journey started in costumedesign and publishing before
moving into the world of salesand eventually becoming a
Salesforce admin.
Along the way, she found herniche in the marketing tech
space, blending creativity,systems thinking and human
insight.
So, andy, welcome.
Thanks for joining us.
Andy Caron (00:42):
Excited to be here.
Michael Hartmann (00:44):
By the way, I
try to do this and I still
forget.
Did I pronounce your last namecorrectly, caron?
See, I should have asked.
Andy Caron (00:54):
That's fine.
Many people say Karen and Idon't even correct.
It's totally fine, karen.
Caron, my first name isactually Androlin, which is one
of the reasons I go by Andy andit frequently gets mispronounced
as Adrenalin.
So my last name beingpronounced correctly or
incorrectly is the least of myworries when you have a first
name like that.
Michael Hartmann (01:15):
I mean I won't
take this off, but my name is
spelled.
It's not that hard, but it'sspelled oddly enough with the
two Ns that I do getmisspellings.
It used to really bother me andI finally am like it doesn not
that hard, but it's spelledoddly enough with the two ends
that I do get misspellings.
It used to really bother me andI finally I'm like that's
bothering me.
But there was an episode at the, at the driver's license office
one time that there's a storythere.
(01:36):
So anybody who's listening if Isee you, see me, you know you
can.
Andy Caron (01:40):
You're welcome to
ask me about that at some point
and I'll tell you the story forMr Her own, frequently enough
that, uh, you know I just allbets are off, it's all good.
Michael Hartmann (01:49):
Wow, yeah,
yeah, here we go.
All right, well, sorry aboutthat, we'll carry on, but let's
yeah, let's start with yourbackground.
Um, you, you, you began incostume design and publishing,
and publishing, um.
So I'm just curious, like,first off, what you may not know
if you've listened to a lot ofepisodes.
You're not the first personwho's got some sort of
connection to, like theaterworld.
Um, that's been a guest and isin marketing ops now.
(02:11):
So I think that's aninteresting fact.
But what, what drew you to that?
How did how did you evolve,like, tell that story?
Andy Caron (02:20):
uh, I fell bass
awkward into mops is, is, is the
long and short of it.
But, um, the the journey, uh,in a sense, I guess, was that I
started out pretty young with aninterest in theater and did a
lot of school plays and uh waspart of, you know, audition
(02:40):
group, choir and all that kindof stuff.
And in high school I had theopportunity to work at our local
repertory theaters wonderful,um, small theater up in the
Sierra Nevadas.
Uh did great, uh, greatproductions and I did everything
from backstage work to soundproduction and back again.
(03:01):
And my mom actually had abackground in clothing design
and I grew up with that.
I grew up with someone who wasalways sewing and making things.
I showed a proclivity and atalent for it early as well.
You know Barbie, clothes andall the things you'd expect of a
child, with scraps of fabricand a needle to hand.
(03:22):
So I ended up on a productionrun of Annie Get your Gun,
stitching up a very overzealousdancer's pants every night
because he would tear them, andfrom that got recruited into the
costuming division of thetheater.
(03:44):
I absolutely loved it.
I've always had a passion forcostume and the idea of being
able to create an experiencedpersona through clothing Just
absolutely fascinating.
So after I moved on to college Istarted picking up local
theater work and when Igraduated I actually moved down
(04:06):
to LA to pursue costuming infilm.
I discovered I wantedabsolutely nothing to do with
the movie industry, as it turnsout, not my vibe, as the kids
would say and so from that Iactually had a referral from a
friend from college into gearbook publishing, of all things.
(04:27):
It was my entree into AdobeInDesign 2.0 and also all things
you know, education oftechnology.
To those that haven'tencountered it before, don't
know how to use it.
So, working with middleschoolers and high schoolers to
(04:49):
help them learn Adobe, help themlearn journalism.
Michael Hartmann (04:54):
So it wasn't
just the, it wasn't just the
publishing of the yearbooks.
No, it was also helping thestudents.
Andy Caron (05:00):
Yes, the end
production and delivery of the
yearbooks themselves.
So I got to work with about 40schools across the Los Angeles
basin for nearly four years andabsolutely loved it.
It was a hard industry to be in, especially with yearbook sales
kind of going down at thatpoint, particularly in that
(05:22):
region, and ultimately Irelocated out of LA and at that
time also particularly in thatregion, and ultimately I I
relocated out of LA and um atthat time also exited that
company.
But very cool experience, um,really cool uh sort of training
wheels for some of the futurethings that I would encounter a
lot of sales, a lot of marketingin that, in that role to bring
(05:42):
on new schools uh into the, thepublishing um uh pantheon on on
that side and uh, thatdefinitely carried forward uh
with me, along with the creativeaspects of that, as I started
to look for where I wouldeventually land Um land.
(06:03):
Oddly enough for someone who wasraised by a very out in the
world working mom, I had somehowbeen raised with this idea that
I would grow up to be a stay athome mom.
I don't know why my mominstilled that or kind of put
that in there, but she wasactually the first female
photographer to work for FordMotorsports.
(06:23):
So I don't know if maybe shejust thought, instilled that or
kind of put that in there, but,um, she was actually the first
female photographer to work forFord Motorsports.
So I don't know if maybe shejust thought, you know, wanted
something less complex for me,or whatever, but that was my
thinking.
And so in my twenties I was,frankly, treading water with
jobs, not careers.
Um, because of that you knowmentality and um, in, uh, in the
(06:49):
year I turned 30, I took a jobwith a startup, um, and that set
me on my path on unbeknownst tome, as you know, my first day
on the job there, uh, to where Isit now.
So, um, it's, it's definitelynot, uh, a direct or even a
(07:09):
squiggly path to get here.
Michael Hartmann (07:11):
That, uh, that
you hopped across a couple of
lily pads to get there.
Andy Caron (07:15):
It feels like
several, including a very big
one to Chicago.
Michael Hartmann (07:20):
So yeah, yeah.
So before we get to that, I'mcurious, like the things you you
kind of hinted at, like thereare things that you learn from
those experiences in the theaterand with the yearbook company
that you still probably use on adaily basis in your current
role.
Like what, what are some of thethings, like lessons learned
(07:42):
from that that you still use?
Andy Caron (07:48):
things like less
like lessons learned from that
that you still use.
I think the biggest lesson fromthe costuming piece is the more
complicated and complex youmake it, the more complicated
and complex you make it foryourself later.
Michael Hartmann (08:04):
Yeah.
Andy Caron (08:05):
So the way that you
put something together so that
it can be deconstructed later isvery important in costuming,
especially when you're resizing,having to do a lot of things.
So building things in a waywhere they can be reworked when
someone changes their mind orthe role changes, as it were, et
cetera, very much applies in inmops and I think in um, the B2B
(08:31):
world in particular.
Uh, across the board, um, andfrom the yearbook publishing
side.
I think it is sort of twofoldOne anyone who has the right
(08:54):
level of curiosity and willpowercan learn to do anything.
First, I totally agree, yeah, Itotally agree, yeah.
And secondly, it was that youknow you can do any job to the
job description, but the oneswhere you actually sit down and
(09:15):
pull up a chair and scooch inand get your elbows dirty, even
if it's not your job, theoutcomes will always far exceed
what would have happened if youjust let people kind of run on
their own without the assistancethat they maybe could have used
from your perspective inside ofit.
Michael Hartmann (09:35):
Yeah, I think
that's great, I love it.
So you hinted at like youjumped a big, big lily pad jump.
Andy Caron (09:43):
You remember you
moved to chicago, jobless, right
in in 2008, of all years, whichright before the crash happened
super, super good timing.
I have exceptional timing.
Um, that's what that tells me.
Uh, yes, I did.
I I was originally fromnorthern california, had moved
to los angeles post-graduationand I was in this phase of my
(10:05):
life where I was consideringmoving back to Northern
California, but that felt likegoing backward.
I came out to Chicago on a tripand it just felt like home and
I thought you know, in my latetwenties, it's the time to do a
crazy thing.
If I'm going to do it, now'sthe time.
Crazy thing If it's.
(10:26):
If I'm going to do it, now isthe time.
And so I, I picked up and Imoved and, uh, it was a bumpy
time for finding jobs.
I, I took temp work, I didtables, I did all the things I
need to do, sometimessimultaneously.
Michael Hartmann (10:35):
All the things
you did when you were in
theater.
World right you know that too.
Andy Caron (10:39):
Yeah, exactly, Just
going back to basic.
But ultimately I land with aChicago based startup.
I was their 12th hire, wasactually brought in in a sales
role.
The entirety of my jobdescription was open enterprise
level opportunities period, andI like that Actually, I think
(11:01):
over.
Michael Hartmann (11:01):
I think a lot
of job descriptions are
overengineered these days.
Andy Caron (11:04):
Fair.
The enablement, though, waschallenging.
My very first day on the job,they gave me a computer that had
seen better days, with aspreadsheet on it where the
columns and rows did not allmatch data wise, with 300
companies that they wanted topursue, uh, and a phone, and I
(11:27):
laughed at them and told them togo get me Salesforce.
And on day two of that job, Ibecame the de facto Salesforce
admin and thus uh, thus launchedme into sales ops.
I ultimately um helped to buildout uh, out their SDR team.
They brought in email marketingand then evolved that into
(11:50):
automation and eventuallyMarketo in 2012.
And was with that organizationfor five years, during which I
was both marketing and salesadmin on both sides, until, I
think, the final year when they,when they did bring in a
Salesforce admin, uh, who, who?
(12:10):
I off-boarded you know those,those parts to it and and on the
Marketo side.
But, um, there was no uhdocumentation anywhere online on
how to do any of this.
They were Marketo user groups.
There were, um, you know, peerswithin the community and, and
you know the online help meboards and that type of thing,
(12:33):
but the the term we have today,marketing operations didn't
exist.
I had, you know, database,specialist sort of titles and
all that kind of thing duringthat tenure.
Michael Hartmann (12:48):
And it's
e-marketing.
E-marketing on your titlesomewhere.
Andy Caron (12:50):
I didn't know.
No, e-marketing, no, no, yes,yeah I.
I don't think I had a marketingtechnology title until 2018.
Michael Hartmann (13:08):
That seems
interesting.
I'm just imagining that, likethe Glenn Gary, glenn Ross scene
, right when you first walked in, right?
That's what I'm imagining.
Right, we've got you the leadsright.
The leads are good.
Andy Caron (13:23):
I was there 12th
higher, so I don't think it was
even necessarily thatcoordinated um yeah, I bet it
was a, it was a wonderful, um,sort of birthing ground for me
to have the, the support andcapacity to figure it out and
(13:44):
try it and hand raise, and Ithink that was really just
exceptional, as far as you know,as someone who is a hand raiser
, to be able to be in a spacewhere I could say I'll give it a
shot, let's see if I can do itand have leadership, support
that and then reward it when itwas successful was was, really,
was, really, really cool.
Michael Hartmann (14:02):
Yeah, that's,
that's fantastic.
You mentioned that term handraisers.
So when we talked earlier yousaid that you think curious hand
raisers tend to excel in thespace space being like ops.
So first did I get it right?
And then like, why do you thinkthat's the case?
Andy Caron (14:22):
I have found that
people who are tinkerers, people
who took apart VCRs when theywere kids, who were willing to
break a thing, to figure out howto fix a thing, tend to excel
in mobs because it's, you know,historically been kind of an
uncharted space.
An uncharted space, one whereyou have to cut your own path
(14:49):
and sort of navigate and maybeuse maps that are left, you know
, from a previous, you knowincarnation or role there or
elsewhere in the community.
But I find that hand raisersare willing to fail and when
you're trying something new, thewillingness to fall flat on
your face and learn how not todo it five, five ways first or
50 ways first is, um, is reallya boon.
Michael Hartmann (15:11):
Yeah, was it
Thomas Edison who said um, I've
never failed Like I've, I'd belike I found 500 ways not to
make a light bulb or somethinghe did with it.
Andy Caron (15:22):
I don't think he
ever actually said it yeah, okay
that, uh that quote, regardlessof who coined it yes, yeah,
right, right, right.
Michael Hartmann (15:30):
Well, it's
interesting you bring up
tinkerers, because that's a term.
I always like I'm not atinkerer but I in some ways I
think about my childhood.
I, I don't even know, I alwayssay as a tinkerer, but like I
was willing to try crazy stuffand like take things up.
Like literally, my neighbor andI we took apart two bicycles,
recreated a new one out of partsfrom both, painted it, made
(15:53):
ramps, hurt ourselves.
You know the whole bit.
Yeah, um, and we made go-kartsand you know like we did all
kinds of stuff.
I don't really do much that,but but it was like purpose,
like it was purposeful tinkering.
What I don't like is justtinkering for the sake of
tinkering, like we there weinherited and I just tinkering
for the sake of tinkering Fair.
We inherited it.
Somebody gave us an oldsailboat.
(16:15):
I didn't grow up sailing and itneeded lots of work.
And when I went out there I wasall excited.
I was like I want to sail.
It sounds great.
And then I noticed thateverybody out there was
literally out there all the time.
All they did was just tinkerwith their boats and I was like
that, like I want to enjoy theboat.
I'm willing to do that if I can.
But they didn't seem to bedoing that right.
(16:36):
They were just messing aroundso that like.
So I think on the one hand Ifeel like a tanker, but on the
other hand like I don't.
It's not just for that sake,does that?
Andy Caron (16:45):
make sense.
It does it.
Yeah, definitely makes senseyeah, but I've.
Michael Hartmann (16:50):
It's
interesting, that combination of
what you talked about from yourearly roles in that description
I also think of.
My first job out of college waswith a consulting firm Price
Warehouse and one of the thingsthey did is they sent the
consultants at that time They'dactually shortened it.
It was only three months oftraining on their methodology
and their way of doing stuff,and you had these cohorts from
(17:13):
all over the country.
Most of us were people like mewith, like, an engineering or
business background or somethinglike that in terms of our
education.
There's this one woman who,super bright um, was a religion
major and I was like how is shegoing to survive going through
this?
Because we were doingprogramming and stuff like that.
But it turns out what Irealized from that is like she
(17:35):
was curious, she was willing totry and she knew how to learn
and she, like she had a lot ofother traits that I realized
were just as valuable and itkind of made me go like, yeah, I
can probably code better andfaster than she can now, but
she's gonna figure it out yes soyeah
(17:56):
yeah, it's funny.
Um, yeah, so totally get allthat.
So, yeah, another thing that weyou and I talked about was, um,
you studied human psychology,yeah, um, and maybe this comes
from some of your work in thetheater stuff too.
I suspect there's a lot of likehuman psychology that, yeah,
and maybe this comes from someof your work in the theater
stuff too.
I suspect there's a lot of likehuman psychology that comes
into play there, but I'mprobably overgeneralizing.
(18:18):
So what do you think there areLike?
Is that a skill set?
Do you think more marketing opsfolks should be developing
Right?
Is it underrated?
What do you like?
What's your take on that?
Andy Caron (18:31):
I know a lot of
people in mops that have
psychology degrees.
Actually it's it's a, it's atrend similar to the theater
trend that has definitely beennoted on on my part and I I
think that you know, first andforemost, when a lot of us went
(18:51):
and got our degrees, there wasno mops, there was no Martech in
some cases.
So what could possibly prepareyou for this future role and
sector of the market?
And I think psychology as awhole in marketing, in business
(19:15):
operations, whether it'smarketing or sales or rev ops or
other areas of operations, inleadership all of those are so
integral to working with peopleand to figuring out how to get
people to want to work with you.
Um, and so psychology and sales, psychology and marketing I
(19:39):
think those are natural, butwhen you are architecting
systems, to react and accelerateleads toward the goal that you
want them to want to get to.
I think there's a lot ofpsychology that goes into that
and the, the if, then whatstatements all have that you
know, getting into the mindsetof the customer associated with
(20:02):
them, while also getting intothe mindset of the business and
the stakeholders and what theywant and what their initiatives
and goals and requirements fordata and insights and, you know,
roi for these initiatives are.
So I think it's all psychology.
Um, you know, I think a lot ofpeople do it without thinking
(20:24):
about it, with or without adegree.
Um, it just so happened that,you know, I actually started out
thinking I would go intotherapy, that I would become a
therapist, and when I wasgetting ready to graduate
undergrad, I remember havingthis thought of being on a sofa
(20:46):
or on an armchair, across fromsomeone on a sofa who was there
to talk to me about, you know,his middle-aged life, with his
wife not understanding him andhis kids hating him and him
hating his job and me having hadnone of those things outside of
school, just being in aposition where I completely was
unable to help this individual,and the um, uh, irresponsibility
(21:12):
of moving to that next phase ofmy education and then career
without having had some exposureto it.
And so I ultimately decidedthat maybe I'd wait until I was
like an empty nester and go backto school and that would be my,
my retirement plan to retire asa therapist or something.
Um, ultimately, I went back tograd school when I was 31.
And I actually was focused onforensic psychology at that time
(21:35):
a deep interest in psychologyand the law and sort of helping
in that arena.
And in the midst of all of that,that's when I started working
with Marketo at the startup andI it, it, it was the right spot.
I could dig in and blink andfour hours later I'd missed
(21:57):
lunch and I'd done all thesecool things and I loved it.
And so I I pivoted intopsychology of organizational
leadership at at that point and,you know, went, went forward.
But I I think about thatjourney and I think psychology
is probably a really good leadin for these, these types of of
roles, even going forward,especially with with AI and how
(22:20):
that changes the, the morals andthe thinking and the
fundamental sort of architectureof how people react and respond
to it For sure.
Michael Hartmann (22:29):
Yeah, it's
interesting Cause I think I
think if I, if, if I was to putmyself in my shoes 20 years ago,
I probably would have rolled myeyes at the idea that
psychology was an importantthing to understand in the
business world right, becauseit's like this right, like the
false impression that things areeveryone acts rationally in the
(22:50):
business world right, is justbullshit and I don't think I
realized it till later.
And now I agree with you Like Ithink the there's probably a lot
of people who are listening orwatching who might go like oh
yeah, it really is, like it'sjust silly and it's not that
important, and I would tell themto like rethink it.
(23:10):
But I think for those who areinterested, right, they go, like
you mentioned, like some peoplekind of naturally are able to
do it, but I'm a bit like and sothey go.
That's a soft skill in quotes,right and term I hate because
it's a skill, right, and somepeople like any skill, some
people are going to be moregifted naturally than others and
that.
But you can, like everyone canbecome confident, I think, if
(23:32):
they put their mind to it.
Andy Caron (23:34):
Yeah, there's a
fantastic book that was done
around deep research in whatmakes a good CEO.
It's called CEO Excellence andMcKinsey group had put that
together.
It was absolutely wonderful.
But there's a chapter sectionin there called the soft stuff
is the hard stuff, and I alwaysthink about that, right, Because
(23:55):
working with people the softskills right, that's the hard
stuff.
We're more complex than halfthe systems we work in, Right Um
, I also also always laugh aboutthe fact that I had thought
that I might be, you know, atherapist one day and eventually
found my way into um, mops andMarTech consulting and and uh,
(24:18):
essentially became a, a, a mopsor Marketo therapist, right Uh,
consulting is often sort of aform of therapy.
Michael Hartmann (24:26):
It is a form
of therapy, for sure, yeah.
Yeah, sure, yeah, yeah exactlyyeah, I mean having been a
consultant and hired consultants.
I think sometimes, sometimesit's easier to be honest with
the consultant than it is withpeople internal to the company,
which is a hard thing to say sayit straight to your folks, when
(24:47):
maybe when you say it itdoesn't quite translate, or
register or hit yeah for sure.
Yeah, so it's interesting.
I think I told you that one ofour early podcast episodes we
had Brandy Sanders on and shementioned psychology and chess
right Understanding chess alsois like that yes.
Andy Caron (25:06):
Four plus years
later, I still think of that on
a daily basis, probably at leasta weekly basis, like how
important the idea of likeunderstanding how things are
connected, thinking ahead, andthen the human psychology part
of it yes, I think that playingforward into possible paths and
actions and those, you know,checkmates from your opponents,
(25:29):
aka systems or processes ordepartments on, well, this won't
work because of this and thisway I could get here, but if
this happens then it's a failure, and that logic forward almost
like a la three-dimensionalchess from star trek is very
much part of this role.
Michael Hartmann (25:47):
Um, absolutely
, both systems and also the
organizational side of it yeah,totally yeah, I think I still
love that, that idea, um, butlet's maybe shift gears a little
bit.
So something you want we weregoing to talk about was, um, you
know, the idea to talk aboutattribution everyone's favorite,
uh, subject to love or hate,right and when, and when we were
(26:11):
talking, I thought it wasinteresting that somehow we
landed on the Hitchhiker's Guideto the Galaxy.
Yeah, and how?
The answer to the question,right is a great analogy for the
trap that we could fall intoabout being obsessed with data.
So you want to walk throughwhat kind of the thought process
there?
Andy Caron (26:31):
Yeah, so I struck on
this a couple of years ago and
I actually got the opportunityto present on this at the first
year from Upsalpalooza, whichwas really, really fun.
I walked away with a 42 tattooa la the tattoo booth as well
sort of a fun remembrance ofthat, but it's interesting.
(26:53):
So he's a favorite author ofmine and and I love you know,
sci-fi and and fantasy and andanything that marries that
together with insights onhumanity, and in particular I'm
I'm always drawn to and DouglasAdams just did such a phenomenal
job of that.
But within his first book, inthe Hitchhiker's Guide to the
(27:14):
Galaxy and the subsequent ones,that you find out the story of
this sort of super beings thatwanted to know the meaning of
life, the universe andeverything.
And they built a massivecomputer to get the answer to
the meaning of life, theuniverse and everything.
And this thing processed forgenerations and when it was
(27:38):
finally times like a mediacircus and they built up so much
around it within their society.
And the computer powers itsscreen on and comes up and says
the answer to the meaning oflife.
It screen on and comes up andsays the answer to the meaning
of life, the universe andeverything is 42 and you know
you can envision in the bookkind of not just being able to
hear a pin drop with like the um, and ultimately the computer
(28:03):
indicates that 42 is the answer.
But they need the question forthe answer to make sense and
that it's not smart enough togive them the question and they
have to build a bigger, smarter,much more complex, much longer
running uh computer and systemto give them the question so
that then the answer will makesense.
(28:23):
And in pure British comedy,douglas Adams style, the
computers destroyed momentsbefore it's set to spit out the
question, much to no one'ssurprise, if you, if you're
expecting it, which I think mostpeople are.
But the parable of that is justis fascinating to me as someone
(28:47):
who has, you know, implementedand run many attribution systems
and processes in previousorganizations and has done so,
you know, as a consultant, many,many times, the politics
surrounding it and sort of thethinking and process of we have
to build this very complexsystem to get at this massive
(29:10):
question right.
And then we get an answer andit's too complex and it doesn't
make sense because we're notspeaking the computer's language
and the computer isn't speakingour language and instead of
figuring out how to translate orto ask better questions, we
build bigger, more complexsystems to then tell us more
(29:32):
stuff, which then ultimatelycrashes, and, in the interim, no
progress or optimization to thebusiness is being done or made
on a data based level.
And the irony in all of thisbeing that Douglas Adams was a
huge technophile and one of theearlier coding languages the one
that purportedly he favored 42,was a keystroke for an asterisk
(29:56):
which, if you work in data,asterisk is a placeholder.
It's literally, in my mind,translates to whatever you need
it to mean for you.
And if what the computer wassaying was the meaning of life,
the universe and everything isliterally whatever you need it
to mean for you, that is quite adeep, profound answer.
(30:24):
Absolutely yeah, withattribution, in particular,
around asking huge questions notunderstanding the data that
comes back and then adding inmore complexity instead of going
backward, adding specificity orworking on better understanding
the data or getting the data totranslate into more common
language that they canunderstand, and it creates this
(30:47):
feedback loop of complexity andlack of faith and more
complexity and lack of faiththat ultimately undermines and
defeats any initiative aroundwhat ultimately should be budget
optimization and allocation ofspend in a more data driven way
into a political credit or, youknow, decrediting of the whole
(31:16):
idea of attribution as a whole,even the inverse, to the end
that you know a lot of timesit's after much blood, sweat,
tears and dollars it'sjettisoned.
Michael Hartmann (31:28):
Yeah, it's
interesting to me because I
think I was an early fan of theidea of attribution, because I
truly believe, with all thisdata, we should be able to get a
better insight into what'sworking and not working right.
Where should we put our nextdollar in marketing investment?
I wouldn't say I've gonecompletely the other way.
I think it still has a place,but I basically stopped
(31:50):
interacting with the debates toget online about, like, which
model to use, because I don'tthink it really matters that
much, right?
My general take is pick a model, stick with it, look for trends
.
Don't expect it to be theanswer.
Right, which is your point.
Andy Caron (32:03):
I'm on the opposite
end of that, which is my
favorite question anyone everasks me, is what model should I
use?
Because it inherently is aflawed question.
That shows the psychology ofthe thinking, which is there are
a slew of models related toattribution, because each one
(32:24):
answers a different question andso the answer is all of them to
answer the different questionsand report against the different
OKRs and KPIs that you arebenchmarking for and in how
you're allocating budget.
If you're allocating budgetprimarily to acquire, then
(32:45):
you're going to look at your topof funnel, you're going to look
at a U U shaped model what haveyou and go from there.
Right, I think people get stuckin building a perfect model or
building one model.
If there was a one model to winthem all, we would just have
one model.
And it gets tricky where people,especially, I think, conflate
(33:06):
building a model with the models, and so they'll think like
we're just going to build amodel.
Especially, I think, conflatebuilding a model with the models
, and so they'll think likewe're just going to build a
model, as opposed to we're goingto do different layers of data
analysis from statistical modelswhich is what they really are
at the end of the day againstthe larger data set.
I think people also get stuckin this idea of having to have
(33:27):
100% of the data instead ofbeing at statistical
significance.
If you talk to someone who youknow has studied and worked
within math, especially highermaths, you know 5% is
statistically significant, right?
Yeah, of course you know ifwe're trying to get to 80, 90,
100% data completeness or parity, like it's a fool's errand data
(33:53):
completeness or parity, likeit's a fool's errand.
Michael Hartmann (33:55):
It's totally
so.
I I tell I say this all thetime that people are looking for
they.
They want to eliminateambiguity and I think, just by
the nature of this data and thelack of controls around it, like
you're never going to get tothat.
But I think you're hitting onanother thing.
So my training is in operationsresearch.
My college degree is, whichpart of that is linear
(34:15):
programming, and so the linearprogramming is guided towards.
Give me a real life example ofwhat people who I never
practiced it, but the idea islike one of the applications
right now that I know of isbeing used by, like professional
sports leagues to come up withschedules for their teams that
(34:36):
have a lot of constraints andrequirements about.
You know when a facility isavailable, how often can teams
play, how far can they travel,you know what other events, and
so there's there's not.
This gets to the point likethere's not an answer.
Right, there are probablynearly an infinite number of
potential answers, and so youcan optimize for one or a
(35:02):
handful of them or some basketof them, but you've got like
this is this difference we hadanother guest on.
We talked about about.
He told me about the kenevanframework, um, which is kind of
talks about like there'sdifferent kinds of questions and
problems, but there's this um,the difference between complex
and um.
Um, I'm going to play anyway.
(35:25):
But like, complex, is, uh, amodern race car, right, where,
like, there's an answer.
Like there's a modern race carright when, like there's an
answer.
Like there's a problem, there'san answer If you know enough
about it.
Complicated is right.
This is the other one.
Is, uh, you really don't knowthe answer, right, you can take
a guess, you can take an action,and I think people think that
they have a complex problemright where there is an answer,
(35:46):
when really it's a complicatedone where there is not a single
answer.
So the best you can do is tryto make a step in the right
direction, learn from it, adjustright.
Yes, and I think so.
When I think about attribution,it is one of many potential
things that you could bemeasuring I would tend to lean
towards the things that I know Ihave more control over.
Did my email get opened anddelivered?
(36:10):
There's stuff that people aremoving away from in a lot of
cases To me.
I'm thinking about the fulljourney, all these places where
things can break down and youcan prove them.
Andy Caron (36:19):
Bot activity and
other things have created such
false positives around some ofthose things that I have seen a
trend to move away from one ofsome of the like more almost
vanity metrics, if you will.
But I think the biggest thingthat people also struggle with
in that space and thatdiscipline is causation versus
correlation right.
Like.
I saw this fantastic video thatwas comparing the rise in ice
(36:44):
cream sales, in a nearly perfectchart line, with shark attacks.
And so there's correlation,right, because as it gets warmer
, people buy more ice cream.
People go more in the water.
Not a shocker when you thinkabout it.
It gets warmer, people buy moreice cream, people go more in
(37:04):
the water Not a shocker when youthink about it.
But someone looking at thatchart could potentially if they
were passing glance at it ormaybe didn't stop to look at
what the metrics really werecreate a causational.
Oh well, when people eat moreice cream, sharks bite them,
right, like kind of weird.
But I think and that's aridiculous example, but I think
it illustrates that causationcorrelation that people can get
(37:28):
stuck between, and also the factthat they want causation when
sometimes the best you can getis correlation, but you have a
better, um, you know, uh datametric against.
Like people are more likely toget bit by sharks when they are
spending the day at the beach,right, that arc of beach
(37:48):
visitors versus shark bites Likethat's.
That's a much more parallel andcausational metric, potentially
, so um we just had a guest.
Michael Hartmann (37:58):
We just had a
guest on.
Andy Caron (37:59):
We just talked about
this like correlations, not
equal causation but correlationis correlation right so there
may be something you can inferor learn from it, but not
necessarily and if we move thislever does this happen.
(38:19):
Okay, so now we can actuallyshow causation and not just
correlation, and that gives usthe methodology to then, you
know, pump the brakes on thatspend if it's not performing, or
to increase, right, like.
I think that's where it goes toa scientific methodology of
we're going to test a hypothesisand see what happens and then
use that to then go from therewith the proof points to then
(38:42):
modify or optimize, you knowbudget, until we get to a point
of, you know, either diminishingreturns or kind of if maxed out
, and even if they diminish,it's still better than what we
had.
So let's do more, you know yeah, yeah, so it's interesting.
Michael Hartmann (38:55):
So the I
thought the previous guest that
I talked about.
One of the things that we weretalking about is AI-driven
models for some stuff and howthey can generate things that
look like they're correlated andpeople that just don't
necessarily make sense.
One example that he used wasall the people who returned a
(39:19):
product were the ones who boughtright, which is, of course,
right.
So this gets into the potentialof AI, right?
So I'm curious about whatyou're seeing in terms of AI
impacting the role, but alsomaybe this specific one where,
like I've said for a while, Iactually am really excited about
the potential of AI to uncoverthings that don't require a
(39:43):
hypothesis necessarily to goinvestigate, especially a large
data set.
It's just you have to narrowdown what you're going to do if
you're going to use thatscientific method so they might
be able to churn through allthis data and look for things
that might be related, but youneed a human, like I might
believe is like, until furtherevidence.
Right, you still need a human.
(40:03):
Like my belief is, like, untilfurther evidence.
Andy Caron (40:04):
Right, you still
need a human to interpret that I
for a long time now have saidand I will hold to this that
most LLMs that I have playedwith or use regularly sit
somewhere in the very eager andfairly smart intern position.
Michael Hartmann (40:26):
Yeah, it's not
a bad analogy.
Andy Caron (40:29):
They're not ready to
be hired on full time to take
on one of the paid roles, butthey can certainly contribute
and participate and assist.
But oftentimes you'll ask themto do something and they'll come
back and you have to correct itand then correct it and correct
it to get it the direction thatyou, that you need it to go
(40:49):
into.
I hold and will continue tohold the line that I think this
sort of pervasive, underlyingfear that a lot of professionals
have that they'll be replacedby AI is, in the short and
midterm, not realistic for tworeasons.
One, we're not there yet withAI.
(41:13):
I mean, how often do you gointo GPT and ask it for
something and it gives youmisinformation and when you call
it out on it, it goes oops,sorry, my bad, you're right.
And when you call it out on it,it goes oops, sorry, my bad,
you're right.
So the fallibility of that andthe capacity for it to to give
them if we're really focusing ona deep T aspect of of that but
(41:52):
I don't think that anyone's jobis is going to be replaced by AI
for the most part in the nearfuture, other than some of what
we see with like automation atcheckout stands, as opposed to
having cashiers and that type ofthing, although we've seen a
resurgence back toward cashiersagain.
So I think you know it will bea progress that we'll see over
(42:14):
time.
I do think people will bereplaced by people that are AI
activated and AI capable andhave the capacity and know how
to work with and activate AIwithin the role.
I think that will happen.
The other thing is just a ROIfactor.
(42:36):
There's a absolutely fantasticbook called A World Without Work
.
It was actually written in 2019.
Daniel Suskin is the authorwell ahead of his time
fascinating mind, and what hesaid you know literally more
than five years ago, well beforethe emergence of everyday AI
(43:00):
utilization was that, yes, hedoes think that eventually we'll
get to a place where, you know,society doesn't necessarily
have to work because AI can dothe bulk of it, and that comes
with its own challenges andinteresting philosophical
contemplations.
But what he said, which wasreally interesting, was that
(43:23):
early postulization was that youknow, the low level, kind of
grunt work would be what wasreplaced by machines and AI, but
he didn't think that that wouldbe what would happen, because
the cost efficiency isn't there.
There are things that humanswill do, at whatever rate that
(43:48):
you cannot build a robot to doat the same cost efficiency.
I've got an example Right, andso if that's the case, then AI
becomes inefficient andtherefore doesn't make sense.
Michael Hartmann (44:04):
Yeah, so the
example, just if you're curious.
So I'd like to say I'm a flyfisherman but like I haven't
been in a couple of years.
But one thing I learned is thatso if you've ever gone fly
fishing and you had to buy flieslike they're expensive for kind
of what you get from them, butat least they feel like it, and
(44:27):
I don't know if I have asked,have asked.
It's like like where are theymade?
And like I assumed that theywere, there was some sort of
like factory?
well, there is, but it's allpeople.
So apparently the process ofdoing that is doing well, it's
all.
Yeah, they're all hand like andso it's very human like it.
Just I guess they have notfigured out, maybe that was.
It's been probably 10 yearssince I asked that kind of
question, but yeah, so I thinkyou're right.
I mean, there are things outthere that just don't like,
(44:48):
either don't make sense from,like you said, an ROI standpoint
to automate, or that are just,you know, whatever the dexterity
capabilities of the machines isnot there.
Andy Caron (45:00):
Yeah, exactly so, um
, you know all of that said, uh,
rp is has been an interestingplace over the last several
years.
In, uh, spring of 2023, we satdown and and made the decision
to do AI training and enablementacross the entirety of our team
(45:24):
everybody, hr, finance sales,all of it and, of course, across
our consultative team, becausewe knew that was coming.
And we have continued androlled out new training and
enablement over the last twoyears to not only keep pace but
to, you know, be ahead of ofthat curve.
And we have been doing a lot ofAI automation and augmentation
(45:49):
for for clients and you know areare coming to the market with
an, an agent that you know is amops cops, um sort of partner in
crime, if you will.
I always say he's not anautopilot, he's not a co-pilot,
he's a wingman, because we'restill not at a place where I
want AI QA-ing things or, um,you know, without a layer of
(46:13):
humanity in between, to checkand make sure that everything
looks good, because errors canhappen with humans, errors can
happen with AI, but you know thelevel of of error ratio we have
a 99.9% confidence at thispoint with him um is low.
It's probably less than folks.
I've had working inside of myinstances over the years, but,
(46:34):
um, what I'm seeing with AI isthe capacity not only to train
and create agents that truly arevery specialized in a
particular area, but toorchestrate multiple agents into
processes and to then automatethose in with MCP models where,
(47:03):
you know, this one goes and doesthis, this one goes and calls
that API because it knows how todo that process, and so on and
so forth.
That scale is just superexciting and really, really cool
and, um, definitely moving inthe direction, um, that I think
all of us kind of hoped that wecould go with AI, which is there
(47:24):
was someone, oh gosh, about ayear and a half ago and I pardon
me, I don't remember her name,but she, she.
I recall she said she was sodisappointed with AI because she
thought AI would give her, aiwould give her more time to
write and create music and takecare of the dishes and the
(47:47):
laundry, and instead it'swriting and creating music and
giving her more time to dodishes.
Yeah, and I think this type ofthing is that scenario where it
clears, you know, has thepotential to, and can actively
(48:07):
clear and clutter away thelaundry and the dishes and the
day to day sort of repetitivetasks that you don't want to be
doing, so you can get to thatcorner of desk stuff, so you can
get to the strategic work, soyou're not like just you know,
all day long a task taker that's, that's dispositioning stuff
for other people, um and insteadcan get to the the more fun,
(48:30):
more interesting, more strategicwork.
And so I think that emergentpiece there is, um is really
(48:56):
exciting and excellent andbecoming part of the tech stack
as a much larger piece of theoverall deployments that we
would see in the scale of thatthat he was predicting was both
somewhat humbling and mindparadigm shift from, you know,
homegrown tech versus going outand buying a SaaS platform.
(49:28):
Like the SaaS platform is.
Anytime someone says it's anin-house developed platform or
tech, you kind of cringe alittle bit.
You're like, oh no, what's itgoing to do?
What doesn't it do?
But this idea of you know,developed in-house AI agents
that have very specific rolesand processes I think we'll see
a rise of that over AI-poweredenabled tech and I love that.
(49:53):
I think it's really cool, it'sreally smart.
Michael Hartmann (49:57):
Yeah, I'm
seeing that actually already.
So, yeah, I think it's going tobe fascinating.
I think the short version ofwhat I think is like the people
who are not learning how to useAI as a tool to help them in
their jobs are the ones who aregoing to be in trouble.
I think it's, but it is a toolright.
It's just like any other newtechnology.
It's got our phones right.
(50:18):
They all potentially causedisruptions, and so we need to
learn to adapt and takeadvantage of it I think in the
workspace it's, it's awesome, it.
Andy Caron (50:28):
It scares me a
little bit on the, the social
side of things, to be honest.
I think it goes down a wholedifferent rabbit hole there.
But for for, from a work sideof things, I think it's an
accelerator.
It's, it's so exciting, it'sreally, really cool and, um, you
know, the, the, just the, thevelocity of innovation and
(50:49):
discovery around it has justbeen so cool.
Michael Hartmann (50:52):
Yeah, it's
mind blowing.
Um, I wish we could go on.
This has been a blast.
Um, now maybe we'll just callthis episode the answer is 42
and see if people can figure outwhat it is.
That's a blast, Andy.
Thank you so much.
If folks want to kind ofconnect with you, learn more,
maybe just brainstorm on thiskind of stuff with you.
(51:14):
What's the best way for them todo that?
Andy Caron (51:16):
You can find me at
LinkedIn.
Easy peasy, just Andy Caron.
You can send me a quick email,andy, at Revenuepulsecom, always
looking forward to connectingwith people.
And, of course, I am on theSlack community as well, so feel
free to look me up there.
Michael Hartmann (51:31):
Fantastic.
Well, thank you again, andy.
Thanks to our listeners andsupporters, we always appreciate
you.
If you have ideas for topics orguests or want to be a guest,
reach out to Naomi, mike or meand we'd be happy to talk to you
about that.
Until next time, bye everybody.