September 1, 2024 • 28 mins
In this episode, Quentin Reil (Founder & Chief AI Officer) and Andreas Welsch discuss the four key challenges of Generative AI that you can't ignore. Quentin shares his story about leveraging business data as a differentiator and provides valuable advice for listeners looking to overcome the most common issues surrounding Generative AI applications.
Key topics:
- Describe the impact of the evolving Generative AI landscape on business strategies
- Assess the “fitness” of different Large Language Models to solve particular customer needs
- Select technologies that can address the challenges of Generative AI
- Understand the role of data in the current regulatory landscape
Listen to the full episode to hear how you can:
- Concentrate on the problem rather than the technology
- Leverage your business data as a differentiator of your AI applications
- Drive business outcomes based on data-centric products and applications
Watch this episode on YouTube:
https://youtu.be/GKQUyiekGAM
Questions or suggestions? Send me a Text Message.
***********
Disclaimer: Views are the participants’ own and do not represent those of any participant’s past, present, or future employers. Participation in this event is independent of any potential business relationship (past, present, or future) between the participants or between their employers.
Level up your AI Leadership game with the AI Leadership Handbook:
https://www.aileadershiphandbook.com
More details:
https://www.intelligence-briefing.com
All episodes:
https://www.intelligence-briefing.com/podcast
Get a weekly thought-provoking post in your inbox:
https://www.intelligence-briefing.com/newsletter
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Andreas Welsch (00:00):
Today we'll talk about the four generative AI
challenges that you cannotignore.
And who better to talk about itthan someone who's passionately
sharing advice on them.
Quentin Reul.
Hey Quentin, thank you so muchfor joining.
Quentin Reul (00:13):
Hi, thank you for having me today.
Andreas Welsch (00:16):
Wonderful.
Why don't you tell our audiencea little bit about yourself, who
you are, and what you do.
Quentin Reul (00:21):
Yeah, so my name is Quentin Reul.
I'm originally from Belgium.
Spent my bachelor degree and myPhD in the UK, and I've now been
in Chicago in the US for 10years.
I from a, on the side of work, Ido martial arts with my kids and
my wife.
(00:41):
And it's a lot of fun.
From a work perspective I workedat Fortune 500 before, and I'm
currently looking at helpingcompanies with their journey
into the AI strategies byadopting UX but also the
technology and the deepknowledge in technology in doing
so.
Andreas Welsch (01:02):
That's awesome.
Again, thank you so much forbeing on the show today.
And I know you've been sharing alot of your advice and
experiences on social medialately.
Should we play a little game tokick things off?
Wonderful.
Hang on.
Let's see.
This game is called In Your OwnWords, and when I hit the
(01:22):
buzzer, the wheels will startspinning.
When they stop, you'll see asentence, and would love for you
to answer with the first thingthat comes to mind, and why.
In Your Own Words.
To make it a little moreinteresting, you only have 60
seconds for your answer.
And folks, for those of you inthe audience, you can
participate too.
Put your answer and why in thechat as well.
Quentin, are you ready forwhat's the buzz?
(01:45):
Okay, here we go.
Almost.
Here we go.
It is a live show.
If AI were a fruit, What wouldit be?
60 seconds on the clock, and go.
I think
Quentin Reul (02:00):
I would be an apple, in as much as you have a
lot of layers, and, It tastesgood if you eat the skins, but
if you remove the skin and thelayers and you try to dig deeper
and eat the apple, the more youknow and the better you can
leverage it.
Andreas Welsch (02:17):
Wonderful.
And I think there's a love thereat the core as well that you
need to know about and how toget close to it.
Wonderful.
And
Quentin Reul (02:27):
it grows again if you use the
Andreas Welsch (02:28):
seeds, so it's regenerative.
Haha, perfect! I love thatanswer.
Excellent.
And I see comments here fromAshwin, he's joining us from
Hyderabad, and he said, yeah,something sweet.
Exactly.
I think that's where we're stillat in that phase where it's
sweet and we're figuring outwhat else can we do with it?
Can we make apple pie and applesauce and apple cider and
(02:50):
whatever else?
So very versatile too.
Okay.
So with that out of the way whydon't we talk about the topic of
the show?
The four generative AIchallenges that, that you need
to know about, and if you lookat the, history the, last what,
two and a half years, ever sinceChatGPT has come on the map, I
(03:11):
think there's, been not only somuch hype, but also a already so
much maturity, so much evolutionthat I'm curious what are you
seeing?
How is generative AI evolvingthat the business strategy and
the impact on businessstrategies?
Quentin Reul (03:28):
Yeah, I think if we look at what happened
November 2022 to maybe July 2023a lot of companies are at the
foremost the fear of missing outon leveraging generative AI, and
they were looking at a lot ofdifferent use cases and what
they could do it, what advantageand what they what gap that was
(03:48):
filling compared to moretraditional machine learning out
of the box, you had thousands ofdata points that can you now use
and model that were pre trainedfor problems that were complex
compared to before where youneeded to spend years creating
the training sets to, togenerate that.
(04:08):
But I think now, there is arealization that maybe The
direction that the big LLMproviders like OpenAI or
Anthropic and where thecompanies are going is not
necessarily in the samedirection.
In as much as OpenAI ispromoting artificial general
intelligence and I think thatfor the businesses, where their
(04:32):
value is more in narrow AI.
It is not They will gain valuein content creation and
marketing by leveraging AGI.
But it's where they are veryspecific to their niche that
really the differentiation isgoing to come.
So I think that there we'regoing to start seeing a
(04:52):
realization that one, not onlyAll use cases that you add could
be fulfilled with the largelanguage models.
If you want to do the predictionof what to put on your shelf
tomorrow, LLM is not going tohelp you.
But if you are, as I say, liketrying to create content, or
you're trying like to answerquestions, it's there and it can
(05:15):
help you.
Andreas Welsch (05:16):
See, that's the part where I'm always curious to
to, see The two be combined.
We, we know that LLMshallucinate, they have factual
inaccuracies, they have somebiases they, are not even able
to do certain tasks that narrowAI and even predictive analytics
and machine learning can do.
I feel if you combine the two orif you create a prompt where you
(05:38):
substitute some information andsome variables and you put in
those data points that you havegenerated using other proven
technologies and methods, thenit's a, good vehicle, right?
To leverage the best of bothworlds.
What are you seeing there?
Quentin Reul (05:54):
I think that like what I was working on there,
Fortune 500 and we were workingwith a lot of different models
and GPT 4 and there was, to behonest, not a lot of use cases
for which we needed somethingdifferent than GPT 4 because
they weren't generic enough.
(06:14):
Now, I think that once you getto the point that you just
mentioned, which I tend to callor refer to as a cost to be
wrong.
So the higher the cost to bewrong, the higher you have a
risk of hallucination.
If your cost to be wrong is nilor zero, I call it creativity.
And I think that's another thingthat business need to think
about is that there are plentyof use cases where the
(06:37):
creativity is important andactually useful.
Like I have the blank pagesyndrome.
So whenever I try to write and Iwrote a PhD and it took me a
long time and if I had agenerative AI, it would have
been much faster.
I get stuck on the page and Iwant to rewrite, everything.
The LLMs are very good for thatproblem.
(06:59):
But if I try to make a judgment,let's say, predict or give a
treatment to a patient orpredict how to apply law for
particular problems.
I can't be wrong.
I can cause someone's going tojail or I can cause someone's
death.
So I think that's where we needto, look at the problem at a
(07:21):
different scale.
I think that what is being donewith the AU Act, where they have
that different level of risk forassessment.
And more recently, I saw thatthe IEEE is actually following
that same methodology in termsof accrediting whether your
solution is going to be ethical.
And they look at it from theperspective of low risk, medium
(07:43):
risk, High risk and unacceptablerisk and things that are
considered unacceptable risk arethings like social scoring where
for example You would be in hrand you would use whether people
are attending meetings orwhether they're sending emails
are the way to determine Whetheror not they are predictive and
whether they should keep or stayin the company or not
Andreas Welsch (08:05):
now, I think those are really important
aspects in coming back to thatchallenge, right?
It's first of all understandingwhat can you use the technology
for and what can you maybe useother technologies for that are
better suited, that are maybecheaper, that are maybe more
cost effective to operate and tobuild.
Certainly everything has a tradeoff, but again, you don't need
(08:27):
to have a large language modelfor So many use cases.
Now I'm curious folks for thoseof you in the audience, if you
have a question for Quentin orI, please put it in the chat.
We'll take a look in a minute ortwo and pick up those questions.
Quentin Reul (08:44):
And I think to the last point that you made,
Andreas, with regard to notevery problem is a Generative AI
problem.
I think that's a responsibilityof technology leader.
And to have a deep enoughunderstanding of the technology
to be able to advise thebusiness and their partners
that, yes, you could useGenerative AI, or you could put
(09:05):
an AI label, but that doesn'tmean that you're going to have a
better product for yourcustomers.
Because at the end of the day,as a business, how are you
making money?
What is your return oninvestment?
It is about delighting yourcustomers by providing a
solution that is intuitive.
Integrating AI makes it morecomplicated for the solution or
(09:25):
the problem or the process thatpeople are going through.
Or you're not really helpingyour customers and you're not
necessarily going to get thereturn that you're looking for.
Andreas Welsch (09:35):
Now, you've mentioned you've worked at
Fortune 500.
I did a bit of work in previousroles in corporate with Fortune
500 as well.
And I heard leaders say justrecently at the beginning of the
year, if it's not Generative AI,it's not AI; or we're not
pursuing it.
And I think that's misleading inso many ways.
(09:55):
And it's challenging.
It's troublesome.
To your point you need to knowwhen you need to use a large
language model, when you useyour logistic regression or
other capabilities.
I'm wondering there, from yourexperience, how can you assess
the fitness, if you will, ofdifferent LLMs, of different
(10:15):
approaches, if you want to solvea particular need, so you don't
run into the trap of if it's notGen AI, we're not doing it?
Quentin Reul (10:22):
Yeah, I think there's been a lot of work that
has been done in evaluation andbenchmarks.
Thanks.
And I think they're providinggood insights as to whether or
not a model is better thananother model.
But I think that we have to bevery careful when it comes to be
looking at benchmark, is thatthey're not designed for your
(10:44):
narrow AI problem.
They are designed for verygeneric problem.
The coding problem is looking atdifferent language, but it's
probably not very extensive onSQL.
It's also somewhat misleadingbecause we have seen some
companies being using part ofthe benchmark as their training
set and that's cause overfillingbecause it does what it says
(11:06):
like it's going to do becauseyou train it on.
So I think that's definitely oneaspect.
It's like you can use abenchmark as a pre selection for
what you're going to look basedon the type of problem that you
have.
But I think after that youreally have to test it.
You have the chatbot arena whereyou can put two different LLMs
side by side and you have yourprompt and you see which one is
(11:27):
going to give the best results.
And that is, it's an interestingproblem if you are creative.
But I think if you are a companyyou really have to invest on a
golden set.
So you probably don't need asmany data points as you needed
before.
Like a good training set or agood golden set of 500 data
points is probably going to besufficient, but that is going to
(11:52):
give you an ability to test likethe different models over time
not only like the first time asyou're creating, but a lot of
these models are evolving veryrapidly and because they're
offered as SaaS model they gotoday, tomorrow like the new
version is the old version isgone.
And there is an opaque problemin as much as you don't know
(12:14):
what training data is going in.
So you don't know whether thelength of fitness of your LLM
before and when you do it likeyour previous assessment is
going to remain as you're doinglike your assessment.
So monitoring and having thatgolden set for ongoing
monitoring is very important.
Andreas Welsch (12:30):
That's an important point that you're
bringing up.
And I feel that hasn't beengetting as much attention lately
as it has been probably a yearago, right?
As as models change even if yourAPI stays the same, but if the
LLM underneath changes from GPT3 to 3.5 to 4.0, to whatever is
(12:52):
next, you need to do yourregression testing again.
You need to test if the sameprompts work.
We know that between 3.5 and 4,there are differences, right?
Differences in the creativityand other aspects of these
models and underneath.
So you need to build that intoyour plan as well, and you need
to make sure that you haveresources and you have budget to
(13:15):
do all these changes.
And especially if vendors aredeprecating models that you can
react fast enough to put in thenew ones.
Quentin Reul (13:24):
And it's very much trial and errors.
At the end of the day, to giveyou an example, as I mentioned,
I have the white page syndromeand I started creating an
application to create blogs.
Because most of the solutionsthat exist can get you
somewhere, but it was not givingme the solution that I wanted.
(13:45):
And I used Gemini.
And I was writing my prompt andmy goal was to create blogs that
were of a certain length, eitherin time or in number of words.
And when you do LLMs and itfails the first time, what do
you do?
(14:05):
You go back and you change yourprompt, because it's the
cheapest thing you can do.
And you tell the LLM to actuallyrewrite the prompt in a way that
it would understand it himself.
So it will provide more context,it will write the information in
the right, with the right levelof instructions and so forth.
But you get to do that four,four or five times, and you
realize that it's not the promptthe problem.
(14:26):
And in the particular case, Idid some digging, and what I
realized after a while is thatthere was no training data about
length.
And therefore, no understandingof that.
It could predict length in thenext token, but because there
wasn't a notion of length aspart of the training data, it
was not able to follow myinstruction to the end.
(14:49):
I think that's where you reallyneed to test.
And frameworks like LangChainare very useful for that because
it provides you with an easy wayto integrate with different
providers in a quick way.
About a year and a half ago, ifyou were on Azure, the only
model that you had were theOpenAI models.
That has changed now, with theAzure AI Studio, where you can
(15:13):
have access to Llama, and chooseevery integration through
HuggingFace.
But, before that, you were stuckwith one thing.
So at least now you have a bitmore of that connection, but I
think that something likeLangChain, definitely the PoC or
the early analysis stage is thebest thing like to use.
(15:33):
I wouldn't necessarily use thatframework in production.
Because it's very heavy andleverage a lot of other
libraries.
But in terms of doing thatresearch aspect, like what we
used to call ADA in the moretraditional machine learning if
you do that with something likeLangChain, then you're going to
get faster results and you aregoing to be able to do that
(15:54):
comparison much faster as well.
Andreas Welsch (15:56):
Awesome.
And just looking at the chathere somebody saying if I've
asked it one time, if it cancount, smiley face, it can't.
I'm sure it thought it could.
But we've seen those examples.
It's a good reminder as well.
Now, we've already talked aboutthis notion of don't give in too
(16:19):
much into this hype ofeverything should be Generative
AI.
We talked about, hey, there'ssome evaluation frameworks,
methodologies.
But what are some of thetechnologies that you're seeing
that can address thosechallenges of Generative AI?
Quentin Reul (16:35):
I think, LangChain is definitely one that can help
you with that in personintegration.
I think it's so difficult todayto stay up to date with all the
tools.
Every day, like I'm on a TLDRgetting my new specs on what is
happening in AI.
Thanks.
And many other newsletter, andthere's just no way you can keep
(16:58):
up with all the tools.
You have Llama File, you have alot of things that are
happening.
We spoke about different aspect,but completion as opposed to
chat.
That also has totally differentaspect.
I was for example, I was askinga normal LLM, I think it was the
Llama to write a blog and I'dgiven the instruction and the
(17:24):
Llama model just gave me backthe instruction.
But then I used like the Llamainstruct and I provided like the
same prompt through the Llamainstruct and it created like the
blog that I wanted.
So it's again like with thesetrial and error that you will
find like some of these.
Aspect of it and be able toaddress it as you go forward.
Andreas Welsch (17:45):
I think that's a really important point as well,
right?
Do your evaluation, especiallynow that there is so much choice
and it seems like it only keepsgetting bigger and more complex,
what do you use for?
What do your evaluation and seewhat type of tasks do you have,
what model performs or evenbetter than others.
(18:05):
There's one question fromJennifer here in the chat and
she says, Hey you mentioned thatLangChain is heavy and not
suitable in production.
Can you share more about this?
Or maybe qualify what you meantwhen it comes to lang chain and
similar tools.
Quentin Reul (18:21):
Yeah, so LangChain, because it was
designed like as a community andtrying to address all of the
different LLMs, it bringsTensorFlow, it brings PyTorch.
As you're creating an image,let's say a Docker image based
on LangChain, it's going to bebloated.
I think to recall it wassomething like 6GB.
(18:42):
And my prompt and my code wasmaybe about like 200k.
And it has an implication on howyou are going to productionalize
it.
A lot of the use cases that Isee are still very much on
demand it's not that you have alot of use cases that are flawed
where your LLM is going to run24/7 and processing content all
(19:06):
the time.
One way that you're going to useto minimize your cost and be
able to scale is something likeLambda or serverless, depending
on whatever infrastructure youare.
But there are limitations.
For example, Lambda doesn'tsupport Docker images to be more
than a certain amount of memory.
So now you have to create layerson top.
(19:28):
So you are adding complexity byusing something like LangChain.
That has got a lot of thingsthat may be useful, but if
you're doing OpenAI you don'tneed to have access to Oracle
Cloud Infrastructure.
You don't need access to Bedrockbecause you're not going to use
any of these components.
(19:48):
Whereas if you were usingdirectly the OpenAI or the
Microsoft Azure API it's goingto be much leaner because it's
only going to have what isnecessary to do the job.
Andreas Welsch (19:59):
Wonderful.
Thank you for summarizing that.
I think that was really tangibleand some good advice there.
What are some of thecomplexities that you inherit
when you go down a certain path?
You also mentioned that youdon't have like a continuous
loop where it's not always on,depending on the use cases that
(20:20):
you see, especially inenterprise.
To me, that comes back to data,to pipelines, these kinds of
topics.
And you alluded to that earlier.
Yes, large language models helpus get to production faster.
We might not necessarily need tohave a data scientist or be a
data scientist to get somethingout there, at least something
(20:42):
that's good enough.
I'm curious, what are youseeing?
What's the role of data stillwhen it comes to LLMs in that
current landscape especiallylooking at regulation at the
beginning, you talked about theEU AI Act and IEEE, what role
does data play there?
Quentin Reul (21:00):
Yeah, I think it goes back to the goal as well of
the companies, like OpenAI andAGI, like I mentioned earlier,
and narrow AI if you take alarge language model that is
going to include a lot ofinformation and you want to
apply it to a narrow problem,like what you have in your
business.
Trying to make things unlearnthat may be causing
(21:23):
hallucination is not that easy.
Now, if you take a smallermodel, like a Phi model or
something else, and you finetune it with your own data,
which you've been gathering formany, years, then now you
actually have that unfairadvantage that you want to have
against your competitor.
(21:44):
If you're all working out of GPT4, without any data that is
local to your problem or localto what you've built over the
years.
You have no you're no differentthan the other companies that is
using GPT 4.
Now, if you're using your dataand you're integrating that
either through few shotlearning, but more generally I
(22:06):
think it would be like, throughfine tuning on smaller model
that's the point at which youcan differentiate the worst
shooting.
And today, like from Gemini toH2O, there's so many platforms
that are making it easier forpeople to fine tune their model.
And the cost of fine tuning isalso much lower than what it
(22:26):
used to be.
Like, if you think about preGenerative AI, when you need it
to have hundreds, thousands, orhundred thousands data points to
train your model, while for finetuning, you need maybe a
thousand or two thousand.
And that has proven like toalready live show.
Remarkable difference from usinga model out of the box.
(22:49):
And to be honest, there arecertain problems for which an
LLM is never going to addressthe problem, even though it's a
generation.
If you think about relationaldatabase or knowledge graphs in
every query based languages, yesthere's been like some training
on the models for SQL and theSQL syntax.
(23:12):
But in a lot of these cases,that syntax then needs to map to
the underlying data model, whichis unknown.
So to really be efficient atcreating queries that are going
to go with your data, you needto fine tune it to your schemas,
to your tables, or to your data.
To whatever else you have usedto express that data.
(23:32):
Otherwise, you're only going tohave garbage in and garbage out.
And you're not going to get theanswers that you're looking for.
Andreas Welsch (23:38):
Awesome.
I love that one because I thinkthat's such an important one in
business where there is so muchdata and that's the Key
differentiator that makes yourbusiness unique, that makes your
processes unique, that gives youthe edge and has been doing that
for a number of years.
So bringing that into your LLMapplication to make the output
(23:59):
more specific or thesummarization or the
recommendations if you will.
Again, writing assistant typethings or tweaking things.
I think that's really key.
So thank you for highlightingthat as our fourth challenge
that you still need data if youwant to continue differentiating
and not do things that everybodyelse is doing as well.
(24:21):
Now, Quentin, we're gettingclose to the end of the show,
and I was wondering if you cansummarize the key three
takeaways for our audiencetoday.
Quentin Reul (24:28):
Yeah, the first one is a quote that an old
colleague of mine used to have.
It's fall in love with aproblem, and I mean like the
customer problems, the narrowproblems that you're trying to
solve.
Not the solution.
And in this case, Generative AI.
It doesn't apply to all theproblems that you may have.
Two, as we spoke about (24:46):
data is still your differentiator.
You may have spent a lot ofmoney doing a data warehouse
over time.
It's probably translated nowinto a data swamp, but you have
that data and you are able liketo take like a small slice of it
and put it back into your modeland fine tune it to your
(25:08):
problem.
And, really that the last one: data-driven companies were (25:09):
undefined
already with big data going liketo be the differentiator.
But I think with AI it is goingto be the game changer.
And I get confused when I hearcompanies that were data
companies becoming AI companies.
(25:30):
Because I'm pretty sure that ina few months they would have to
change their labels back to datacompanies because that's going
to be the true reason that'sgoing to resonate to their
customer a lot better than AIcompanies.
Andreas Welsch (25:41):
Awesome.
Wonderful.
Thank you so much.
And maybe one question.
How can people that have joinedto today's session or listened
to it connect with you and learnmore about what you do and maybe
how you can help them?
Quentin Reul (25:53):
Yeah.
I'm on, on LinkedIn.
And I'm also on X, and I haverecently started a YouTube
channel where I'm putting therecordings of different
assignments that I've been givenat different presentations.
So I'm trying to curate thematerial as I go along.
(26:15):
I also have written a few blogsabout how to take the problem of
Generative AI, From inceptionand using things like jobs to be
done framework all the way tohow to consider it how to charge
and how to make to monetize likeyour solution depending on how
you are applying the content andwhatever else so all of that is
(26:38):
on my page and you can reach outto me.
Andreas Welsch (26:41):
So folks do make use of that opportunity and
reach out to Quentin for moreadvice and insights.
Now, Quentin, again, thank youso much for joining us and for
sharing your expertise with us.
I think it was a great session.
I learned a lot and hope you andthe audience did as well.
Quentin Reul (26:57):
Thank you.
Today we'll talk about the four generative AI
challenges that you cannotignore.
And who better to talk about itthan someone who's passionately
sharing advice on them.
Quentin Reul.
Hey Quentin, thank you so muchfor joining.
Quentin Reul (00:13):
Hi, thank you for having me today.
Andreas Welsch (00:16):
Wonderful.
Why don't you tell our audiencea little bit about yourself, who
you are, and what you do.
Quentin Reul (00:21):
Yeah, so my name is Quentin Reul.
I'm originally from Belgium.
Spent my bachelor degree and myPhD in the UK, and I've now been
in Chicago in the US for 10years.
I from a, on the side of work, Ido martial arts with my kids and
my wife.
(00:41):
And it's a lot of fun.
From a work perspective I workedat Fortune 500 before, and I'm
currently looking at helpingcompanies with their journey
into the AI strategies byadopting UX but also the
technology and the deepknowledge in technology in doing
so.
Andreas Welsch (01:02):
That's awesome.
Again, thank you so much forbeing on the show today.
And I know you've been sharing alot of your advice and
experiences on social medialately.
Should we play a little game tokick things off?
Wonderful.
Hang on.
Let's see.
This game is called In Your OwnWords, and when I hit the
(01:22):
buzzer, the wheels will startspinning.
When they stop, you'll see asentence, and would love for you
to answer with the first thingthat comes to mind, and why.
In Your Own Words.
To make it a little moreinteresting, you only have 60
seconds for your answer.
And folks, for those of you inthe audience, you can
participate too.
Put your answer and why in thechat as well.
Quentin, are you ready forwhat's the buzz?
(01:45):
Okay, here we go.
Almost.
Here we go.
It is a live show.
If AI were a fruit, What wouldit be?
60 seconds on the clock, and go.
I think
Quentin Reul (02:00):
I would be an apple, in as much as you have a
lot of layers, and, It tastesgood if you eat the skins, but
if you remove the skin and thelayers and you try to dig deeper
and eat the apple, the more youknow and the better you can
leverage it.
Andreas Welsch (02:17):
Wonderful.
And I think there's a love thereat the core as well that you
need to know about and how toget close to it.
Wonderful.
And
Quentin Reul (02:27):
it grows again if you use the
Andreas Welsch (02:28):
seeds, so it's regenerative.
Haha, perfect! I love thatanswer.
Excellent.
And I see comments here fromAshwin, he's joining us from
Hyderabad, and he said, yeah,something sweet.
Exactly.
I think that's where we're stillat in that phase where it's
sweet and we're figuring outwhat else can we do with it?
Can we make apple pie and applesauce and apple cider and
(02:50):
whatever else?
So very versatile too.
Okay.
So with that out of the way whydon't we talk about the topic of
the show?
The four generative AIchallenges that, that you need
to know about, and if you lookat the, history the, last what,
two and a half years, ever sinceChatGPT has come on the map, I
(03:11):
think there's, been not only somuch hype, but also a already so
much maturity, so much evolutionthat I'm curious what are you
seeing?
How is generative AI evolvingthat the business strategy and
the impact on businessstrategies?
Quentin Reul (03:28):
Yeah, I think if we look at what happened
November 2022 to maybe July 2023a lot of companies are at the
foremost the fear of missing outon leveraging generative AI, and
they were looking at a lot ofdifferent use cases and what
they could do it, what advantageand what they what gap that was
(03:48):
filling compared to moretraditional machine learning out
of the box, you had thousands ofdata points that can you now use
and model that were pre trainedfor problems that were complex
compared to before where youneeded to spend years creating
the training sets to, togenerate that.
(04:08):
But I think now, there is arealization that maybe The
direction that the big LLMproviders like OpenAI or
Anthropic and where thecompanies are going is not
necessarily in the samedirection.
In as much as OpenAI ispromoting artificial general
intelligence and I think thatfor the businesses, where their
(04:32):
value is more in narrow AI.
It is not They will gain valuein content creation and
marketing by leveraging AGI.
But it's where they are veryspecific to their niche that
really the differentiation isgoing to come.
So I think that there we'regoing to start seeing a
(04:52):
realization that one, not onlyAll use cases that you add could
be fulfilled with the largelanguage models.
If you want to do the predictionof what to put on your shelf
tomorrow, LLM is not going tohelp you.
But if you are, as I say, liketrying to create content, or
you're trying like to answerquestions, it's there and it can
(05:15):
help you.
Andreas Welsch (05:16):
See, that's the part where I'm always curious to
to, see The two be combined.
We, we know that LLMshallucinate, they have factual
inaccuracies, they have somebiases they, are not even able
to do certain tasks that narrowAI and even predictive analytics
and machine learning can do.
I feel if you combine the two orif you create a prompt where you
(05:38):
substitute some information andsome variables and you put in
those data points that you havegenerated using other proven
technologies and methods, thenit's a, good vehicle, right?
To leverage the best of bothworlds.
What are you seeing there?
Quentin Reul (05:54):
I think that like what I was working on there,
Fortune 500 and we were workingwith a lot of different models
and GPT 4 and there was, to behonest, not a lot of use cases
for which we needed somethingdifferent than GPT 4 because
they weren't generic enough.
(06:14):
Now, I think that once you getto the point that you just
mentioned, which I tend to callor refer to as a cost to be
wrong.
So the higher the cost to bewrong, the higher you have a
risk of hallucination.
If your cost to be wrong is nilor zero, I call it creativity.
And I think that's another thingthat business need to think
about is that there are plentyof use cases where the
(06:37):
creativity is important andactually useful.
Like I have the blank pagesyndrome.
So whenever I try to write and Iwrote a PhD and it took me a
long time and if I had agenerative AI, it would have
been much faster.
I get stuck on the page and Iwant to rewrite, everything.
The LLMs are very good for thatproblem.
(06:59):
But if I try to make a judgment,let's say, predict or give a
treatment to a patient orpredict how to apply law for
particular problems.
I can't be wrong.
I can cause someone's going tojail or I can cause someone's
death.
So I think that's where we needto, look at the problem at a
(07:21):
different scale.
I think that what is being donewith the AU Act, where they have
that different level of risk forassessment.
And more recently, I saw thatthe IEEE is actually following
that same methodology in termsof accrediting whether your
solution is going to be ethical.
And they look at it from theperspective of low risk, medium
(07:43):
risk, High risk and unacceptablerisk and things that are
considered unacceptable risk arethings like social scoring where
for example You would be in hrand you would use whether people
are attending meetings orwhether they're sending emails
are the way to determine Whetheror not they are predictive and
whether they should keep or stayin the company or not
Andreas Welsch (08:05):
now, I think those are really important
aspects in coming back to thatchallenge, right?
It's first of all understandingwhat can you use the technology
for and what can you maybe useother technologies for that are
better suited, that are maybecheaper, that are maybe more
cost effective to operate and tobuild.
Certainly everything has a tradeoff, but again, you don't need
(08:27):
to have a large language modelfor So many use cases.
Now I'm curious folks for thoseof you in the audience, if you
have a question for Quentin orI, please put it in the chat.
We'll take a look in a minute ortwo and pick up those questions.
Quentin Reul (08:44):
And I think to the last point that you made,
Andreas, with regard to notevery problem is a Generative AI
problem.
I think that's a responsibilityof technology leader.
And to have a deep enoughunderstanding of the technology
to be able to advise thebusiness and their partners
that, yes, you could useGenerative AI, or you could put
(09:05):
an AI label, but that doesn'tmean that you're going to have a
better product for yourcustomers.
Because at the end of the day,as a business, how are you
making money?
What is your return oninvestment?
It is about delighting yourcustomers by providing a
solution that is intuitive.
Integrating AI makes it morecomplicated for the solution or
(09:25):
the problem or the process thatpeople are going through.
Or you're not really helpingyour customers and you're not
necessarily going to get thereturn that you're looking for.
Andreas Welsch (09:35):
Now, you've mentioned you've worked at
Fortune 500.
I did a bit of work in previousroles in corporate with Fortune
500 as well.
And I heard leaders say justrecently at the beginning of the
year, if it's not Generative AI,it's not AI; or we're not
pursuing it.
And I think that's misleading inso many ways.
(09:55):
And it's challenging.
It's troublesome.
To your point you need to knowwhen you need to use a large
language model, when you useyour logistic regression or
other capabilities.
I'm wondering there, from yourexperience, how can you assess
the fitness, if you will, ofdifferent LLMs, of different
(10:15):
approaches, if you want to solvea particular need, so you don't
run into the trap of if it's notGen AI, we're not doing it?
Quentin Reul (10:22):
Yeah, I think there's been a lot of work that
has been done in evaluation andbenchmarks.
Thanks.
And I think they're providinggood insights as to whether or
not a model is better thananother model.
But I think that we have to bevery careful when it comes to be
looking at benchmark, is thatthey're not designed for your
(10:44):
narrow AI problem.
They are designed for verygeneric problem.
The coding problem is looking atdifferent language, but it's
probably not very extensive onSQL.
It's also somewhat misleadingbecause we have seen some
companies being using part ofthe benchmark as their training
set and that's cause overfillingbecause it does what it says
(11:06):
like it's going to do becauseyou train it on.
So I think that's definitely oneaspect.
It's like you can use abenchmark as a pre selection for
what you're going to look basedon the type of problem that you
have.
But I think after that youreally have to test it.
You have the chatbot arena whereyou can put two different LLMs
side by side and you have yourprompt and you see which one is
(11:27):
going to give the best results.
And that is, it's an interestingproblem if you are creative.
But I think if you are a companyyou really have to invest on a
golden set.
So you probably don't need asmany data points as you needed
before.
Like a good training set or agood golden set of 500 data
points is probably going to besufficient, but that is going to
(11:52):
give you an ability to test likethe different models over time
not only like the first time asyou're creating, but a lot of
these models are evolving veryrapidly and because they're
offered as SaaS model they gotoday, tomorrow like the new
version is the old version isgone.
And there is an opaque problemin as much as you don't know
(12:14):
what training data is going in.
So you don't know whether thelength of fitness of your LLM
before and when you do it likeyour previous assessment is
going to remain as you're doinglike your assessment.
So monitoring and having thatgolden set for ongoing
monitoring is very important.
Andreas Welsch (12:30):
That's an important point that you're
bringing up.
And I feel that hasn't beengetting as much attention lately
as it has been probably a yearago, right?
As as models change even if yourAPI stays the same, but if the
LLM underneath changes from GPT3 to 3.5 to 4.0, to whatever is
(12:52):
next, you need to do yourregression testing again.
You need to test if the sameprompts work.
We know that between 3.5 and 4,there are differences, right?
Differences in the creativityand other aspects of these
models and underneath.
So you need to build that intoyour plan as well, and you need
to make sure that you haveresources and you have budget to
(13:15):
do all these changes.
And especially if vendors aredeprecating models that you can
react fast enough to put in thenew ones.
Quentin Reul (13:24):
And it's very much trial and errors.
At the end of the day, to giveyou an example, as I mentioned,
I have the white page syndromeand I started creating an
application to create blogs.
Because most of the solutionsthat exist can get you
somewhere, but it was not givingme the solution that I wanted.
(13:45):
And I used Gemini.
And I was writing my prompt andmy goal was to create blogs that
were of a certain length, eitherin time or in number of words.
And when you do LLMs and itfails the first time, what do
you do?
(14:05):
You go back and you change yourprompt, because it's the
cheapest thing you can do.
And you tell the LLM to actuallyrewrite the prompt in a way that
it would understand it himself.
So it will provide more context,it will write the information in
the right, with the right levelof instructions and so forth.
But you get to do that four,four or five times, and you
realize that it's not the promptthe problem.
(14:26):
And in the particular case, Idid some digging, and what I
realized after a while is thatthere was no training data about
length.
And therefore, no understandingof that.
It could predict length in thenext token, but because there
wasn't a notion of length aspart of the training data, it
was not able to follow myinstruction to the end.
(14:49):
I think that's where you reallyneed to test.
And frameworks like LangChainare very useful for that because
it provides you with an easy wayto integrate with different
providers in a quick way.
About a year and a half ago, ifyou were on Azure, the only
model that you had were theOpenAI models.
That has changed now, with theAzure AI Studio, where you can
(15:13):
have access to Llama, and chooseevery integration through
HuggingFace.
But, before that, you were stuckwith one thing.
So at least now you have a bitmore of that connection, but I
think that something likeLangChain, definitely the PoC or
the early analysis stage is thebest thing like to use.
(15:33):
I wouldn't necessarily use thatframework in production.
Because it's very heavy andleverage a lot of other
libraries.
But in terms of doing thatresearch aspect, like what we
used to call ADA in the moretraditional machine learning if
you do that with something likeLangChain, then you're going to
get faster results and you aregoing to be able to do that
(15:54):
comparison much faster as well.
Andreas Welsch (15:56):
Awesome.
And just looking at the chathere somebody saying if I've
asked it one time, if it cancount, smiley face, it can't.
I'm sure it thought it could.
But we've seen those examples.
It's a good reminder as well.
Now, we've already talked aboutthis notion of don't give in too
(16:19):
much into this hype ofeverything should be Generative
AI.
We talked about, hey, there'ssome evaluation frameworks,
methodologies.
But what are some of thetechnologies that you're seeing
that can address thosechallenges of Generative AI?
Quentin Reul (16:35):
I think, LangChain is definitely one that can help
you with that in personintegration.
I think it's so difficult todayto stay up to date with all the
tools.
Every day, like I'm on a TLDRgetting my new specs on what is
happening in AI.
Thanks.
And many other newsletter, andthere's just no way you can keep
(16:58):
up with all the tools.
You have Llama File, you have alot of things that are
happening.
We spoke about different aspect,but completion as opposed to
chat.
That also has totally differentaspect.
I was for example, I was askinga normal LLM, I think it was the
Llama to write a blog and I'dgiven the instruction and the
(17:24):
Llama model just gave me backthe instruction.
But then I used like the Llamainstruct and I provided like the
same prompt through the Llamainstruct and it created like the
blog that I wanted.
So it's again like with thesetrial and error that you will
find like some of these.
Aspect of it and be able toaddress it as you go forward.
Andreas Welsch (17:45):
I think that's a really important point as well,
right?
Do your evaluation, especiallynow that there is so much choice
and it seems like it only keepsgetting bigger and more complex,
what do you use for?
What do your evaluation and seewhat type of tasks do you have,
what model performs or evenbetter than others.
(18:05):
There's one question fromJennifer here in the chat and
she says, Hey you mentioned thatLangChain is heavy and not
suitable in production.
Can you share more about this?
Or maybe qualify what you meantwhen it comes to lang chain and
similar tools.
Quentin Reul (18:21):
Yeah, so LangChain, because it was
designed like as a community andtrying to address all of the
different LLMs, it bringsTensorFlow, it brings PyTorch.
As you're creating an image,let's say a Docker image based
on LangChain, it's going to bebloated.
I think to recall it wassomething like 6GB.
(18:42):
And my prompt and my code wasmaybe about like 200k.
And it has an implication on howyou are going to productionalize
it.
A lot of the use cases that Isee are still very much on
demand it's not that you have alot of use cases that are flawed
where your LLM is going to run24/7 and processing content all
(19:06):
the time.
One way that you're going to useto minimize your cost and be
able to scale is something likeLambda or serverless, depending
on whatever infrastructure youare.
But there are limitations.
For example, Lambda doesn'tsupport Docker images to be more
than a certain amount of memory.
So now you have to create layerson top.
(19:28):
So you are adding complexity byusing something like LangChain.
That has got a lot of thingsthat may be useful, but if
you're doing OpenAI you don'tneed to have access to Oracle
Cloud Infrastructure.
You don't need access to Bedrockbecause you're not going to use
any of these components.
(19:48):
Whereas if you were usingdirectly the OpenAI or the
Microsoft Azure API it's goingto be much leaner because it's
only going to have what isnecessary to do the job.
Andreas Welsch (19:59):
Wonderful.
Thank you for summarizing that.
I think that was really tangibleand some good advice there.
What are some of thecomplexities that you inherit
when you go down a certain path?
You also mentioned that youdon't have like a continuous
loop where it's not always on,depending on the use cases that
(20:20):
you see, especially inenterprise.
To me, that comes back to data,to pipelines, these kinds of
topics.
And you alluded to that earlier.
Yes, large language models helpus get to production faster.
We might not necessarily need tohave a data scientist or be a
data scientist to get somethingout there, at least something
(20:42):
that's good enough.
I'm curious, what are youseeing?
What's the role of data stillwhen it comes to LLMs in that
current landscape especiallylooking at regulation at the
beginning, you talked about theEU AI Act and IEEE, what role
does data play there?
Quentin Reul (21:00):
Yeah, I think it goes back to the goal as well of
the companies, like OpenAI andAGI, like I mentioned earlier,
and narrow AI if you take alarge language model that is
going to include a lot ofinformation and you want to
apply it to a narrow problem,like what you have in your
business.
Trying to make things unlearnthat may be causing
(21:23):
hallucination is not that easy.
Now, if you take a smallermodel, like a Phi model or
something else, and you finetune it with your own data,
which you've been gathering formany, years, then now you
actually have that unfairadvantage that you want to have
against your competitor.
(21:44):
If you're all working out of GPT4, without any data that is
local to your problem or localto what you've built over the
years.
You have no you're no differentthan the other companies that is
using GPT 4.
Now, if you're using your dataand you're integrating that
either through few shotlearning, but more generally I
(22:06):
think it would be like, throughfine tuning on smaller model
that's the point at which youcan differentiate the worst
shooting.
And today, like from Gemini toH2O, there's so many platforms
that are making it easier forpeople to fine tune their model.
And the cost of fine tuning isalso much lower than what it
(22:26):
used to be.
Like, if you think about preGenerative AI, when you need it
to have hundreds, thousands, orhundred thousands data points to
train your model, while for finetuning, you need maybe a
thousand or two thousand.
And that has proven like toalready live show.
Remarkable difference from usinga model out of the box.
(22:49):
And to be honest, there arecertain problems for which an
LLM is never going to addressthe problem, even though it's a
generation.
If you think about relationaldatabase or knowledge graphs in
every query based languages, yesthere's been like some training
on the models for SQL and theSQL syntax.
(23:12):
But in a lot of these cases,that syntax then needs to map to
the underlying data model, whichis unknown.
So to really be efficient atcreating queries that are going
to go with your data, you needto fine tune it to your schemas,
to your tables, or to your data.
To whatever else you have usedto express that data.
(23:32):
Otherwise, you're only going tohave garbage in and garbage out.
And you're not going to get theanswers that you're looking for.
Andreas Welsch (23:38):
Awesome.
I love that one because I thinkthat's such an important one in
business where there is so muchdata and that's the Key
differentiator that makes yourbusiness unique, that makes your
processes unique, that gives youthe edge and has been doing that
for a number of years.
So bringing that into your LLMapplication to make the output
(23:59):
more specific or thesummarization or the
recommendations if you will.
Again, writing assistant typethings or tweaking things.
I think that's really key.
So thank you for highlightingthat as our fourth challenge
that you still need data if youwant to continue differentiating
and not do things that everybodyelse is doing as well.
(24:21):
Now, Quentin, we're gettingclose to the end of the show,
and I was wondering if you cansummarize the key three
takeaways for our audiencetoday.
Quentin Reul (24:28):
Yeah, the first one is a quote that an old
colleague of mine used to have.
It's fall in love with aproblem, and I mean like the
customer problems, the narrowproblems that you're trying to
solve.
Not the solution.
And in this case, Generative AI.
It doesn't apply to all theproblems that you may have.
Two, as we spoke about (24:46):
data is still your differentiator.
You may have spent a lot ofmoney doing a data warehouse
over time.
It's probably translated nowinto a data swamp, but you have
that data and you are able liketo take like a small slice of it
and put it back into your modeland fine tune it to your
(25:08):
problem.
And, really that the last one: data-driven companies were (25:09):
undefined
already with big data going liketo be the differentiator.
But I think with AI it is goingto be the game changer.
And I get confused when I hearcompanies that were data
companies becoming AI companies.
(25:30):
Because I'm pretty sure that ina few months they would have to
change their labels back to datacompanies because that's going
to be the true reason that'sgoing to resonate to their
customer a lot better than AIcompanies.
Andreas Welsch (25:41):
Awesome.
Wonderful.
Thank you so much.
And maybe one question.
How can people that have joinedto today's session or listened
to it connect with you and learnmore about what you do and maybe
how you can help them?
Quentin Reul (25:53):
Yeah.
I'm on, on LinkedIn.
And I'm also on X, and I haverecently started a YouTube
channel where I'm putting therecordings of different
assignments that I've been givenat different presentations.
So I'm trying to curate thematerial as I go along.
(26:15):
I also have written a few blogsabout how to take the problem of
Generative AI, From inceptionand using things like jobs to be
done framework all the way tohow to consider it how to charge
and how to make to monetize likeyour solution depending on how
you are applying the content andwhatever else so all of that is
(26:38):
on my page and you can reach outto me.
Andreas Welsch (26:41):
So folks do make use of that opportunity and
reach out to Quentin for moreadvice and insights.
Now, Quentin, again, thank youso much for joining us and for
sharing your expertise with us.
I think it was a great session.
I learned a lot and hope you andthe audience did as well.
Quentin Reul (26:57):
Thank you.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Therapy Gecko
An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.