August 26, 2025 • 38 mins
In this episode of How I AI, Dr. Alex Kihm, founder of POMA, explains a practical way to make AI more accurate while using less compute and energy. We talk about his path from early plug-in hybrid work in Germany to building tools that reduce waste and deliver more reliable results in everyday workflows.
🔥 Topics We Cover:
- How thoughtful data prep and retrieval improve answer quality with fewer tokens
- Why “made-up answers” happen and simple ways to curb them
- When smaller models can beat bigger ones in practice
- Where energy-aware design lowers costs for teams
Tools and Platforms Mentioned:
- Tools we mention include frameworks like POMA’s approach, plus platforms such as LlamaIndex, LangChain, Mistral, ChatGPT, and Claude
Connect with Dr. Alex Kihm & Learn More:
Ready to cut through the overwhelm?
- Grab a 30-minute AI Jumpstart Session and walk away with a clear plan to match the right tools to your goals so you can finally move forward with ease.
- Check out my free AI Starter Guide and get the basics explained simply so you can start using AI with confidence.
- Explore Collective AI memberships and masterminds
If you enjoy this episode, please rate and review the show. Sharing it with a friend who’s AI-curious helps this growing brand reach more people.
Want to be featured? Apply at howiaipodcast.com
More About Brooke:
Instagram: thebrookegram
Website: brookex.com
LinkedIn: Brooke Gramer
More About the Podcast:
Instagram: howiai.podcast
Website: howiaipodcast.com
"How I AI" is a concept and podcast series created and produced by Brooke Gramer of EmpowerFlow Strategies LLC. All rights reserved.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Brooke (00:03):
Welcome to How I AI the podcast featuring real people,
real stories, and real AI inaction.
I'm Brooke Gramer, your host andguide on this journey into the
real world impact of artificialintelligence.
For over 15 years, I've workedin creative marketing events and
business strategy, wearing allthe hats.
(00:24):
I know the struggle of trying toscale and manage all things
without burning out, but here'sthe game changer, AI.
This isn't just a podcast.
How I AI is a community, a spacewhere curious minds like you
come together, share ideas, andI'll also bring exclusive
discounts, and insiderresources, because AI isn't just
(00:47):
a trend, it's a shift, and thesooner we embrace it, the more
freedom, creativity, andopportunities will unlock.
How I AI is brought to you inpartnership with the Collective
AI, a space designed toaccelerate your learning and AI
adoption.
I joined the collective and it'scompletely catapulted my
(01:09):
learning, expanded my network,and showed me what's possible
with AI.
Whether you're just startingout, seeking community or want
done for you solutions, thecollective gives you the
resources to grow your businesswith AI.
So stay tuned to learn more atthe end of this episode, or
check my show notes for myexclusive invite link..
(01:31):
Hi everyone.
I just sat down with Dr.
Alexander Kihm.
He's the founder of POMA.
His work is reshaping how largelanguage models operate, and
more importantly, how they'reconsuming resources.
In this episode, Alex explainshow POMA's way of using RAG and
(01:51):
chunking, which we'll get intowhat that is later, helps AI
give better answers while usingup to 80% less energy than usual
methods..
As more people are starting touse AI every day, it's important
to connect the way we run AIwith the bigger problem of the
(02:12):
world's energy and resourcecrisis.
He also shares fascinatinginsight into how Germans are
approaching AI adoption withprivacy as the top priority.
If you care about AI's potentialand its impact on our planet,
this episode will give you awhole new lens on both.
(02:33):
Alright, let's dive into today'sepisode.
Hi everyone.
Welcome to another episode ofHow I AI.
I'm your host, Brooke Gramer.
Today I'm joined by Dr.
Alexander Kihm.
He's an engineer, economist, andserial entrepreneur who's
tackling AI's biggest challengeswith POMA, a breakthrough
(02:54):
solution that reduceshallucinations and energy in
large language models.
Dr.
Kihm, thank you.
Welcome.
Dr. Kihm (03:03):
Thanks for having me.
Brooke (03:05):
Yes, I'd love to open this space and hear more about
you.
Please share about yourbackground and how you ended up
doing what you do today.
Dr. Kihm (03:15):
Yes, sure.
I'm an engineer by trainingoriginally um, where lots of
Germans are.
That's always the joke, but it'sstill kind of true.
And I played with LEGO veryearly.
And I went to legal techniquevery early on uh, and then I had
the luck of getting a hand medown computer from my parents
when it was already I think fouryears old or something.
At five I dis assembled it.
(03:36):
At six I started coding.
It's kind of my passion.
So it became kind of obvious uh,decide that I want to become an
engineer.
And there's this program inGermany, I mean now with
Bachelor and Masters like yougot in the US but back then it
was diploma.
So it's combined Bachelor andmaster's in one..
Brooke (03:53):
Okay.
And
Dr. Kihm (03:53):
then, for the industrial engineering degree,
you could basically choosewhatever you want and I chose a
lot of energy engineering, sofrom nuclear power plants,
conventional plants, and thensolar cells, wind turbines.
I, I had learned it all andtowards the end of my studies I
realized okay, if you want toreally have an impact, and the
maximum you can get there isbecome the guy responsible for
(04:14):
certain screws and certain microdevices that are inside a big
power plant.
I love what I learned, but Irealized that won't be my career
path.
Mm.
Brooke (04:22):
Um,
Dr. Kihm (04:24):
And back then I already had run a little
startup.
I started when I turned 18together with my neighbor And it
was what later turned out tobecome Germany's first and one
of the largest legal techcompanies.
Brooke (04:34):
Wow.
Dr. Kihm (04:35):
And we are still running it, it's basically a
lawyers network.
So we help lawyers beingreplaced for court appointments
by other lawyers.
So whenever you have this modernlegal tech like you, you just
click a form online and then thenext day there's a lawyer suing
someone for you in court.
That's basically us behind it inthe API that we can automate it.
(04:55):
And I realized, okay, I like torun companies.
That's more like my thing.
But first I wasn't done witheducation, so I went to the
German Aerospace center and didmy PhD there.
Could leverage a bit of myenergy background co-developing
an electric car.
I actually have developed thefirst plugin hybrid car in
Germany.
That was my project.
Yeah, it was super exciting.
It was really nice.
I learned a lot.
And actually during that time Itransitioned more and more from
(05:19):
classic engineering questions,more towards big data, large
scale coding data questions.
Brooke (05:25):
Mm-hmm.
Um,
Dr. Kihm (05:26):
So in the end we did the forecast model for, okay, we
have this car, now we understandthis car.
How will it, penetrate themarket?
What is the forecast forelectric cars?
Depending on fuel prices and soon.
And I was really proud for whatwe built in the end because we
had this model that used verytraditional approaches, but on a
very large scale to predict ifwe now have these incentives or
(05:46):
these taxes or whatnot, how manypeople will drive the electric
cars and, back then, it was thefirst time I tried to use AI or
what was called AI back then.
Mm-hmm.
I would rather call it ad likeartificial dumbness.
Um, uh, What we had back thenwas like neural networks.
But for bigger purposes, itwasn't fit.
So that was still my firstcontact with very large models,
(06:11):
but not like AI models.
I realized after my PhD that, Ilearned a lot, but I'm no one
that works in big organizationsscience is very cool, but it's
still normally run by bigorganizations,
Brooke (06:23):
right.
Dr. Kihm (06:23):
I've started founding my second company which is a
FinTech.
So we disrupted the Germanpension market, we did some
financial innovation there,combined insurance solutions and
fund management stuff.
And in 2019 we sold it toRaisin, which now is one of, the
larger fintechs we are even inthe US now.
And raisin is pretty successful.
Pretty large company.
(06:44):
I'm still very proud of thecombined solution we have there
and the wealth managementplatform.
But obviously after some yearsof, integrating this stuff and
helping and so on, it startednagging at me.
Brooke (06:55):
Mm-hmm.
Dr. Kihm (06:55):
Like, uh, okay.
You need a new challenge.
And it was around 2022, GPT, Ithink between two and three,
where you saw the first time.
Oh, wow.
That's something different thanthe cat detector models we had
before.
Brooke (07:09):
Yes.
Dr. Kihm (07:09):
There's something coming up and I'm fascinated by
LLMs.
Long story short, 2023, westarted dabbling a bit more
seriously.
Of what we can do with thisstuff.
Brooke (07:19):
Mm-hmm.
And,
Dr. Kihm (07:20):
um, Given that I told you the background and some
certain legal text, we thought,okay, we have already have
10,000 law firms as customers,let's try to build something for
them.
So what could they use?
How could they make use of thesenew language models and um.
we quickly realized what somepeople realized painfully that
if you just ask the standardversion of, you remember ChatGPT
(07:41):
back then when it came out.
Yeah.
Can you help me?
This and that.
I have this thing to present incourt.
It would just make up things.
Brooke (07:47):
And,
Dr. Kihm (07:47):
Everyone knows the story or the stories by now
plural of several lawyersgetting disbarred or whatnot
because they just enter courtwith made up stuff.
Brooke (07:57):
Mm-hmm.
Dr. Kihm (07:57):
And people thought, okay, maybe we train the models
more and more.
And it didn't help.
Even the new reasoning models,they hallucinate even more
because they are in this kind ofecho chamber with themselves.
Others who thought fine tuningworks didn't either because it's
made for something totallydifferent.
And then there was the RAGscene.
The so-called rag scene isretrieval augmented generation.
Brooke (08:19):
Yes.
Dr. Kihm (08:19):
And it means you avoid hallucinations by giving the
models some context for yourquestions.
So basically, here's the book.
Answer a question based on thisbook.
And I found this very promising.
I found this very, verypromising.
And I still think the idea isabsolutely right to have the
intelligence from training thestyle and the, let's say
(08:41):
craftsmanship from fine tuning,but then the information from
augmenting the retrieval oraugmenting the question with
some retrieved context.
Brooke (08:50):
I wanna pause you on, pause on please.
Pause.
Pause right there.
Because for those who arelistening, First of all, wow,
incredible amount of experience.
You're still so young and you'veaccomplished so much.
And for those who might notunderstand what RAG is and the
importance of what you're just.
(09:10):
Noting could you maybe break itdown on Oh, absolutely.
Why this is important and how itrelates to hallucination?
Dr. Kihm (09:16):
Yes, absolutely.
I'm sorry I skipped that part.
Yeah.
So it basically means uh, let'sgo back to this first lawyer and
first ChatGPT So you just ask amodel a question and the model.
Relies only on what it haslearned.
So it was trained on a lot ofstuff, the whole of Wikipedia,
55,000 unpublished books andwhatnot.
And it actually helped a lot todevelop some sort of
(09:39):
intelligence.
It's fascinating.
So the thing is veryintelligent, but think of a
child that went through 10universities but doesn't have
any books anymore.
So the child must now be quiteintelligent.
But when you ask it like acertain question, like, okay.
How does this work?
It just connects some dots, butit doesn't have the primary
information in front of it, soit can just only go so far.
(10:02):
And this leads to thesehallucinations we are all
talking about.
Like, it just comes up with AlexKihm is a geographer from the
18th century because it somehowconnects Alex and my last name
and geography for some reason.
Brooke (10:14):
Okay.
And
Dr. Kihm (10:14):
it's not me.
This is the stuff that comes upwhen you just rely on training.
Brooke (10:18):
Yes.
Dr. Kihm (10:18):
And what RAG does?
RAG stands for Retrievalaugmented Generation.
So generation means when you aska question, you basically just
say the tell the model.
Okay, here's my question andbased on all these words that is
my question.
Generate an answer.
So the generation meansbasically answering.
And if you say retrievalaugmented generation, that means
(10:40):
you augment the generation byretrieving some stuff first.
Which means like, okay, I havemy library here.
And I say, please my child,here's this book I found on the
topic of X.
Can you please dissect it for meand answer my question based on
what you see in this book?
And so like this, I can rely onthe ground truth, knowledge,
(11:01):
that here's the wisdom, there'sthe intelligence, please
Intelligent child, take thisbook and then tell me what the
answer to my question is.
That is basically rag in verybroad terms.
But it's still actually what RAGdoes.
It's
Brooke (11:14):
yeah,
Dr. Kihm (11:15):
it's quite accurate to be honest.
Brooke (11:17):
Thank you.
Yeah, thank you so much forbreaking that down.
Because I think it's soimportant to talk about what RAG
is and I Yes.
And I know that you've alsomentioned the term chunking.
Yes.
Is that the same thing as Rag?
Dr. Kihm (11:31):
Let's say it's one of the building blocks.
Okay.
So what is interesting, and thatalso relates to our history how
we got to POMA.
Brooke (11:37):
So
Dr. Kihm (11:37):
at first we thought RAG is a solution.
So we just build a rag stack.
There's several softwares for itand people who are interested.
You can just Google, forexample, LlamaIndex or a Lang
chain.
You just Google how to set up arag.
There's super nice articles andtutorials about it, it all
relates to this talk to yourdata.
It's always, I have data, I wanta chat bot or an answer
(11:58):
generation machine of some sort.
And I recommend to everyone tojust dabble with it a bit.
And when you do that, you findout the same thing that we found
out.
And that is, okay, I have thislibrary of stuff like articles,
PDFs, whatever texts.
Then when I try RAG what itdoes, it, it gives me the pieces
of information that arenecessary for the question.
(12:19):
And at first you don't see anychunking.
This is the interesting part.
It's an invisible part of thiswhole pipeline.
And when we tried our first,little dabbling legal bot, we
realized that answering onequestion would cost like around
$1.
Which is kind of much for us.
I mean, just the demo night costme like my monthly rent.
And I realized, wow, what ishappening there?
(12:40):
Why is this so expensive?
And I found out when you lookinto, a bit deeper in the logs,
you find that okay.
It just put everything thatremotely relates to the
question.
It just puts into this, here, mychild here is this backpack full
of information, and I only wantto tell you what is pasta made
of?
And so this is dependent on thechunking because when you ingest
(13:05):
data into RAGs, so let's say youbuild your library, you have all
these books, and then you wantto get them into the library for
your RAG.
This is called ingestion.
But what you don't know when youjust start out with it is that
it's not like, here's the wholebook and it somehow magically
gets the whole content into thisdatabase without you thinking
(13:26):
how it works.
But what it secretly does is itchops up the information.
So it basically tears out pages.
So you could say it tears outpages.
And then each page gets what iscalled embedded, so represented
by mathematical information tofind it later, To find them
later?
You have to somehow index it andthey have to index each page.
You with me so far?
Brooke (13:46):
Yes.
Works for
Dr. Kihm (13:47):
people.
Yes.
Brooke (13:48):
I've heard rag and chunking explained.
And it's.
Interesting to hear behind thescenes what our ais are doing
when we're using them yeah,particularly LLMs.
And you bring up such importanttopics of why we're honing in on
this and why it's so importantis one, AI is very expensive and
(14:09):
two it takes up a lot ofresources and energy, which Yes.
Is so important because a lot ofpeople have caused quite the
concern for using these toolsand technology.
So one thing I'd really love toexplore with you and all the
work you're doing at POMA is theenvironmental side of all of
(14:32):
this.
Yes.
'cause, you know, a lot ofpeople are talking about
hallucinations with AI andyou've been able to successfully
find solutions, but they're notnecessarily connecting that to
the energy and resourceconsumption.
Yes.
That you are helping with.
So if you wanna expand on that,since you've been in the AI and
(14:52):
the tech space for so long andyou've seen this progression and
how we're becoming moreefficient today, I'd love to
start there.
Dr. Kihm (15:00):
Absolutely.
Yes.
Also, one of the factors reallydriving us we have a big
advantage here, and that is thecost that people are talking
about.
So if you have people who don'tcare about the environment,
they're just purely, costoptimizers, yes.
You can even tell them, whatyou're paying for.
Yes, there is a big margin, butbasically the only cost of AI is
actually resources, is energyconsumption.
(15:21):
The good thing is we canactually, make both happy.
We can make the controllershappy because it's cheaper, but
at the same time, we save theenvironment because it's
directly proportional.
This is the cool thing here.
That being said like forexample, when I told you about
this demo night back then youpaid like, let's say a dollar
per question because of all thestuff you gave to each question
and the more you give them inyour question, the more
(15:43):
resources the AI consumes.
This is like the trade off here.
So if you just ask it, it'squite efficient, but it makes
stuff up.
Brooke (15:50):
And now
Dr. Kihm (15:51):
you have this bad trade off that, okay, I want to
have the truth and not made ofstuff, but if I overburden it
with information.
I mean, literally every wordcounts.
All is, for example, if you aska very long question, it takes
up like five times the energy ofa question of the fifth, that
size.
This is, few people know this,but wow.
Yeah.
This is the interesting part.
(16:11):
At least the reading phase ofthe ai then it starts answering.
So if you want a long answer, italso consumes more.
And if you just say yes or no.
Very efficient, very German.
But it's that's the interestingpart.
So each word, or to be honest,it's called token.
It's like half a word.
It's like a syllable.
Brooke (16:25):
Mm-hmm.
Dr. Kihm (16:26):
The thing is more there, more energy, simple as
that, and models become a bitcheaper.
People see this over time, andthis is because they do some
shortcuts.
They do clever optimization.
But in the end, and this is alsowhy, I always answer, why we, we
will still be there in 10 years.
In the end, there's only so faryou can get to energy
consumption per token, and itwill always be there.
(16:48):
So it will, and it becomesincreasingly important.
The more people, use ai, themore tokens are burned.
We always call it burn tokens,that if imagine for every
question you have in your littlechatbot assistant here and there
and here and whatnot, imagineyou always burn like a book.
It's crazy.
It's wow.
It's something that if thatscales, we really have a
(17:09):
problem.
People are already building newnuclear plants and, forfeit
their green goals and whatnot.
Because now there's so muchenergy that is needed right now.
Yes.
And the big lever to that isjust put less.
Less ballast into questions itmakes sense to have good quality
answers.
And this is based on how muchcontext you give.
(17:29):
But if you give pointlesscontext, like for example, I
have a question on, I want totalk like Ted, so I take this
book, but I only want my intro,so I would normally just go for
the intro and not like, give thewhole book to the AI.
Just doesn't make sense.
And right now, rags have thiskind of, I call it the Dodge Ram
approach.
So not very surgical, but justblast through the door and
(17:50):
here's all the information andthat is inefficient resource
burning.
And so the good thing is that wekind of have a little advantage
even for, against those peoplewho say, who cares?
And that is, the answers evenget better.
Brooke (18:04):
So like
Dr. Kihm (18:05):
always imagine I giving you a whole book for one
little question.
You have to go through thiswhole thing and it's exhausting.
It's not only good for theenvironment, but also better for
you if I give you the surgicalinformation.
Brooke (18:16):
Yes.
You bring up something that justsparked in me because I've heard
a lot.
First of all, thank you so muchfor expanding on that.
Small language models.
Yeah.
Can you explain the differencebetween small language models
and large language models yeah.
Let's start there.
Dr. Kihm (18:32):
Yes.
Yes.
I'm fascinated by them.
I'm very fascinated and actuallywe in our internal processes,
like what we do with POMA, Iwill expand on that later.
We actually use some sort ofwell smallish language model.
And I think they are the futurefor certain purpose tasks,
especially this agentic stuff.
So like if you.
Only book flights for people.
(18:52):
You don't need to have readgooder for it.
So that obviously, translatesperfect to certain focused task
models and small language modelsare especially nice if you give
them context.
So you don't need a PhD academicto recite stuff from recipes.
That's basically the idea behindsmall language models combined
with RAG.
So if you have a certain.
(19:15):
Task and you have theinformation to fulfill this
task.
It's a bit like, Hey, can youcook this?
And here's the recipe.
You don't need a PhD for that.
Brooke (19:22):
Mm.
And
Dr. Kihm (19:22):
That's the idea behind small language models and I
think there will be many of themand they will all be RAG powered
because you will never use asmall language model and just
give it too much freedom ofgenerating an answer with
whatever little.
Intelligence it has.
Brooke (19:38):
But
Dr. Kihm (19:38):
I think this combination, and you're already
seeing this come up likepurpose-built models, smaller
ones that then have the perfectinformation and they don't need
this overpowered brain.
And they are very fast, they'revery efficient.
And I think that is for certaintasks is the future.
Brooke (19:54):
That's beautiful.
So it sounds like POMA issolving a lot of problems, that
is a very
Dr. Kihm (20:00):
simple thing.
Yeah.
Brooke (20:01):
Yeah.
It, it helps AI give betteranswers using less energy and
organizing the information in away that's easier for it to not
hallucinate.
And this must be really usefulfor industries like healthcare
where being accurate reallymatters.
(20:23):
Like, could you describe yourtypical client that you are
working with and maybe anysuccessful case studies?
Dr. Kihm (20:31):
So generally I mean, I always have this discussion with
my team that I would say we canhelp everyone.
But that's always kind of myattitude.
I love stuff that, can begenerally important and
impactful.
Yes.
But don't get me wrong, I thinkin terms of when right now
people are using large languagemodels in an overpowered way for
everything.
Mm-hmm.
So you ask basically if you askChatGPT, oh, I want to go to
(20:53):
Rome it will consume half of theinternet to to find everything
about Rome and then generateyour answer.
I don't think we will do this insome years.
And so we would even help therebut what I call the high stakes
industries is obviously where weare the most important.
So you said healthcare already,then there's finance, there's
legal.
And then there's government or,when you have stuff defense or
(21:14):
something like that.
You know what I mean?
You don't want to be wrong.
Like in an assessment ofpolitical risks or anything.
Of course.
So that is stuff that'sobviously our first cases and
right now we see most interestfrom the legal fraction.
So because also they have longtexts, that are inherently
structured.
So the funny thing is peoplethink if I read a law, it has a
(21:35):
structure.
The problem is the structure isonly visible to the human eye.
Mm-hmm.
It's not encoded.
So if you have, for example,financial data, you can think of
it as a spreadsheet.
So you can tell the modelthere's the P&L and so on.
But if you have a legal text,people think it's codified.
Because they can read it.
But only they can read it.
Computers can't.
Brooke (21:56):
Right.
Dr. Kihm (21:56):
So what we basically do is we make unstructured
information structured so theLLM can understand it like the
human mind does.
And this is obviously the superquick win is legal.
I wouldn't say we become likelegal focused, but it is really
easy right now.
Then we go into other industrieswhere we will do, let's say
(22:17):
bespoke solutions.
So for example, if you have likecertain type of library, like
stuff that you've never seen,there's sometimes there was a
wind turbine energy engineer Iknow.
And he needs to fulfill allthese norms and they all have a
certain style and format andwhatever.
I just parsed the USConstitution, for example, as a
demo and the US constitutionsincluding all amendments.
(22:38):
And then I led a standard systemlose on the US Constitution
asking some questions.
And then later I did theretrieval with POMA and it was
more than 90% less tokens.
Wow.
Wow.
That's
Brooke (22:49):
incredible.
Dr. Kihm (22:51):
And we're really proud of this.
So we are now building somedemonstrators.
So we actually.
Don't have use cases externallythat I can talk about, but we
are building like this littledemonstration where you can
really see stuff like the USConstitution with some questions
and then what is the difference?
Or, I recently had, like tariffsis a thing right now, as we all
know.
So I.
Yes, downloaded all the tariffcontracts and then you ask a
(23:14):
question and you see how normalRAG models, they start
stumbling, putting informationmm-hmm.
from all sorts in it.
And then the POMA is like thelaser focused information that
you want.
And wow, that saves a lot ofenergy and also time, but it's
90% energy on motivation.
Brooke (23:29):
Wow, that's incredible.
I really commend you for thework that you're doing.
And how long do you feel likethere's gonna be more wide
adoption into this space?
Because right now we're allwalking around with these tools
in this technology using bigpowered brains for small powered
tasks.
So at what point, do more peopletake radical responsibility and
(23:54):
be more conscious.
Of what it is that they trulyneed.
When do you see that switchhappening?
Because right now it's just kindof the wild, wild west.
Dr. Kihm (24:03):
Yes.
I agree.
It's one little anecdote.
For example, people use like theconsumer tools like chat, GPT
for a lot of stuff.
I sometimes also do.
Yes.
And the funny thing is it alwayshas this kind of free for you
effect.
Like I even have some friends.
Oh yeah.
I just do everything with chat,GPT and even the free version,
which is, I have to tell people,guys you're building training
(24:25):
data.
If you don't pay for it, you'rethe product.
But also right now the AIcompanies, they operate at a
loss with these kind of cases.
On the other hand, when I gohardcore coding for a big
project and I need some AIagents for it, I see the API
pricing and it's a lot more.
Yesterday I went through 50euros of or$50 of of tokens.
Yeah, sometimes I have to, I'msorry because I needed to crunch
(24:47):
like a very large data set.
And then you see it and then yourealize, okay, the tipping point
becomes when the actual cost ispassed onto the user.
And then the user not only seesthe cost, but also sees, okay,
wow, what did I do?
It's a bit like loading my carfive times what I just burned
through.
Brooke (25:06):
And um,
Dr. Kihm (25:08):
and that is something, that then creates awareness.
But also I have to say.
For a long time now, we onlyused LLMs as solutions, so SLMs
are just coming up, there's alsovery, special purpose LLMs and
like dev trial, for example, bymistrial, you could run on your
computer and it helps youcoding.
There's some very fascinating,models between SLM and LLM and,
(25:31):
and I think, the more attractivethey become the more people will
use it.
And there's also then the finalprivacy dimension.
For example, on apple could becandidate where stuff runs on
phones later and not likeeverything is sent to the cloud
using a lot of bandwidthprocessed there by overpowered
models and then sent back.
I think once these tools becomemore available, then people will
(25:52):
use it because it will also bethe time when the big companies
will realize, okay, we cannotjust build one data center after
another to Yes, burn our ownmoney and the energy.
It doesn't make sense.
Brooke (26:04):
Absolutely.
The White House just releasedits, plan for ai and I know that
they've been working reallyclosely with a lot of these big
tech companies and they'rebuilding all of these data
centers all around the world.
And to your point scaling backon a lot of initial
environmental goals, and youbring up such a, an important
(26:25):
point because I know thatsometimes it comes down to
needing to place the burden onthe consumer and the client for
change to actually happen.
I know a really good example ofthis is the city of San
Francisco has its own rules andregulations in place when it
comes to recycling and wastemanagement and taxing.
(26:47):
And as a result, they're one ofthe most efficient cities in
America, because they put thatburden on, the person and how
many cans that they use andbenefits for recycling.
And so when that is put on theconsumer and the clients, like,
that's when change actuallyhappens.
(27:07):
So you bring up such animportant point there.
My next question is since youare essentially a very smart guy
who, builds tools to solvecomplicated problems and
well-versed in math andcomputers and, have already
tackled such an importantproblem we're currently facing
(27:29):
with the environmental impactand recalling data and
information.
What's next for you?
What are you ideating, becauseyou've had such a journey to
this point.
What does the future look like?
Dr. Kihm (27:44):
To be honest, this will be a very boring answer.
We are far from being done withPOMA.
I mean, POMA, we handed, wesubmitted the patent, we have
built the system, and first ofall, we now have to get it out
there.
The goal of POMA is to really bein every rack.
I won't go to bed before we canchunk everyone's library to
(28:04):
digestible chunks that willreduce their consumption.
So that still is a mission.
I mean, we are really not thereyet, to be honest.
We're still a small company.
We're growing fast and we have acool thing, but.
Honestly, my goal right now,what drives me is I want to see
POMA everywhere.
And the interesting part is, wedo an invisible stuff in an
obscure thing called rag.
(28:25):
And still, I think in some yearswe could touch everyone's life,
which is super crazy.
Brooke (28:31):
Yeah.
And
Dr. Kihm (28:31):
and this is still something that will drive me for
many years.
And then also what we see is.
Like you said, s SLMs and othermodels and other sources of
data, like for example.
I just, I had the problem withmy heat pump here and I re, I
wired it a bit into my homeautomation system so that now I
get this live feed of itself
Brooke (28:50):
And
Dr. Kihm (28:51):
stuff like this, like live information how you can
plug this in into yourintelligence systems that we
will see more and more.
It's a bit in the Antech Smarthome, but for like with ai.
There's so much stuff that willcome up that sounds funny, but
there's so much data and dataneeds chunking.
So I'm, yeah I'm obsessed withthis because there's the
leverage you get by chunking itwell is something Yeah.
(29:12):
It doesn't get old.
It's really, it's the beauty ofseeing it, it's still driving me
and.
I don't need another goal fornow.
Like this is really the carrotthat is so far and even if I'm
running faster and faster, it'sstill dangling, like at the same
distance.
So I am very motivated by thatand I don't think the journey is
over very soon.
So I'm actually cool with this,obviously.
(29:35):
You always extrapolate in thisdirection or another and so on.
And I have several ideas, but Iwouldn't act on any of them for
now because I really also chosePOMA from like a set of 20
concepts I developed over manyyears.
And I actually, I have my teamwhen we started this as an
incubator, like, okay, here'sthese five ideas and let's start
(29:56):
running with five of them inparallel and very quickly POMA
won out.
Brooke (30:01):
That's beautiful and quite the lofty goal for
everybody to be using rag andprops to you.
I think it's such a great goalto have and one that's going to
profoundly impact and shape theworld and the environment.
So I really commend you foreverything that you're doing at
POMA, and my next question foryou.
(30:24):
Because I love speaking to otherpeople in different countries.
Give me the insight of what'sgoing on in Germany when it
comes to ai.
Maybe I'm in a bit of a bubblemyself in the US, but it comes
up in almost every conversationI have now.
In every seminar or talk ormastermind, they talk about AI.
(30:48):
How is it being challenged oradopted?
What's the political climatearound AI?
Give me the insights of what'sgoing on over there.
Dr. Kihm (30:59):
Absolutely.
Absolutely.
That's it's a fascinating topic.
Also in terms of what thedifferences are and I always
like the different cultural,vectors that societies take on
topics like this.
Brooke (31:09):
Yes.
Dr. Kihm (31:09):
Germany is a technology nation, so obviously
Germany's not only philosophyaround ai.
I mean, if you look at the lastnames of several of the, AI
inventors, you see okay,alright.
And then you hear them withtheir Danish speaking you
realize, okay in terms ofdeveloping this stuff, Germany
was always strong.
And there's like an academia isvery well versed in the
theoretics of it, in terms ofapplying it.
(31:31):
We are always a bit late.
Like there's always, it's aclassic, like the German idea
gets some American money andthen it finally flies.
Okay.
And it's super funny.
That being said I mean Germanyis, I think it's catching up
quite well.
Um, There's a lot ofinitiatives.
There's one thing in Germanythat differentiates the
discussion from many others, andthat's the privacy angle.
(31:52):
I mean, for a good reason,Germans are not only stickers on
privacy, but don't forget likehalf of Germany was the biggest
spy state in the history ofmankind.
Mm-hmm.
Um.
Yeah, what many people alwaysoverlook that.
We had like 90% of people werebasically moonlighting as spies
in the east.
And
Brooke (32:09):
Wow, that
Dr. Kihm (32:10):
makes something with the society that you always fear
surveillance and privacy issues.
So obviously.
We always have this sovereignty,not in terms of we need to build
our own stuff, but it needs tobe on our server.
So for example, our customershere, they get a secure cloud,
the same models as like OpenAI,but on a German server, they
control it and so on.
(32:31):
That's a very big discussion.
So in Germany, if you buildsomething with AI and you say,
yeah, some abstract AIsomewhere, and then you're
already out of the door.
So everyone is about privacy,security, and in a political
discussion it's a bit moresafety related, and I kind of,
like that.
Yeah.
So it's more like, let's talkabout the potential harms.
I mean, not only thebioterrorist harms that everyone
(32:52):
knows, but can be somethingeven, more invisible.
Like, it can teach your childthat it basically doesn't like
itself that's something, yeah.
So all these societal harms,it's important to discuss them?
Yes, and we have many people whodiscuss them, and I deeply
cherish that.
Sometimes the problem it's a bitpartisan.
You have like pro AI and anti aiand I think in the US it's
(33:13):
similar, but you have these veryintelligent people in Germany
that are good philosophers aboutthe harms of ai.
And then you have the same forpro ai, fewer people are
thinking about the middleground.
Yeah, I think in terms ofgetting it right I wanna take
you up on this example with therecycling in San Francisco.
Yes.
When I look out of my househere, I see my three different
(33:34):
bins there.
And the funny thing is, inGermany they cost differently.
So the whatever bin is like 10times the price of the other
bin.
Yes.
Where you can only put paper orwhere you can only put cans and
Yeah.
If you put too much random stuffin the canned bin, then
obviously they take it from you.
So I love these incentives andlike a price on CO2 and then
that, you know the market,decide the rest.
(33:55):
If I get punished or rewardedfor my behavior, then I will
adapt and the Germans, theysometimes get this right, but
they sometimes wanna regulate.
So
Brooke (34:04):
yes.
Dr. Kihm (34:05):
If you start an AI company in Germany the problem
is not the AI regulation, butlike, everything else the red
tape, and I think now that AIaccelerates so much stuff, so
hardcore mean people startcompanies overnight and so on.
I can build a company with thehelp of AI overnight, but then
need eight weeks to incorporateit, or, Yeah.
(34:26):
People have people think in afaster way because of ai.
Now.
There's like these companies whohelp digitize or, transform your
company into the age of ai.
They are running hot in Germanyright now, which is a good sign.
Brooke (34:39):
Thank you so much for that insight.
And yeah, it makes so much sensethat you bring of just the
history of Germany and Americacould do a lot better when it
comes to things, compared toEurope who uses GDPR and just
privacy and securing customerdata.
One thing I want to close withis give you the space to share
(35:03):
one main takeaway you'd lovelisteners to get from this
episode.
Dr. Kihm (35:09):
There's a lot of cool tools people can use.
It's very promising.
It is the answers to all yourquestions.
You can just build a softwareovernight.
You don't have to bring anythingto the table.
And I love the empowerment ofthis, that you can actually.
Do whatever you want, but Iwould reformulate it as that.
You can try everything out.
Don't be overconfident in.
(35:29):
Oh, cool.
I just build an app with a chat,and I don't know nothing about
coding.
I like that you could build aprototype, but don't take ai.
Like at face value yet becauseit's really dangerous to overly
on this stuff.
It's a bit like going to courtwith something just formulated
together and if you use the deepresearch mode and let AI grind a
(35:51):
lot, be aware that.
This is a lot of work that youmight now get for a low price,
but that's not sustainable andit will change at one point.
And if ever you build a library,call us.
Brooke (36:06):
Very key points indeed.
Thank you for sharing that.
And thank you so much for yourtime today.
You have such a vast backgroundand experience so I really
enjoyed getting to learn moreabout you and the amazing work
that you've done.
How can listeners get in touchwith you if they wanna connect?
Dr. Kihm (36:27):
Yes.
We are old school founders, sothere's still our website,
poma-ai.com where we also havethe sign up for our newsletters.
There we also have linked ourmedium articles where we
describe in more detail how POMAworks and what we do.
We will come up with morecontent where now we focus a lot
of on development.
We also are on LinkedIn where weencourage people to follow us
(36:49):
and catch up on our newestdevelopments.
Brooke (36:51):
Great.
Dr. Kihm (36:51):
And then otherwise I maybe see you again here.
Yes, the next product.
Brooke (36:55):
Absolutely, I would love to have a version two
conversation down the line andalways an open door and happy to
connect with you.
Thank you again so much for thisconversation.
It was a little bit higher levelof understanding, but I'm hoping
that my listeners at this pointhave been growing and learning
and they're ready for that nextlevel of understanding.
(37:18):
LLMs and how they're using yourinformation to give you the
answers.
And your final note aboutdiscernment when it comes to
accepting the information thatwe receive from AI is just so
important and rings true for meas well.
So thank you again Dr.
Kihm, so much.
Dr. Kihm (37:36):
Thank you.
Brooke (37:37):
Wow I hope today's episode opened your mind to
what's possible with AI.
Do you have a cool use case onhow you're using AI and wanna
share it?
DM me.
I'd love to hear more andfeature you on my next podcast.
Until next time, here's toworking smarter, not harder.
See you on the next episode ofHow I AI This episode was made
(37:58):
possible in partnership with theCollective AI, a community
designed to help entrepreneurs,creators, and professionals
seamlessly integrate AI intotheir workflows.
One of the biggest game changersin my own AI journey was joining
this space.
It's where I learned, connectedand truly enhanced my
understanding of what's possiblewith ai.
(38:20):
And the best part, they offermultiple membership levels to
meet you where you are.
Whether you want to DIY, your AIlearning or work with a
personalized AI consultant foryour business, The Collective
has you covered.
Learn more and sign up using myexclusive link in the show
notes.
Welcome to How I AI the podcast featuring real people,
real stories, and real AI inaction.
I'm Brooke Gramer, your host andguide on this journey into the
real world impact of artificialintelligence.
For over 15 years, I've workedin creative marketing events and
business strategy, wearing allthe hats.
(00:24):
I know the struggle of trying toscale and manage all things
without burning out, but here'sthe game changer, AI.
This isn't just a podcast.
How I AI is a community, a spacewhere curious minds like you
come together, share ideas, andI'll also bring exclusive
discounts, and insiderresources, because AI isn't just
(00:47):
a trend, it's a shift, and thesooner we embrace it, the more
freedom, creativity, andopportunities will unlock.
How I AI is brought to you inpartnership with the Collective
AI, a space designed toaccelerate your learning and AI
adoption.
I joined the collective and it'scompletely catapulted my
(01:09):
learning, expanded my network,and showed me what's possible
with AI.
Whether you're just startingout, seeking community or want
done for you solutions, thecollective gives you the
resources to grow your businesswith AI.
So stay tuned to learn more atthe end of this episode, or
check my show notes for myexclusive invite link..
(01:31):
Hi everyone.
I just sat down with Dr.
Alexander Kihm.
He's the founder of POMA.
His work is reshaping how largelanguage models operate, and
more importantly, how they'reconsuming resources.
In this episode, Alex explainshow POMA's way of using RAG and
(01:51):
chunking, which we'll get intowhat that is later, helps AI
give better answers while usingup to 80% less energy than usual
methods..
As more people are starting touse AI every day, it's important
to connect the way we run AIwith the bigger problem of the
(02:12):
world's energy and resourcecrisis.
He also shares fascinatinginsight into how Germans are
approaching AI adoption withprivacy as the top priority.
If you care about AI's potentialand its impact on our planet,
this episode will give you awhole new lens on both.
(02:33):
Alright, let's dive into today'sepisode.
Hi everyone.
Welcome to another episode ofHow I AI.
I'm your host, Brooke Gramer.
Today I'm joined by Dr.
Alexander Kihm.
He's an engineer, economist, andserial entrepreneur who's
tackling AI's biggest challengeswith POMA, a breakthrough
(02:54):
solution that reduceshallucinations and energy in
large language models.
Dr.
Kihm, thank you.
Welcome.
Dr. Kihm (03:03):
Thanks for having me.
Brooke (03:05):
Yes, I'd love to open this space and hear more about
you.
Please share about yourbackground and how you ended up
doing what you do today.
Dr. Kihm (03:15):
Yes, sure.
I'm an engineer by trainingoriginally um, where lots of
Germans are.
That's always the joke, but it'sstill kind of true.
And I played with LEGO veryearly.
And I went to legal techniquevery early on uh, and then I had
the luck of getting a hand medown computer from my parents
when it was already I think fouryears old or something.
At five I dis assembled it.
(03:36):
At six I started coding.
It's kind of my passion.
So it became kind of obvious uh,decide that I want to become an
engineer.
And there's this program inGermany, I mean now with
Bachelor and Masters like yougot in the US but back then it
was diploma.
So it's combined Bachelor andmaster's in one..
Brooke (03:53):
Okay.
And
Dr. Kihm (03:53):
then, for the industrial engineering degree,
you could basically choosewhatever you want and I chose a
lot of energy engineering, sofrom nuclear power plants,
conventional plants, and thensolar cells, wind turbines.
I, I had learned it all andtowards the end of my studies I
realized okay, if you want toreally have an impact, and the
maximum you can get there isbecome the guy responsible for
(04:14):
certain screws and certain microdevices that are inside a big
power plant.
I love what I learned, but Irealized that won't be my career
path.
Mm.
Brooke (04:22):
Um,
Dr. Kihm (04:24):
And back then I already had run a little
startup.
I started when I turned 18together with my neighbor And it
was what later turned out tobecome Germany's first and one
of the largest legal techcompanies.
Brooke (04:34):
Wow.
Dr. Kihm (04:35):
And we are still running it, it's basically a
lawyers network.
So we help lawyers beingreplaced for court appointments
by other lawyers.
So whenever you have this modernlegal tech like you, you just
click a form online and then thenext day there's a lawyer suing
someone for you in court.
That's basically us behind it inthe API that we can automate it.
(04:55):
And I realized, okay, I like torun companies.
That's more like my thing.
But first I wasn't done witheducation, so I went to the
German Aerospace center and didmy PhD there.
Could leverage a bit of myenergy background co-developing
an electric car.
I actually have developed thefirst plugin hybrid car in
Germany.
That was my project.
Yeah, it was super exciting.
It was really nice.
I learned a lot.
And actually during that time Itransitioned more and more from
(05:19):
classic engineering questions,more towards big data, large
scale coding data questions.
Brooke (05:25):
Mm-hmm.
Um,
Dr. Kihm (05:26):
So in the end we did the forecast model for, okay, we
have this car, now we understandthis car.
How will it, penetrate themarket?
What is the forecast forelectric cars?
Depending on fuel prices and soon.
And I was really proud for whatwe built in the end because we
had this model that used verytraditional approaches, but on a
very large scale to predict ifwe now have these incentives or
(05:46):
these taxes or whatnot, how manypeople will drive the electric
cars and, back then, it was thefirst time I tried to use AI or
what was called AI back then.
Mm-hmm.
I would rather call it ad likeartificial dumbness.
Um, uh, What we had back thenwas like neural networks.
But for bigger purposes, itwasn't fit.
So that was still my firstcontact with very large models,
(06:11):
but not like AI models.
I realized after my PhD that, Ilearned a lot, but I'm no one
that works in big organizationsscience is very cool, but it's
still normally run by bigorganizations,
Brooke (06:23):
right.
Dr. Kihm (06:23):
I've started founding my second company which is a
FinTech.
So we disrupted the Germanpension market, we did some
financial innovation there,combined insurance solutions and
fund management stuff.
And in 2019 we sold it toRaisin, which now is one of, the
larger fintechs we are even inthe US now.
And raisin is pretty successful.
Pretty large company.
(06:44):
I'm still very proud of thecombined solution we have there
and the wealth managementplatform.
But obviously after some yearsof, integrating this stuff and
helping and so on, it startednagging at me.
Brooke (06:55):
Mm-hmm.
Dr. Kihm (06:55):
Like, uh, okay.
You need a new challenge.
And it was around 2022, GPT, Ithink between two and three,
where you saw the first time.
Oh, wow.
That's something different thanthe cat detector models we had
before.
Brooke (07:09):
Yes.
Dr. Kihm (07:09):
There's something coming up and I'm fascinated by
LLMs.
Long story short, 2023, westarted dabbling a bit more
seriously.
Of what we can do with thisstuff.
Brooke (07:19):
Mm-hmm.
And,
Dr. Kihm (07:20):
um, Given that I told you the background and some
certain legal text, we thought,okay, we have already have
10,000 law firms as customers,let's try to build something for
them.
So what could they use?
How could they make use of thesenew language models and um.
we quickly realized what somepeople realized painfully that
if you just ask the standardversion of, you remember ChatGPT
(07:41):
back then when it came out.
Yeah.
Can you help me?
This and that.
I have this thing to present incourt.
It would just make up things.
Brooke (07:47):
And,
Dr. Kihm (07:47):
Everyone knows the story or the stories by now
plural of several lawyersgetting disbarred or whatnot
because they just enter courtwith made up stuff.
Brooke (07:57):
Mm-hmm.
Dr. Kihm (07:57):
And people thought, okay, maybe we train the models
more and more.
And it didn't help.
Even the new reasoning models,they hallucinate even more
because they are in this kind ofecho chamber with themselves.
Others who thought fine tuningworks didn't either because it's
made for something totallydifferent.
And then there was the RAGscene.
The so-called rag scene isretrieval augmented generation.
Brooke (08:19):
Yes.
Dr. Kihm (08:19):
And it means you avoid hallucinations by giving the
models some context for yourquestions.
So basically, here's the book.
Answer a question based on thisbook.
And I found this very promising.
I found this very, verypromising.
And I still think the idea isabsolutely right to have the
intelligence from training thestyle and the, let's say
(08:41):
craftsmanship from fine tuning,but then the information from
augmenting the retrieval oraugmenting the question with
some retrieved context.
Brooke (08:50):
I wanna pause you on, pause on please.
Pause.
Pause right there.
Because for those who arelistening, First of all, wow,
incredible amount of experience.
You're still so young and you'veaccomplished so much.
And for those who might notunderstand what RAG is and the
importance of what you're just.
(09:10):
Noting could you maybe break itdown on Oh, absolutely.
Why this is important and how itrelates to hallucination?
Dr. Kihm (09:16):
Yes, absolutely.
I'm sorry I skipped that part.
Yeah.
So it basically means uh, let'sgo back to this first lawyer and
first ChatGPT So you just ask amodel a question and the model.
Relies only on what it haslearned.
So it was trained on a lot ofstuff, the whole of Wikipedia,
55,000 unpublished books andwhatnot.
And it actually helped a lot todevelop some sort of
(09:39):
intelligence.
It's fascinating.
So the thing is veryintelligent, but think of a
child that went through 10universities but doesn't have
any books anymore.
So the child must now be quiteintelligent.
But when you ask it like acertain question, like, okay.
How does this work?
It just connects some dots, butit doesn't have the primary
information in front of it, soit can just only go so far.
(10:02):
And this leads to thesehallucinations we are all
talking about.
Like, it just comes up with AlexKihm is a geographer from the
18th century because it somehowconnects Alex and my last name
and geography for some reason.
Brooke (10:14):
Okay.
And
Dr. Kihm (10:14):
it's not me.
This is the stuff that comes upwhen you just rely on training.
Brooke (10:18):
Yes.
Dr. Kihm (10:18):
And what RAG does?
RAG stands for Retrievalaugmented Generation.
So generation means when you aska question, you basically just
say the tell the model.
Okay, here's my question andbased on all these words that is
my question.
Generate an answer.
So the generation meansbasically answering.
And if you say retrievalaugmented generation, that means
(10:40):
you augment the generation byretrieving some stuff first.
Which means like, okay, I havemy library here.
And I say, please my child,here's this book I found on the
topic of X.
Can you please dissect it for meand answer my question based on
what you see in this book?
And so like this, I can rely onthe ground truth, knowledge,
(11:01):
that here's the wisdom, there'sthe intelligence, please
Intelligent child, take thisbook and then tell me what the
answer to my question is.
That is basically rag in verybroad terms.
But it's still actually what RAGdoes.
It's
Brooke (11:14):
yeah,
Dr. Kihm (11:15):
it's quite accurate to be honest.
Brooke (11:17):
Thank you.
Yeah, thank you so much forbreaking that down.
Because I think it's soimportant to talk about what RAG
is and I Yes.
And I know that you've alsomentioned the term chunking.
Yes.
Is that the same thing as Rag?
Dr. Kihm (11:31):
Let's say it's one of the building blocks.
Okay.
So what is interesting, and thatalso relates to our history how
we got to POMA.
Brooke (11:37):
So
Dr. Kihm (11:37):
at first we thought RAG is a solution.
So we just build a rag stack.
There's several softwares for itand people who are interested.
You can just Google, forexample, LlamaIndex or a Lang
chain.
You just Google how to set up arag.
There's super nice articles andtutorials about it, it all
relates to this talk to yourdata.
It's always, I have data, I wanta chat bot or an answer
(11:58):
generation machine of some sort.
And I recommend to everyone tojust dabble with it a bit.
And when you do that, you findout the same thing that we found
out.
And that is, okay, I have thislibrary of stuff like articles,
PDFs, whatever texts.
Then when I try RAG what itdoes, it, it gives me the pieces
of information that arenecessary for the question.
(12:19):
And at first you don't see anychunking.
This is the interesting part.
It's an invisible part of thiswhole pipeline.
And when we tried our first,little dabbling legal bot, we
realized that answering onequestion would cost like around
$1.
Which is kind of much for us.
I mean, just the demo night costme like my monthly rent.
And I realized, wow, what ishappening there?
(12:40):
Why is this so expensive?
And I found out when you lookinto, a bit deeper in the logs,
you find that okay.
It just put everything thatremotely relates to the
question.
It just puts into this, here, mychild here is this backpack full
of information, and I only wantto tell you what is pasta made
of?
And so this is dependent on thechunking because when you ingest
(13:05):
data into RAGs, so let's say youbuild your library, you have all
these books, and then you wantto get them into the library for
your RAG.
This is called ingestion.
But what you don't know when youjust start out with it is that
it's not like, here's the wholebook and it somehow magically
gets the whole content into thisdatabase without you thinking
(13:26):
how it works.
But what it secretly does is itchops up the information.
So it basically tears out pages.
So you could say it tears outpages.
And then each page gets what iscalled embedded, so represented
by mathematical information tofind it later, To find them
later?
You have to somehow index it andthey have to index each page.
You with me so far?
Brooke (13:46):
Yes.
Works for
Dr. Kihm (13:47):
people.
Yes.
Brooke (13:48):
I've heard rag and chunking explained.
And it's.
Interesting to hear behind thescenes what our ais are doing
when we're using them yeah,particularly LLMs.
And you bring up such importanttopics of why we're honing in on
this and why it's so importantis one, AI is very expensive and
(14:09):
two it takes up a lot ofresources and energy, which Yes.
Is so important because a lot ofpeople have caused quite the
concern for using these toolsand technology.
So one thing I'd really love toexplore with you and all the
work you're doing at POMA is theenvironmental side of all of
(14:32):
this.
Yes.
'cause, you know, a lot ofpeople are talking about
hallucinations with AI andyou've been able to successfully
find solutions, but they're notnecessarily connecting that to
the energy and resourceconsumption.
Yes.
That you are helping with.
So if you wanna expand on that,since you've been in the AI and
(14:52):
the tech space for so long andyou've seen this progression and
how we're becoming moreefficient today, I'd love to
start there.
Dr. Kihm (15:00):
Absolutely.
Yes.
Also, one of the factors reallydriving us we have a big
advantage here, and that is thecost that people are talking
about.
So if you have people who don'tcare about the environment,
they're just purely, costoptimizers, yes.
You can even tell them, whatyou're paying for.
Yes, there is a big margin, butbasically the only cost of AI is
actually resources, is energyconsumption.
(15:21):
The good thing is we canactually, make both happy.
We can make the controllershappy because it's cheaper, but
at the same time, we save theenvironment because it's
directly proportional.
This is the cool thing here.
That being said like forexample, when I told you about
this demo night back then youpaid like, let's say a dollar
per question because of all thestuff you gave to each question
and the more you give them inyour question, the more
(15:43):
resources the AI consumes.
This is like the trade off here.
So if you just ask it, it'squite efficient, but it makes
stuff up.
Brooke (15:50):
And now
Dr. Kihm (15:51):
you have this bad trade off that, okay, I want to
have the truth and not made ofstuff, but if I overburden it
with information.
I mean, literally every wordcounts.
All is, for example, if you aska very long question, it takes
up like five times the energy ofa question of the fifth, that
size.
This is, few people know this,but wow.
Yeah.
This is the interesting part.
(16:11):
At least the reading phase ofthe ai then it starts answering.
So if you want a long answer, italso consumes more.
And if you just say yes or no.
Very efficient, very German.
But it's that's the interestingpart.
So each word, or to be honest,it's called token.
It's like half a word.
It's like a syllable.
Brooke (16:25):
Mm-hmm.
Dr. Kihm (16:26):
The thing is more there, more energy, simple as
that, and models become a bitcheaper.
People see this over time, andthis is because they do some
shortcuts.
They do clever optimization.
But in the end, and this is alsowhy, I always answer, why we, we
will still be there in 10 years.
In the end, there's only so faryou can get to energy
consumption per token, and itwill always be there.
(16:48):
So it will, and it becomesincreasingly important.
The more people, use ai, themore tokens are burned.
We always call it burn tokens,that if imagine for every
question you have in your littlechatbot assistant here and there
and here and whatnot, imagineyou always burn like a book.
It's crazy.
It's wow.
It's something that if thatscales, we really have a
(17:09):
problem.
People are already building newnuclear plants and, forfeit
their green goals and whatnot.
Because now there's so muchenergy that is needed right now.
Yes.
And the big lever to that isjust put less.
Less ballast into questions itmakes sense to have good quality
answers.
And this is based on how muchcontext you give.
(17:29):
But if you give pointlesscontext, like for example, I
have a question on, I want totalk like Ted, so I take this
book, but I only want my intro,so I would normally just go for
the intro and not like, give thewhole book to the AI.
Just doesn't make sense.
And right now, rags have thiskind of, I call it the Dodge Ram
approach.
So not very surgical, but justblast through the door and
(17:50):
here's all the information andthat is inefficient resource
burning.
And so the good thing is that wekind of have a little advantage
even for, against those peoplewho say, who cares?
And that is, the answers evenget better.
Brooke (18:04):
So like
Dr. Kihm (18:05):
always imagine I giving you a whole book for one
little question.
You have to go through thiswhole thing and it's exhausting.
It's not only good for theenvironment, but also better for
you if I give you the surgicalinformation.
Brooke (18:16):
Yes.
You bring up something that justsparked in me because I've heard
a lot.
First of all, thank you so muchfor expanding on that.
Small language models.
Yeah.
Can you explain the differencebetween small language models
and large language models yeah.
Let's start there.
Dr. Kihm (18:32):
Yes.
Yes.
I'm fascinated by them.
I'm very fascinated and actuallywe in our internal processes,
like what we do with POMA, Iwill expand on that later.
We actually use some sort ofwell smallish language model.
And I think they are the futurefor certain purpose tasks,
especially this agentic stuff.
So like if you.
Only book flights for people.
(18:52):
You don't need to have readgooder for it.
So that obviously, translatesperfect to certain focused task
models and small language modelsare especially nice if you give
them context.
So you don't need a PhD academicto recite stuff from recipes.
That's basically the idea behindsmall language models combined
with RAG.
So if you have a certain.
(19:15):
Task and you have theinformation to fulfill this
task.
It's a bit like, Hey, can youcook this?
And here's the recipe.
You don't need a PhD for that.
Brooke (19:22):
Mm.
And
Dr. Kihm (19:22):
That's the idea behind small language models and I
think there will be many of themand they will all be RAG powered
because you will never use asmall language model and just
give it too much freedom ofgenerating an answer with
whatever little.
Intelligence it has.
Brooke (19:38):
But
Dr. Kihm (19:38):
I think this combination, and you're already
seeing this come up likepurpose-built models, smaller
ones that then have the perfectinformation and they don't need
this overpowered brain.
And they are very fast, they'revery efficient.
And I think that is for certaintasks is the future.
Brooke (19:54):
That's beautiful.
So it sounds like POMA issolving a lot of problems, that
is a very
Dr. Kihm (20:00):
simple thing.
Yeah.
Brooke (20:01):
Yeah.
It, it helps AI give betteranswers using less energy and
organizing the information in away that's easier for it to not
hallucinate.
And this must be really usefulfor industries like healthcare
where being accurate reallymatters.
(20:23):
Like, could you describe yourtypical client that you are
working with and maybe anysuccessful case studies?
Dr. Kihm (20:31):
So generally I mean, I always have this discussion with
my team that I would say we canhelp everyone.
But that's always kind of myattitude.
I love stuff that, can begenerally important and
impactful.
Yes.
But don't get me wrong, I thinkin terms of when right now
people are using large languagemodels in an overpowered way for
everything.
Mm-hmm.
So you ask basically if you askChatGPT, oh, I want to go to
(20:53):
Rome it will consume half of theinternet to to find everything
about Rome and then generateyour answer.
I don't think we will do this insome years.
And so we would even help therebut what I call the high stakes
industries is obviously where weare the most important.
So you said healthcare already,then there's finance, there's
legal.
And then there's government or,when you have stuff defense or
(21:14):
something like that.
You know what I mean?
You don't want to be wrong.
Like in an assessment ofpolitical risks or anything.
Of course.
So that is stuff that'sobviously our first cases and
right now we see most interestfrom the legal fraction.
So because also they have longtexts, that are inherently
structured.
So the funny thing is peoplethink if I read a law, it has a
(21:35):
structure.
The problem is the structure isonly visible to the human eye.
Mm-hmm.
It's not encoded.
So if you have, for example,financial data, you can think of
it as a spreadsheet.
So you can tell the modelthere's the P&L and so on.
But if you have a legal text,people think it's codified.
Because they can read it.
But only they can read it.
Computers can't.
Brooke (21:56):
Right.
Dr. Kihm (21:56):
So what we basically do is we make unstructured
information structured so theLLM can understand it like the
human mind does.
And this is obviously the superquick win is legal.
I wouldn't say we become likelegal focused, but it is really
easy right now.
Then we go into other industrieswhere we will do, let's say
(22:17):
bespoke solutions.
So for example, if you have likecertain type of library, like
stuff that you've never seen,there's sometimes there was a
wind turbine energy engineer Iknow.
And he needs to fulfill allthese norms and they all have a
certain style and format andwhatever.
I just parsed the USConstitution, for example, as a
demo and the US constitutionsincluding all amendments.
(22:38):
And then I led a standard systemlose on the US Constitution
asking some questions.
And then later I did theretrieval with POMA and it was
more than 90% less tokens.
Wow.
Wow.
That's
Brooke (22:49):
incredible.
Dr. Kihm (22:51):
And we're really proud of this.
So we are now building somedemonstrators.
So we actually.
Don't have use cases externallythat I can talk about, but we
are building like this littledemonstration where you can
really see stuff like the USConstitution with some questions
and then what is the difference?
Or, I recently had, like tariffsis a thing right now, as we all
know.
So I.
Yes, downloaded all the tariffcontracts and then you ask a
(23:14):
question and you see how normalRAG models, they start
stumbling, putting informationmm-hmm.
from all sorts in it.
And then the POMA is like thelaser focused information that
you want.
And wow, that saves a lot ofenergy and also time, but it's
90% energy on motivation.
Brooke (23:29):
Wow, that's incredible.
I really commend you for thework that you're doing.
And how long do you feel likethere's gonna be more wide
adoption into this space?
Because right now we're allwalking around with these tools
in this technology using bigpowered brains for small powered
tasks.
So at what point, do more peopletake radical responsibility and
(23:54):
be more conscious.
Of what it is that they trulyneed.
When do you see that switchhappening?
Because right now it's just kindof the wild, wild west.
Dr. Kihm (24:03):
Yes.
I agree.
It's one little anecdote.
For example, people use like theconsumer tools like chat, GPT
for a lot of stuff.
I sometimes also do.
Yes.
And the funny thing is it alwayshas this kind of free for you
effect.
Like I even have some friends.
Oh yeah.
I just do everything with chat,GPT and even the free version,
which is, I have to tell people,guys you're building training
(24:25):
data.
If you don't pay for it, you'rethe product.
But also right now the AIcompanies, they operate at a
loss with these kind of cases.
On the other hand, when I gohardcore coding for a big
project and I need some AIagents for it, I see the API
pricing and it's a lot more.
Yesterday I went through 50euros of or$50 of of tokens.
Yeah, sometimes I have to, I'msorry because I needed to crunch
(24:47):
like a very large data set.
And then you see it and then yourealize, okay, the tipping point
becomes when the actual cost ispassed onto the user.
And then the user not only seesthe cost, but also sees, okay,
wow, what did I do?
It's a bit like loading my carfive times what I just burned
through.
Brooke (25:06):
And um,
Dr. Kihm (25:08):
and that is something, that then creates awareness.
But also I have to say.
For a long time now, we onlyused LLMs as solutions, so SLMs
are just coming up, there's alsovery, special purpose LLMs and
like dev trial, for example, bymistrial, you could run on your
computer and it helps youcoding.
There's some very fascinating,models between SLM and LLM and,
(25:31):
and I think, the more attractivethey become the more people will
use it.
And there's also then the finalprivacy dimension.
For example, on apple could becandidate where stuff runs on
phones later and not likeeverything is sent to the cloud
using a lot of bandwidthprocessed there by overpowered
models and then sent back.
I think once these tools becomemore available, then people will
(25:52):
use it because it will also bethe time when the big companies
will realize, okay, we cannotjust build one data center after
another to Yes, burn our ownmoney and the energy.
It doesn't make sense.
Brooke (26:04):
Absolutely.
The White House just releasedits, plan for ai and I know that
they've been working reallyclosely with a lot of these big
tech companies and they'rebuilding all of these data
centers all around the world.
And to your point scaling backon a lot of initial
environmental goals, and youbring up such a, an important
(26:25):
point because I know thatsometimes it comes down to
needing to place the burden onthe consumer and the client for
change to actually happen.
I know a really good example ofthis is the city of San
Francisco has its own rules andregulations in place when it
comes to recycling and wastemanagement and taxing.
(26:47):
And as a result, they're one ofthe most efficient cities in
America, because they put thatburden on, the person and how
many cans that they use andbenefits for recycling.
And so when that is put on theconsumer and the clients, like,
that's when change actuallyhappens.
(27:07):
So you bring up such animportant point there.
My next question is since youare essentially a very smart guy
who, builds tools to solvecomplicated problems and
well-versed in math andcomputers and, have already
tackled such an importantproblem we're currently facing
(27:29):
with the environmental impactand recalling data and
information.
What's next for you?
What are you ideating, becauseyou've had such a journey to
this point.
What does the future look like?
Dr. Kihm (27:44):
To be honest, this will be a very boring answer.
We are far from being done withPOMA.
I mean, POMA, we handed, wesubmitted the patent, we have
built the system, and first ofall, we now have to get it out
there.
The goal of POMA is to really bein every rack.
I won't go to bed before we canchunk everyone's library to
(28:04):
digestible chunks that willreduce their consumption.
So that still is a mission.
I mean, we are really not thereyet, to be honest.
We're still a small company.
We're growing fast and we have acool thing, but.
Honestly, my goal right now,what drives me is I want to see
POMA everywhere.
And the interesting part is, wedo an invisible stuff in an
obscure thing called rag.
(28:25):
And still, I think in some yearswe could touch everyone's life,
which is super crazy.
Brooke (28:31):
Yeah.
And
Dr. Kihm (28:31):
and this is still something that will drive me for
many years.
And then also what we see is.
Like you said, s SLMs and othermodels and other sources of
data, like for example.
I just, I had the problem withmy heat pump here and I re, I
wired it a bit into my homeautomation system so that now I
get this live feed of itself
Brooke (28:50):
And
Dr. Kihm (28:51):
stuff like this, like live information how you can
plug this in into yourintelligence systems that we
will see more and more.
It's a bit in the Antech Smarthome, but for like with ai.
There's so much stuff that willcome up that sounds funny, but
there's so much data and dataneeds chunking.
So I'm, yeah I'm obsessed withthis because there's the
leverage you get by chunking itwell is something Yeah.
(29:12):
It doesn't get old.
It's really, it's the beauty ofseeing it, it's still driving me
and.
I don't need another goal fornow.
Like this is really the carrotthat is so far and even if I'm
running faster and faster, it'sstill dangling, like at the same
distance.
So I am very motivated by thatand I don't think the journey is
over very soon.
So I'm actually cool with this,obviously.
(29:35):
You always extrapolate in thisdirection or another and so on.
And I have several ideas, but Iwouldn't act on any of them for
now because I really also chosePOMA from like a set of 20
concepts I developed over manyyears.
And I actually, I have my teamwhen we started this as an
incubator, like, okay, here'sthese five ideas and let's start
(29:56):
running with five of them inparallel and very quickly POMA
won out.
Brooke (30:01):
That's beautiful and quite the lofty goal for
everybody to be using rag andprops to you.
I think it's such a great goalto have and one that's going to
profoundly impact and shape theworld and the environment.
So I really commend you foreverything that you're doing at
POMA, and my next question foryou.
(30:24):
Because I love speaking to otherpeople in different countries.
Give me the insight of what'sgoing on in Germany when it
comes to ai.
Maybe I'm in a bit of a bubblemyself in the US, but it comes
up in almost every conversationI have now.
In every seminar or talk ormastermind, they talk about AI.
(30:48):
How is it being challenged oradopted?
What's the political climatearound AI?
Give me the insights of what'sgoing on over there.
Dr. Kihm (30:59):
Absolutely.
Absolutely.
That's it's a fascinating topic.
Also in terms of what thedifferences are and I always
like the different cultural,vectors that societies take on
topics like this.
Brooke (31:09):
Yes.
Dr. Kihm (31:09):
Germany is a technology nation, so obviously
Germany's not only philosophyaround ai.
I mean, if you look at the lastnames of several of the, AI
inventors, you see okay,alright.
And then you hear them withtheir Danish speaking you
realize, okay in terms ofdeveloping this stuff, Germany
was always strong.
And there's like an academia isvery well versed in the
theoretics of it, in terms ofapplying it.
(31:31):
We are always a bit late.
Like there's always, it's aclassic, like the German idea
gets some American money andthen it finally flies.
Okay.
And it's super funny.
That being said I mean Germanyis, I think it's catching up
quite well.
Um, There's a lot ofinitiatives.
There's one thing in Germanythat differentiates the
discussion from many others, andthat's the privacy angle.
(31:52):
I mean, for a good reason,Germans are not only stickers on
privacy, but don't forget likehalf of Germany was the biggest
spy state in the history ofmankind.
Mm-hmm.
Um.
Yeah, what many people alwaysoverlook that.
We had like 90% of people werebasically moonlighting as spies
in the east.
And
Brooke (32:09):
Wow, that
Dr. Kihm (32:10):
makes something with the society that you always fear
surveillance and privacy issues.
So obviously.
We always have this sovereignty,not in terms of we need to build
our own stuff, but it needs tobe on our server.
So for example, our customershere, they get a secure cloud,
the same models as like OpenAI,but on a German server, they
control it and so on.
(32:31):
That's a very big discussion.
So in Germany, if you buildsomething with AI and you say,
yeah, some abstract AIsomewhere, and then you're
already out of the door.
So everyone is about privacy,security, and in a political
discussion it's a bit moresafety related, and I kind of,
like that.
Yeah.
So it's more like, let's talkabout the potential harms.
I mean, not only thebioterrorist harms that everyone
(32:52):
knows, but can be somethingeven, more invisible.
Like, it can teach your childthat it basically doesn't like
itself that's something, yeah.
So all these societal harms,it's important to discuss them?
Yes, and we have many people whodiscuss them, and I deeply
cherish that.
Sometimes the problem it's a bitpartisan.
You have like pro AI and anti aiand I think in the US it's
(33:13):
similar, but you have these veryintelligent people in Germany
that are good philosophers aboutthe harms of ai.
And then you have the same forpro ai, fewer people are
thinking about the middleground.
Yeah, I think in terms ofgetting it right I wanna take
you up on this example with therecycling in San Francisco.
Yes.
When I look out of my househere, I see my three different
(33:34):
bins there.
And the funny thing is, inGermany they cost differently.
So the whatever bin is like 10times the price of the other
bin.
Yes.
Where you can only put paper orwhere you can only put cans and
Yeah.
If you put too much random stuffin the canned bin, then
obviously they take it from you.
So I love these incentives andlike a price on CO2 and then
that, you know the market,decide the rest.
(33:55):
If I get punished or rewardedfor my behavior, then I will
adapt and the Germans, theysometimes get this right, but
they sometimes wanna regulate.
So
Brooke (34:04):
yes.
Dr. Kihm (34:05):
If you start an AI company in Germany the problem
is not the AI regulation, butlike, everything else the red
tape, and I think now that AIaccelerates so much stuff, so
hardcore mean people startcompanies overnight and so on.
I can build a company with thehelp of AI overnight, but then
need eight weeks to incorporateit, or, Yeah.
(34:26):
People have people think in afaster way because of ai.
Now.
There's like these companies whohelp digitize or, transform your
company into the age of ai.
They are running hot in Germanyright now, which is a good sign.
Brooke (34:39):
Thank you so much for that insight.
And yeah, it makes so much sensethat you bring of just the
history of Germany and Americacould do a lot better when it
comes to things, compared toEurope who uses GDPR and just
privacy and securing customerdata.
One thing I want to close withis give you the space to share
(35:03):
one main takeaway you'd lovelisteners to get from this
episode.
Dr. Kihm (35:09):
There's a lot of cool tools people can use.
It's very promising.
It is the answers to all yourquestions.
You can just build a softwareovernight.
You don't have to bring anythingto the table.
And I love the empowerment ofthis, that you can actually.
Do whatever you want, but Iwould reformulate it as that.
You can try everything out.
Don't be overconfident in.
(35:29):
Oh, cool.
I just build an app with a chat,and I don't know nothing about
coding.
I like that you could build aprototype, but don't take ai.
Like at face value yet becauseit's really dangerous to overly
on this stuff.
It's a bit like going to courtwith something just formulated
together and if you use the deepresearch mode and let AI grind a
(35:51):
lot, be aware that.
This is a lot of work that youmight now get for a low price,
but that's not sustainable andit will change at one point.
And if ever you build a library,call us.
Brooke (36:06):
Very key points indeed.
Thank you for sharing that.
And thank you so much for yourtime today.
You have such a vast backgroundand experience so I really
enjoyed getting to learn moreabout you and the amazing work
that you've done.
How can listeners get in touchwith you if they wanna connect?
Dr. Kihm (36:27):
Yes.
We are old school founders, sothere's still our website,
poma-ai.com where we also havethe sign up for our newsletters.
There we also have linked ourmedium articles where we
describe in more detail how POMAworks and what we do.
We will come up with morecontent where now we focus a lot
of on development.
We also are on LinkedIn where weencourage people to follow us
(36:49):
and catch up on our newestdevelopments.
Brooke (36:51):
Great.
Dr. Kihm (36:51):
And then otherwise I maybe see you again here.
Yes, the next product.
Brooke (36:55):
Absolutely, I would love to have a version two
conversation down the line andalways an open door and happy to
connect with you.
Thank you again so much for thisconversation.
It was a little bit higher levelof understanding, but I'm hoping
that my listeners at this pointhave been growing and learning
and they're ready for that nextlevel of understanding.
(37:18):
LLMs and how they're using yourinformation to give you the
answers.
And your final note aboutdiscernment when it comes to
accepting the information thatwe receive from AI is just so
important and rings true for meas well.
So thank you again Dr.
Kihm, so much.
Dr. Kihm (37:36):
Thank you.
Brooke (37:37):
Wow I hope today's episode opened your mind to
what's possible with AI.
Do you have a cool use case onhow you're using AI and wanna
share it?
DM me.
I'd love to hear more andfeature you on my next podcast.
Until next time, here's toworking smarter, not harder.
See you on the next episode ofHow I AI This episode was made
(37:58):
possible in partnership with theCollective AI, a community
designed to help entrepreneurs,creators, and professionals
seamlessly integrate AI intotheir workflows.
One of the biggest game changersin my own AI journey was joining
this space.
It's where I learned, connectedand truly enhanced my
understanding of what's possiblewith ai.
(38:20):
And the best part, they offermultiple membership levels to
meet you where you are.
Whether you want to DIY, your AIlearning or work with a
personalized AI consultant foryour business, The Collective
has you covered.
Learn more and sign up using myexclusive link in the show
notes.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.