September 25, 2024 • 32 mins
As the scale of artificial intelligence continues to evolve, open technology like many of IBM’s Granite models are helping enhance transparency in AI and improve efficiency across businesses. In this episode of Smart Talks with IBM, Jacob Goldstein sat down with Maryam Ashoori, the Director of Product Management and Head of Product for IBM’s watsonx.ai, where she spearheads the product strategy and delivery of IBM’s watsonx Foundation Models. Together, they explored the shift from large general-purpose AI models to smaller, customizable models tailored to specific needs.
This is a paid advertisement from IBM. The conversations on this podcast don't necessarily represent IBM's positions, strategies or opinions.
Visit us at https://ibm.com/smarttalks
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
Hey everyone, it's Robert and Joe here. Today we've got
something a little different to share with you. It's a
new season of the Smart Talks with IBM podcast series.
Speaker 2 (00:09):
This season, on smart Talks, Malcolm Gladwell and team are
diving into the transformative world of artificial intelligence with a
fresh perspective on the concept of open What does open
really mean in the context of AI. It can mean
open source code or open data, but it also encompasses
fostering an ecosystem of ideas, ensuring diverse perspectives are heard,
(00:31):
and enabling new levels of transparency.
Speaker 1 (00:33):
Join hosts from your favorite pushkin podcasts as they explore
how openness and AI is reshaping industries, driving innovation, and
redefining what's possible. You'll hear from industry experts and leaders
about the implication and possibilities of open AI, and of course,
Malcolm Gladwell will be there to guide you through the
season with his unique insights.
Speaker 2 (00:53):
Look out for new episodes of Smart Talks every other
week on the iHeartRadio app, Apple Podcasts, or wherever you
get your podcast and learn more at IBM dot com,
Slash smart Talks.
Speaker 3 (01:10):
Pushkin.
Speaker 4 (01:15):
Hello, Hello, Welcome to Smart Talks with IBM, a podcast
from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Glabo. This season,
we're diving back into the world of artificial intelligence, but
with a focus on the powerful concept of open its possibilities, implications,
and misconceptions. We'll look at openness from a variety of
(01:38):
angles and explore how the concept is already reshaping industries,
ways of doing business and our very notion of what's possible.
In today's episode, Jacob Goldstein sat down with maryam Ashuri,
the Director of Product Management and a Head of Product
for IBM's Watson x dot AI, where she spearheads the
(01:59):
product strategy and delivery of IBM's watsonex foundation models. She
is a technologist with more than fifteen years of experience
developing data driven technologies. The conversation focused on how enterprises
can use technology to build and deliver greater transparency in AI.
(02:19):
With Granite. Mariam explained how Grantite can be utilized to
improve efficiency across various domains. She discussed how these models
are being used in real world business applications, particularly in
areas like customer care, where AI can help enable quick,
accurate responses based on internal company data. Mariam provided a
(02:44):
fascinating look into how enterprises have moved from mere experimentation
with generative AI to actual production, navigating challenges such as
increased latency, cost, and energy consumption. She highlighted how the
emerging trend of smaller models customized with proprietary data can
(03:05):
potentially deliver high performance at a fraction of the cost,
marking a significant shift in how enterprises leverage AI. Whether
you're an AI enthusiast, we're a business leader looking to
harness the power of artificial intelligence, this episode is packed
with valuable insights and forward thinking strategies.
Speaker 3 (03:30):
Let's just start with your background. How did you come
to work at IBM.
Speaker 5 (03:34):
I join IBM right after I graduated. I have an
AI background, and throughout the years, I've held many roles
in design, engineering, development, research, mostly focused on AI application
development and design. In my current job, I'm the product
owner for What's the Next DAYI, which is the IBM
(03:58):
platform for enterprise AI. What excites me about this job,
I would say, is the technology advancements over the last
eighteen months in the market. We've been witnessing how GENERATIVELI
has been changing the market. The way that I see
that is JENNYI has been perhaps one of the largest
paradigm shifts when we think about productivity. The same way
(04:20):
that Internet and personal computers impacted the productivity of workforce,
now we are witnessing another wave of all those opportunities
that it can unlock for especially enterprise AI when it
comes to enhancing the productivity of the workforce and releasing
some time that can potentially be put into creating more
(04:42):
value work for enterprise. So that's the major part that
I picked this team to have an impact on the
market and the community, but also of course using the
skills that I gain through all these years through IBM
to help to establish IBM as the market leader for
(05:02):
enterprise AI.
Speaker 3 (05:04):
So you talked about JENAI as this sort of generational,
transformational technological force, and I'm curious just in terms of
how it's going to come into the world, Like, how
do you see market adoption of GENAI sort of evolving
from here?
Speaker 5 (05:21):
Well, last year was the year of excitement about generative AI.
Most of the companies were experimenting and exploring with GENI.
We see that energy shifted towards how to best monetize
that technology. Almost half of the market has moved from
investigation to pilots. Ten percent has moved to production. When
you're exploring with this technology, you're looking for a valve factor,
(05:45):
You're looking for an AHA moment. That's why very large
general purpose models shine. But as companies move toward production
and scale, they soon realized the past success is not
that straightforward. For example, they're larger the model, the larger
computer resources it requires. That translates to increased latency that's
(06:06):
your response time. That translates to increased cost. That translates
to increase carbon food print, and energy consumption. So think
about that. At the scale of enterprise in production, some
of them can be a showstopper.
Speaker 6 (06:20):
Because of this.
Speaker 5 (06:20):
Reason, what actually c is emerging in the market is
instead of focusing on very large general purpose models, coming
back to very small, trustworthy models that they can customize
on their own proprietary data that's the data about their customers,
(06:41):
that the data about their specific domains to create something
differentiated that is much smaller and delivers the performance that
they want on a target use case for a fraction
of the cost.
Speaker 3 (06:54):
Uh huh. So let's talk a little bit more specifically
about what you're working on. Talk about Granite. First of all,
tell me what is Granite.
Speaker 5 (07:04):
Granite is our industrial leading family of models, flagship IBM models.
These are the models that we train from scratch. When
offered to our platform, we offer indemnification and we stand
behind them today. It comes in four flavors, language, code,
time series, and geospecial models. Granite Language series is covering English, Spanish, German,
(07:32):
Portuguese and Japanese. We have a combination of commercial and
open source language models on Granite. For example, we recently
released the Granite seven B language model, small powerful English model.
On the code front, our models are state of the
art models ranging from three billion to thirty four billion parameters.
(07:55):
These are very powerful models that performs or outperforms in
some cases the popular open source models in their weight class.
So very powerful models.
Speaker 3 (08:06):
So I get the idea a big picture about these models,
but it would be helpful to just get a sense
specifically of what they're doing, Like, can you give me
any specific examples of how these models are being used
in businesses in the real world right now?
Speaker 5 (08:20):
Well, the top use cases for generative AI are really
content generation, summarization, information extraction. Perhaps the most popular use
case that we are seeing in enterprise is content grounded
question and answering. So using these models as a base
to connect them to a body of information let's say,
(08:42):
their policies, their documents that is internal to the enterprise,
and get the model to provide answers based on that question.
One example of that is for customer agents customer care,
when a customer is asking a question. Previously, the agent
that responds to the customer had to answer the question
(09:04):
and if they don't know the answer escalated to the product.
Especially is keeping people on hold on the line to
go figure out the answer for that and then come back.
You can think of the time it takes to resolve
an issue. But now we llms, we have an opportunity
to automatically retrieve the information based on the internal documents
of the company, formulate an answer, show it to the
(09:27):
human agent, and then if they verify with the sources
of varies coming from, they can just translate it directly
to the customer.
Speaker 6 (09:34):
This is a.
Speaker 5 (09:35):
Very simple example of how it's impacting the customer care.
Speaker 3 (09:39):
So one big theme of this season is this idea
of open and one of the things that's interesting to
me about the work you're doing is you are using
not only granted this model IBM developed, but you're also
using third party models right from other places. So tell
me about that work and how that is sort of
(10:01):
fitting into your kind of real world typically enterprise Jenai work.
Speaker 5 (10:05):
When it comes to a model strategy, our strategy is
really focused on two pillars, multimodel and multi deployment. It
means that we don't believe one single model rules all
the use cases. And I think at this point the
market has also realized the enterprise markets in average today
are using five to ten different models for different use cases.
Speaker 3 (10:27):
Oh interesting.
Speaker 5 (10:28):
So in our portfolio, if you look into what's on
Extra DAYI today, we are offering a large sets of
high performing, state of the art models coming from open
source commercial models that we are bringing through our partners
and also IBM developed models. In addition to all of these,
we also have an option for bring your own model
(10:48):
from outside the platform. Let's say you have a custom
model that you made it yourself, you can bring it
to the platform and really helping the customers to navigate
through aid range of models and pick the right model
for their target use case. Throughout that we've been heavily
working with our partners, and you know, this is the
(11:10):
market that is evolving rapidly. We've been at the forefront
of a spit to delivery. One example that I like
to highlight is recently Metal released Lama four or five billion,
such a powerful model. On the same day that it
was released to the market, we made it available in
our platform to our customers the same day. And not
(11:30):
only we delivered it on the same day. We are
offering competitive pricing but also for flexibility in where to deploy.
So we are giving an option to enterprise to deploy
these models on the platform of dage choice, either multi
cloud it can be gcpaws as youre IBM cloud, or
on premises. The same for mistrall Ai. Mistrall Ai recently
(11:54):
released the model misroll launch too on the same day
we delivered that through the platform. That's an example of
a commercial model. Lama as open source, but MS large
two is a commercial model that we made available through
the platform.
Speaker 3 (12:09):
Great, So I want to talk about enterprise grade foundation models.
Just to get into it briefly, what's a foundation model.
Speaker 5 (12:19):
People associate foundation models with a large language model, but
large language models are really a subset of foundation models.
Large language models are focused on language, but foundation models
can be code generators, can be focused on time series
model we talked about, they can be images, it can
be jew special models. So foundation model, as the term
(12:41):
suggests that your foundations to create a series of subsequent
models that can be customized for a downstream use case.
And that's why they are calling them foundation models. Lm
ME is a good example of that as a subset
for language that you can further customize on your space.
Speaker 6 (13:00):
Data to get the model to do other works.
Speaker 5 (13:04):
So the core of these foundation models, they are basically
trained on an ab third amount of data data sets
that most of the institutions today are sourcing them from
the internet. So you can imagine what can potentially go
to those models and then it comes to the enterprise
and they start using it. So for us also, when
(13:25):
we started looking into in particular, it was triggered by
customers asking us to provide client protections on these models,
and we started thinking about, let's look into how the
models are trained and if you are comfortable of fering
client protections on the models that are available in the market.
Speaker 6 (13:43):
And guess what, for a.
Speaker 5 (13:45):
Majority of these models there is absolutely no visibility into
what data vent into those models, not much transparency into
how the model trains, and the responsibility lies on you
as the customers we start using those models.
Speaker 3 (13:59):
So just to be that is presenting like potential risk,
real potential risk to a company that is using these models,
it is.
Speaker 5 (14:07):
It is a potential risk in particular for the customers
in highly regulated industries. So what we did for Granite
was when we started training these models from scratch, Basically
we went to the corpus of data that was available
to us. So, for example, the very first version of
Granite was exposed to twenty percent of its data from
(14:29):
finance and legal because we have a lot of financial
institutions as our clients. We worked directly with our IBM
research to identify detectors for harmful information like haytyp use
and profanity detectors.
Speaker 3 (14:45):
Okay, so we're talking about Granted, we're talking about this
set of models IBM has developed. Let's talk about using
Granite on Watson X compared to downloading open source models,
Like how do those differ?
Speaker 5 (14:57):
By using Granite and what's on ex you get two things.
The first one is the client protection and thementification that
we talked about. You get that if the model is
consumed through our platform.
Speaker 6 (15:09):
And the second.
Speaker 5 (15:10):
One is really the ecosystem of platform capabilities that we
are offering to help you create value on top of
those data. So for example, bringing your data to customize
granted for your own specific use case. But also one
thing that I like to highlight in particular is the
AI governance. So when you get one of these pre
(15:31):
train models, you put it in front of your own users.
Through the input and instructions that the user provides for
the model, they can notdge the model to potentially create
undesired behavior and change the behavior of the model. And
because of this is extremely important to automatically document the
(15:52):
lineage of who touched the model at one point, so
if something happens, you can trace it back and see
where it's coming from. And that's what's an extra governance
is offering automatically documenting the lineage. When you use the
granite within the platform, you get all of those you
can have the end to end governance, you can have
(16:13):
access to all these scalable deployment opportunities that is available
for you, like to allow you deploy them on the
platform of your choice that we talked about, either multiple
cloud or on prem and it also helps you to
have access to avoid range of model customizations, approaches, prompt tuning,
fine tuning, retrival augmented generations agents. There is a series
(16:36):
of them available to use an apply to your model.
Speaker 4 (16:39):
This distinction between large language models and foundation models is
eye opening. Mariam emphasized that foundation models can be tailored
to specific tasks, but with that versatility comes a significant
challenge the lack of transparency and how these models are trained.
This composed a real especially in highly regulated industries like finance. Essentially,
(17:05):
by using Granite and watsonex together, enterprises get powerful and
customizable tools.
Speaker 3 (17:12):
So let's talk about the future a little bit. What
do you think are some of the big developments were
likely to see in the realm of AI models?
Speaker 6 (17:20):
Very good question.
Speaker 5 (17:22):
I feel like the generative AI of the past was
powered by large language models. The generative AI of the
future is going to reason, plan, act and reflect.
Speaker 3 (17:35):
Huh, and so I mean in the context of Granite
in particular, like, what are we likely to see both
you know, in the near term and in the sort
of medium to long term.
Speaker 5 (17:46):
There are multiple elements to implement an agentic workflow that
I just mentioned. One element of that is the LLM
itself to be able to do the planning and reasoning
and acting and doing something that we call tool calling.
So basically, a series of tools are available to the model.
(18:08):
You ask the model to call those and.
Speaker 6 (18:10):
Make a call.
Speaker 5 (18:11):
For example, we can say, hey, Granted, what is the
weather like where Jacob lives. It's connect to web search API,
look up your location. Then it's going to connect to
weather API, calculate the weather and come back and formulate
an answer and respond to that. So during this process,
(18:32):
it first has to plan the task of how to
answer that question, look into what are the tools that
are available to it, and call them, and that's an
ability of the model to do that. What we did
with Granted was we expanded the Granite capabilities to be
able to do function calling. So for example, today we
have an open source granted to an eb function calling
(18:54):
that is available on hugging face to try on and
you can grab the model and the model has capability
to do the tool callings. I'm anticipating that in the
near future the planning and reasoning and acting and reflecting
capabilities of the large language models are going to continue
to evolve.
Speaker 3 (19:12):
So thinking now from the point of view of buyers
and users of AIS, really people who are listening from
that perspective, as people are evaluating AI tools and solutions,
what is the most important thing they should be thinking about?
How do you think about kind of that process?
Speaker 5 (19:33):
I think they should always start with the area at
which they think it would benefit from AI, and then
within that area, look into what data they have available
to potentially fit into those AI service architects do they
have access to quality data? And the second question that
they have to ask themselves is do I have a
(19:55):
trusted partner that can supply what I need to be
able to implement AI. That can be a collection of
the foundation models that you're going to need, that can
be a collection of the platform capabilities that the trusted
partner can offer you to implement such a thing. The
third thing is go and evaluate the regulations. Does regulation
(20:19):
allow you to apploy AI to the specific area that
you are investigating and you're targeting for AI? And the
last part, but not least, is back to the principles
of design, thinking, what is the problem in that area?
I'm solving with AI, and if AI is even appropriate,
(20:39):
because we want to make sure that you use AI
not just because it's a cool, hot toy in the market,
but you are convinced that it can significantly enhance the
user experience of your customers in that area. And once
you have an answer to those all these four questions,
then maybe you have a good candidates to start applying AI.
Speaker 3 (21:00):
What about from the side of project managers who are
trying to just keep up with how fast things are changing,
how fast innovation is happening, Like, what advice would you
give those people?
Speaker 5 (21:12):
My advice would be focused on agility. This is a
market that is evolving rapidly and the winners of the
market would be those that are able to take advantage
of the best the market can offer at any point
of time. So in order to do that, they need
to be open to experimentation, continuous learning, and to rapidly
(21:39):
adopting the new ideas.
Speaker 3 (21:42):
And when you think about the future and GENAI, is
there a particular, say problem that you are most excited
to solve.
Speaker 5 (21:50):
I think that would be productivity. If you look into
the stats that are out there, there are surveys that
confirm that sixty to seventy persons of the time of
our employees can be potentially enhanced to the productivity gains
of generative I For example, I personally myself use my
product for content generation a lot, so the time that
(22:14):
it frees up can be potentially put into generating a
higher value work. And because of that, I'm super excited
with all the opportunities that it represents for enterprises to
go and dedicate the time of the employees to higher
value items.
Speaker 3 (22:32):
Great. Okay, a couple of Granite specific questions. So what
are like the key things you want the world to
know about Granite.
Speaker 5 (22:42):
Granite is open, trusted, and targeted. Two ways to think
about openness. One open as open weights it's available for
public to download, and the second one is open as
in there is less restrictions on how the customers can
legally use these models for a range of use cases.
(23:05):
We have released Grantite open source models on their Apache
license that is enabling a large range of use cases.
The second one was trusted. We talked about that like
it's rooted in the trustworthy governance process that we established
thereund how we are training these models and the responsibility
that we take for these models, and the third one
(23:27):
is targeted, targeted for enterprise. We talked about like exposing
Granted to enterprise data or the domain specific Granted some
of them like Cobalt Java Translation that is targeting to
solve the specific enterprise needs. And that's granite, so open, trusted,
and targeted.
Speaker 3 (23:46):
So there are a lot of models out in the
world all of a sudden, right, it's a crowded market.
Where does granted fit in that universe? What is the
market for granted?
Speaker 5 (23:56):
We talked about the enterprise market shifting away from very
large general purpose models to target a smaller models, and
Granted is a small model that enterprise can pick up
and customize on their proprietary data to create something that
is differentiated for a target use case. So Granted is
(24:19):
well suited as a small, domain specific business, ready tailored
for business and trained on enterprise data to solve enterprise questions.
Speaker 3 (24:30):
You mentioned small as one of the things that granted
is why is that useful in some contexts for enterprise
for businesses.
Speaker 5 (24:40):
The larger the model, the larger computer resources it requires,
it translates to increased latency that's your response time. It
translates to increased cost and in translates to increased carbon
footprint and energy consumption. So at this case of enterprise transactions,
(25:01):
when you move to production and you want to scale,
some of these challenges can be multiple times stronger. Like
costs can add up, the energy consumption can be a
serious thing, and the latency is depending on the application,
can be a showstopper and blocker because for longer, larger models,
(25:24):
more powerful models, it just takes the way longer time
to process and calculate the output.
Speaker 3 (25:29):
For you, we are going to finish up with a
speed round and I want you to just answer with
the first thing that comes to mind. Don't overthink this, Okay,
complete this sentence. In five years, AI.
Speaker 6 (25:43):
Will be invisible.
Speaker 3 (25:46):
Ah, I like that. What do you mean by that?
Speaker 6 (25:49):
Today?
Speaker 5 (25:49):
AI is everywhere. But if you ask my kids at home,
they know AI. But if you say very like how
do you use AI, they don't know the answer because
it's so blended in their life that they don't feel
like it's something that they are using. They are getting
used to that. So when I think of next generation
(26:11):
and the years to come, that generation is so used
to AI being part of their life that they feel
like it's just there. That's one, and the second one
is the simplicity of interaction with AI that you don't
feel like you're interacting with the system. It's just there,
like you talk to AI. Everything is automated. So I
(26:32):
would say the simplicity and being blended to solve the
right problems is the part that I'm referring to as invisible.
Like Internet is everywhere and it's invisible. But we used
to dial in, like you remember the dialing zone to
connect the Internet.
Speaker 6 (26:49):
It's gone. The Internet is completely invisible today.
Speaker 3 (26:53):
Right, Like we used to talk about logging on, right,
and you don't log on anymore because you're always logged on.
Speaker 6 (26:59):
Yeah, always connected.
Speaker 3 (27:00):
Yeah. What's the number one thing that people misunderstand about AI?
Speaker 5 (27:06):
AI is anivitable but should not be feared.
Speaker 3 (27:11):
What advice would you give yourself ten years ago to
better prepare you for today?
Speaker 5 (27:17):
I would say, develop a broad range of skills. Even
if you think they will not help you today, they
may be valuable in the future.
Speaker 3 (27:27):
So on the consumer side, right now, we hear a
lot about chatbots and image generators. But on the business side,
what do you think is the next big business application?
Speaker 6 (27:38):
AI? Influencers generating content.
Speaker 3 (27:41):
Huh how do you use AI in your day to
day life today?
Speaker 5 (27:46):
One simple example is LinkedIn posts. I love it to
just go to my product. I'll give you an example,
which is my favorite one. Lama three point one four
or five b the post that I announced on LinkedIn
on Hey, IBM is releasing the model on the same
day it was generated by lamatory point one four or
five billion. So using the same model to post the
(28:09):
generate the announcement note very elegant.
Speaker 3 (28:13):
Is there anything else I should ask you?
Speaker 5 (28:15):
Oh, we didn't talk about instruct lab. So when you
grab a model, you start from the model, but you
need to then customize it on your proprietary data to
create value on top of that. So instruct lab is
giving you a method based on open source contributions to
(28:36):
collectively contribute to improve the base model. So if you're
an enterprise, you can leverage your internal employees to collectively
all contribute to improve the model. And I'll give you
an example of why it matters. Like if you go
to hugging Pace today and look for Lama, there are
(28:58):
about fifty thousand different lama us coming up. And the
reason is because there is no way to contribute to
the base model. If you're a developer, you have to
make a colon of the copy of the model and
finding need for your own purpose. We figure the method
that we call instruct lab to be able to collectively
collect all that information and contribute to the base model
(29:21):
and enhance.
Speaker 6 (29:21):
So that's instruct lab.
Speaker 5 (29:24):
I just wanted to highlight the value of being open
because that's another topic that has been emerging in the
market over the past eighteen months. In particular, I believe
the future of AI is open, and we've been seeing
how the open source markets has been changing, how the
models are accessible to a wider audience, and good things
(29:47):
typically happen when you make technology pieces accessible to a
broader range of community to stress test that, and that's
the direction that we've been adopting with granted, and I
felt like that's really the adoption that the market kit
is gonna emerge to moving forward.
Speaker 3 (30:02):
Yeah, there's this interesting I think, maybe naively unintuitive, but
it makes sense once you think about it, thing that
open source things are safer. You might naively think, oh no,
put it in a box so nobody can see it
and that'll be safer, But like it turns out of
the world. If you let everybody poke at it, the
world will find the vulnerabilities for you and you can
fix them. Right.
Speaker 6 (30:23):
That's exactly what's going to happen.
Speaker 3 (30:25):
Yeah, great, it was lovely to talk with you. Thank
you so much for your time.
Speaker 6 (30:30):
The same here, thanks Jacob, and.
Speaker 4 (30:34):
That wraps up this episode. A huge thanks to Mariam
and Jacob. Today's conversation open my eyes as to how
open technology and AI are intersecting to create more transparent
and efficient systems for enterprises. From the power of smaller,
more targeted models like granted to the importance of trust
and governance in AI, these developments are reshaping how businesses
(30:58):
operate at their core. As we continue to unpack the
complexities of artificial intelligence, it's clear that openness, whether in data,
technology or collaboration, is not just a concept, but a
driving force that can unlock new possibilities. Smart Talks with
(31:18):
IBM is produced by Matt Romano, Joey fish Ground, Amy
Gains McQuaid, and Jacob Goldstein. We're edited by Lydia Jane
kott Or. Engineers are Sarah Brugerier and Ben Tolliday. Theme
song by Gramoscope special thanks to the eight Bar and
IBM teams, as well as the Pushkin marketing team. Smart
Talks with IBM is a production of Pushkin Industries and
(31:40):
Ruby Studio at iHeartMedia. To find more Pushkin podcasts, listen
on the iHeartRadio app, Apple Podcasts, or wherever you listen
to podcasts. I'm Malcolm Glauwell. This is a paid advertisement
from IBM. The conversations on this podcast don't necessarily represent
(32:00):
IBM's positions, strategies, or opinions.
Hey everyone, it's Robert and Joe here. Today we've got
something a little different to share with you. It's a
new season of the Smart Talks with IBM podcast series.
Speaker 2 (00:09):
This season, on smart Talks, Malcolm Gladwell and team are
diving into the transformative world of artificial intelligence with a
fresh perspective on the concept of open What does open
really mean in the context of AI. It can mean
open source code or open data, but it also encompasses
fostering an ecosystem of ideas, ensuring diverse perspectives are heard,
(00:31):
and enabling new levels of transparency.
Speaker 1 (00:33):
Join hosts from your favorite pushkin podcasts as they explore
how openness and AI is reshaping industries, driving innovation, and
redefining what's possible. You'll hear from industry experts and leaders
about the implication and possibilities of open AI, and of course,
Malcolm Gladwell will be there to guide you through the
season with his unique insights.
Speaker 2 (00:53):
Look out for new episodes of Smart Talks every other
week on the iHeartRadio app, Apple Podcasts, or wherever you
get your podcast and learn more at IBM dot com,
Slash smart Talks.
Speaker 3 (01:10):
Pushkin.
Speaker 4 (01:15):
Hello, Hello, Welcome to Smart Talks with IBM, a podcast
from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Glabo. This season,
we're diving back into the world of artificial intelligence, but
with a focus on the powerful concept of open its possibilities, implications,
and misconceptions. We'll look at openness from a variety of
(01:38):
angles and explore how the concept is already reshaping industries,
ways of doing business and our very notion of what's possible.
In today's episode, Jacob Goldstein sat down with maryam Ashuri,
the Director of Product Management and a Head of Product
for IBM's Watson x dot AI, where she spearheads the
(01:59):
product strategy and delivery of IBM's watsonex foundation models. She
is a technologist with more than fifteen years of experience
developing data driven technologies. The conversation focused on how enterprises
can use technology to build and deliver greater transparency in AI.
(02:19):
With Granite. Mariam explained how Grantite can be utilized to
improve efficiency across various domains. She discussed how these models
are being used in real world business applications, particularly in
areas like customer care, where AI can help enable quick,
accurate responses based on internal company data. Mariam provided a
(02:44):
fascinating look into how enterprises have moved from mere experimentation
with generative AI to actual production, navigating challenges such as
increased latency, cost, and energy consumption. She highlighted how the
emerging trend of smaller models customized with proprietary data can
(03:05):
potentially deliver high performance at a fraction of the cost,
marking a significant shift in how enterprises leverage AI. Whether
you're an AI enthusiast, we're a business leader looking to
harness the power of artificial intelligence, this episode is packed
with valuable insights and forward thinking strategies.
Speaker 3 (03:30):
Let's just start with your background. How did you come
to work at IBM.
Speaker 5 (03:34):
I join IBM right after I graduated. I have an
AI background, and throughout the years, I've held many roles
in design, engineering, development, research, mostly focused on AI application
development and design. In my current job, I'm the product
owner for What's the Next DAYI, which is the IBM
(03:58):
platform for enterprise AI. What excites me about this job,
I would say, is the technology advancements over the last
eighteen months in the market. We've been witnessing how GENERATIVELI
has been changing the market. The way that I see
that is JENNYI has been perhaps one of the largest
paradigm shifts when we think about productivity. The same way
(04:20):
that Internet and personal computers impacted the productivity of workforce,
now we are witnessing another wave of all those opportunities
that it can unlock for especially enterprise AI when it
comes to enhancing the productivity of the workforce and releasing
some time that can potentially be put into creating more
(04:42):
value work for enterprise. So that's the major part that
I picked this team to have an impact on the
market and the community, but also of course using the
skills that I gain through all these years through IBM
to help to establish IBM as the market leader for
(05:02):
enterprise AI.
Speaker 3 (05:04):
So you talked about JENAI as this sort of generational,
transformational technological force, and I'm curious just in terms of
how it's going to come into the world, Like, how
do you see market adoption of GENAI sort of evolving
from here?
Speaker 5 (05:21):
Well, last year was the year of excitement about generative AI.
Most of the companies were experimenting and exploring with GENI.
We see that energy shifted towards how to best monetize
that technology. Almost half of the market has moved from
investigation to pilots. Ten percent has moved to production. When
you're exploring with this technology, you're looking for a valve factor,
(05:45):
You're looking for an AHA moment. That's why very large
general purpose models shine. But as companies move toward production
and scale, they soon realized the past success is not
that straightforward. For example, they're larger the model, the larger
computer resources it requires. That translates to increased latency that's
(06:06):
your response time. That translates to increased cost. That translates
to increase carbon food print, and energy consumption. So think
about that. At the scale of enterprise in production, some
of them can be a showstopper.
Speaker 6 (06:20):
Because of this.
Speaker 5 (06:20):
Reason, what actually c is emerging in the market is
instead of focusing on very large general purpose models, coming
back to very small, trustworthy models that they can customize
on their own proprietary data that's the data about their customers,
(06:41):
that the data about their specific domains to create something
differentiated that is much smaller and delivers the performance that
they want on a target use case for a fraction
of the cost.
Speaker 3 (06:54):
Uh huh. So let's talk a little bit more specifically
about what you're working on. Talk about Granite. First of all,
tell me what is Granite.
Speaker 5 (07:04):
Granite is our industrial leading family of models, flagship IBM models.
These are the models that we train from scratch. When
offered to our platform, we offer indemnification and we stand
behind them today. It comes in four flavors, language, code,
time series, and geospecial models. Granite Language series is covering English, Spanish, German,
(07:32):
Portuguese and Japanese. We have a combination of commercial and
open source language models on Granite. For example, we recently
released the Granite seven B language model, small powerful English model.
On the code front, our models are state of the
art models ranging from three billion to thirty four billion parameters.
(07:55):
These are very powerful models that performs or outperforms in
some cases the popular open source models in their weight class.
So very powerful models.
Speaker 3 (08:06):
So I get the idea a big picture about these models,
but it would be helpful to just get a sense
specifically of what they're doing, Like, can you give me
any specific examples of how these models are being used
in businesses in the real world right now?
Speaker 5 (08:20):
Well, the top use cases for generative AI are really
content generation, summarization, information extraction. Perhaps the most popular use
case that we are seeing in enterprise is content grounded
question and answering. So using these models as a base
to connect them to a body of information let's say,
(08:42):
their policies, their documents that is internal to the enterprise,
and get the model to provide answers based on that question.
One example of that is for customer agents customer care,
when a customer is asking a question. Previously, the agent
that responds to the customer had to answer the question
(09:04):
and if they don't know the answer escalated to the product.
Especially is keeping people on hold on the line to
go figure out the answer for that and then come back.
You can think of the time it takes to resolve
an issue. But now we llms, we have an opportunity
to automatically retrieve the information based on the internal documents
of the company, formulate an answer, show it to the
(09:27):
human agent, and then if they verify with the sources
of varies coming from, they can just translate it directly
to the customer.
Speaker 6 (09:34):
This is a.
Speaker 5 (09:35):
Very simple example of how it's impacting the customer care.
Speaker 3 (09:39):
So one big theme of this season is this idea
of open and one of the things that's interesting to
me about the work you're doing is you are using
not only granted this model IBM developed, but you're also
using third party models right from other places. So tell
me about that work and how that is sort of
(10:01):
fitting into your kind of real world typically enterprise Jenai work.
Speaker 5 (10:05):
When it comes to a model strategy, our strategy is
really focused on two pillars, multimodel and multi deployment. It
means that we don't believe one single model rules all
the use cases. And I think at this point the
market has also realized the enterprise markets in average today
are using five to ten different models for different use cases.
Speaker 3 (10:27):
Oh interesting.
Speaker 5 (10:28):
So in our portfolio, if you look into what's on
Extra DAYI today, we are offering a large sets of
high performing, state of the art models coming from open
source commercial models that we are bringing through our partners
and also IBM developed models. In addition to all of these,
we also have an option for bring your own model
(10:48):
from outside the platform. Let's say you have a custom
model that you made it yourself, you can bring it
to the platform and really helping the customers to navigate
through aid range of models and pick the right model
for their target use case. Throughout that we've been heavily
working with our partners, and you know, this is the
(11:10):
market that is evolving rapidly. We've been at the forefront
of a spit to delivery. One example that I like
to highlight is recently Metal released Lama four or five billion,
such a powerful model. On the same day that it
was released to the market, we made it available in
our platform to our customers the same day. And not
(11:30):
only we delivered it on the same day. We are
offering competitive pricing but also for flexibility in where to deploy.
So we are giving an option to enterprise to deploy
these models on the platform of dage choice, either multi
cloud it can be gcpaws as youre IBM cloud, or
on premises. The same for mistrall Ai. Mistrall Ai recently
(11:54):
released the model misroll launch too on the same day
we delivered that through the platform. That's an example of
a commercial model. Lama as open source, but MS large
two is a commercial model that we made available through
the platform.
Speaker 3 (12:09):
Great, So I want to talk about enterprise grade foundation models.
Just to get into it briefly, what's a foundation model.
Speaker 5 (12:19):
People associate foundation models with a large language model, but
large language models are really a subset of foundation models.
Large language models are focused on language, but foundation models
can be code generators, can be focused on time series
model we talked about, they can be images, it can
be jew special models. So foundation model, as the term
(12:41):
suggests that your foundations to create a series of subsequent
models that can be customized for a downstream use case.
And that's why they are calling them foundation models. Lm
ME is a good example of that as a subset
for language that you can further customize on your space.
Speaker 6 (13:00):
Data to get the model to do other works.
Speaker 5 (13:04):
So the core of these foundation models, they are basically
trained on an ab third amount of data data sets
that most of the institutions today are sourcing them from
the internet. So you can imagine what can potentially go
to those models and then it comes to the enterprise
and they start using it. So for us also, when
(13:25):
we started looking into in particular, it was triggered by
customers asking us to provide client protections on these models,
and we started thinking about, let's look into how the
models are trained and if you are comfortable of fering
client protections on the models that are available in the market.
Speaker 6 (13:43):
And guess what, for a.
Speaker 5 (13:45):
Majority of these models there is absolutely no visibility into
what data vent into those models, not much transparency into
how the model trains, and the responsibility lies on you
as the customers we start using those models.
Speaker 3 (13:59):
So just to be that is presenting like potential risk,
real potential risk to a company that is using these models,
it is.
Speaker 5 (14:07):
It is a potential risk in particular for the customers
in highly regulated industries. So what we did for Granite
was when we started training these models from scratch, Basically
we went to the corpus of data that was available
to us. So, for example, the very first version of
Granite was exposed to twenty percent of its data from
(14:29):
finance and legal because we have a lot of financial
institutions as our clients. We worked directly with our IBM
research to identify detectors for harmful information like haytyp use
and profanity detectors.
Speaker 3 (14:45):
Okay, so we're talking about Granted, we're talking about this
set of models IBM has developed. Let's talk about using
Granite on Watson X compared to downloading open source models,
Like how do those differ?
Speaker 5 (14:57):
By using Granite and what's on ex you get two things.
The first one is the client protection and thementification that
we talked about. You get that if the model is
consumed through our platform.
Speaker 6 (15:09):
And the second.
Speaker 5 (15:10):
One is really the ecosystem of platform capabilities that we
are offering to help you create value on top of
those data. So for example, bringing your data to customize
granted for your own specific use case. But also one
thing that I like to highlight in particular is the
AI governance. So when you get one of these pre
(15:31):
train models, you put it in front of your own users.
Through the input and instructions that the user provides for
the model, they can notdge the model to potentially create
undesired behavior and change the behavior of the model. And
because of this is extremely important to automatically document the
(15:52):
lineage of who touched the model at one point, so
if something happens, you can trace it back and see
where it's coming from. And that's what's an extra governance
is offering automatically documenting the lineage. When you use the
granite within the platform, you get all of those you
can have the end to end governance, you can have
(16:13):
access to all these scalable deployment opportunities that is available
for you, like to allow you deploy them on the
platform of your choice that we talked about, either multiple
cloud or on prem and it also helps you to
have access to avoid range of model customizations, approaches, prompt tuning,
fine tuning, retrival augmented generations agents. There is a series
(16:36):
of them available to use an apply to your model.
Speaker 4 (16:39):
This distinction between large language models and foundation models is
eye opening. Mariam emphasized that foundation models can be tailored
to specific tasks, but with that versatility comes a significant
challenge the lack of transparency and how these models are trained.
This composed a real especially in highly regulated industries like finance. Essentially,
(17:05):
by using Granite and watsonex together, enterprises get powerful and
customizable tools.
Speaker 3 (17:12):
So let's talk about the future a little bit. What
do you think are some of the big developments were
likely to see in the realm of AI models?
Speaker 6 (17:20):
Very good question.
Speaker 5 (17:22):
I feel like the generative AI of the past was
powered by large language models. The generative AI of the
future is going to reason, plan, act and reflect.
Speaker 3 (17:35):
Huh, and so I mean in the context of Granite
in particular, like, what are we likely to see both
you know, in the near term and in the sort
of medium to long term.
Speaker 5 (17:46):
There are multiple elements to implement an agentic workflow that
I just mentioned. One element of that is the LLM
itself to be able to do the planning and reasoning
and acting and doing something that we call tool calling.
So basically, a series of tools are available to the model.
(18:08):
You ask the model to call those and.
Speaker 6 (18:10):
Make a call.
Speaker 5 (18:11):
For example, we can say, hey, Granted, what is the
weather like where Jacob lives. It's connect to web search API,
look up your location. Then it's going to connect to
weather API, calculate the weather and come back and formulate
an answer and respond to that. So during this process,
(18:32):
it first has to plan the task of how to
answer that question, look into what are the tools that
are available to it, and call them, and that's an
ability of the model to do that. What we did
with Granted was we expanded the Granite capabilities to be
able to do function calling. So for example, today we
have an open source granted to an eb function calling
(18:54):
that is available on hugging face to try on and
you can grab the model and the model has capability
to do the tool callings. I'm anticipating that in the
near future the planning and reasoning and acting and reflecting
capabilities of the large language models are going to continue
to evolve.
Speaker 3 (19:12):
So thinking now from the point of view of buyers
and users of AIS, really people who are listening from
that perspective, as people are evaluating AI tools and solutions,
what is the most important thing they should be thinking about?
How do you think about kind of that process?
Speaker 5 (19:33):
I think they should always start with the area at
which they think it would benefit from AI, and then
within that area, look into what data they have available
to potentially fit into those AI service architects do they
have access to quality data? And the second question that
they have to ask themselves is do I have a
(19:55):
trusted partner that can supply what I need to be
able to implement AI. That can be a collection of
the foundation models that you're going to need, that can
be a collection of the platform capabilities that the trusted
partner can offer you to implement such a thing. The
third thing is go and evaluate the regulations. Does regulation
(20:19):
allow you to apploy AI to the specific area that
you are investigating and you're targeting for AI? And the
last part, but not least, is back to the principles
of design, thinking, what is the problem in that area?
I'm solving with AI, and if AI is even appropriate,
(20:39):
because we want to make sure that you use AI
not just because it's a cool, hot toy in the market,
but you are convinced that it can significantly enhance the
user experience of your customers in that area. And once
you have an answer to those all these four questions,
then maybe you have a good candidates to start applying AI.
Speaker 3 (21:00):
What about from the side of project managers who are
trying to just keep up with how fast things are changing,
how fast innovation is happening, Like, what advice would you
give those people?
Speaker 5 (21:12):
My advice would be focused on agility. This is a
market that is evolving rapidly and the winners of the
market would be those that are able to take advantage
of the best the market can offer at any point
of time. So in order to do that, they need
to be open to experimentation, continuous learning, and to rapidly
(21:39):
adopting the new ideas.
Speaker 3 (21:42):
And when you think about the future and GENAI, is
there a particular, say problem that you are most excited
to solve.
Speaker 5 (21:50):
I think that would be productivity. If you look into
the stats that are out there, there are surveys that
confirm that sixty to seventy persons of the time of
our employees can be potentially enhanced to the productivity gains
of generative I For example, I personally myself use my
product for content generation a lot, so the time that
(22:14):
it frees up can be potentially put into generating a
higher value work. And because of that, I'm super excited
with all the opportunities that it represents for enterprises to
go and dedicate the time of the employees to higher
value items.
Speaker 3 (22:32):
Great. Okay, a couple of Granite specific questions. So what
are like the key things you want the world to
know about Granite.
Speaker 5 (22:42):
Granite is open, trusted, and targeted. Two ways to think
about openness. One open as open weights it's available for
public to download, and the second one is open as
in there is less restrictions on how the customers can
legally use these models for a range of use cases.
(23:05):
We have released Grantite open source models on their Apache
license that is enabling a large range of use cases.
The second one was trusted. We talked about that like
it's rooted in the trustworthy governance process that we established
thereund how we are training these models and the responsibility
that we take for these models, and the third one
(23:27):
is targeted, targeted for enterprise. We talked about like exposing
Granted to enterprise data or the domain specific Granted some
of them like Cobalt Java Translation that is targeting to
solve the specific enterprise needs. And that's granite, so open, trusted,
and targeted.
Speaker 3 (23:46):
So there are a lot of models out in the
world all of a sudden, right, it's a crowded market.
Where does granted fit in that universe? What is the
market for granted?
Speaker 5 (23:56):
We talked about the enterprise market shifting away from very
large general purpose models to target a smaller models, and
Granted is a small model that enterprise can pick up
and customize on their proprietary data to create something that
is differentiated for a target use case. So Granted is
(24:19):
well suited as a small, domain specific business, ready tailored
for business and trained on enterprise data to solve enterprise questions.
Speaker 3 (24:30):
You mentioned small as one of the things that granted
is why is that useful in some contexts for enterprise
for businesses.
Speaker 5 (24:40):
The larger the model, the larger computer resources it requires,
it translates to increased latency that's your response time. It
translates to increased cost and in translates to increased carbon
footprint and energy consumption. So at this case of enterprise transactions,
(25:01):
when you move to production and you want to scale,
some of these challenges can be multiple times stronger. Like
costs can add up, the energy consumption can be a
serious thing, and the latency is depending on the application,
can be a showstopper and blocker because for longer, larger models,
(25:24):
more powerful models, it just takes the way longer time
to process and calculate the output.
Speaker 3 (25:29):
For you, we are going to finish up with a
speed round and I want you to just answer with
the first thing that comes to mind. Don't overthink this, Okay,
complete this sentence. In five years, AI.
Speaker 6 (25:43):
Will be invisible.
Speaker 3 (25:46):
Ah, I like that. What do you mean by that?
Speaker 6 (25:49):
Today?
Speaker 5 (25:49):
AI is everywhere. But if you ask my kids at home,
they know AI. But if you say very like how
do you use AI, they don't know the answer because
it's so blended in their life that they don't feel
like it's something that they are using. They are getting
used to that. So when I think of next generation
(26:11):
and the years to come, that generation is so used
to AI being part of their life that they feel
like it's just there. That's one, and the second one
is the simplicity of interaction with AI that you don't
feel like you're interacting with the system. It's just there,
like you talk to AI. Everything is automated. So I
(26:32):
would say the simplicity and being blended to solve the
right problems is the part that I'm referring to as invisible.
Like Internet is everywhere and it's invisible. But we used
to dial in, like you remember the dialing zone to
connect the Internet.
Speaker 6 (26:49):
It's gone. The Internet is completely invisible today.
Speaker 3 (26:53):
Right, Like we used to talk about logging on, right,
and you don't log on anymore because you're always logged on.
Speaker 6 (26:59):
Yeah, always connected.
Speaker 3 (27:00):
Yeah. What's the number one thing that people misunderstand about AI?
Speaker 5 (27:06):
AI is anivitable but should not be feared.
Speaker 3 (27:11):
What advice would you give yourself ten years ago to
better prepare you for today?
Speaker 5 (27:17):
I would say, develop a broad range of skills. Even
if you think they will not help you today, they
may be valuable in the future.
Speaker 3 (27:27):
So on the consumer side, right now, we hear a
lot about chatbots and image generators. But on the business side,
what do you think is the next big business application?
Speaker 6 (27:38):
AI? Influencers generating content.
Speaker 3 (27:41):
Huh how do you use AI in your day to
day life today?
Speaker 5 (27:46):
One simple example is LinkedIn posts. I love it to
just go to my product. I'll give you an example,
which is my favorite one. Lama three point one four
or five b the post that I announced on LinkedIn
on Hey, IBM is releasing the model on the same
day it was generated by lamatory point one four or
five billion. So using the same model to post the
(28:09):
generate the announcement note very elegant.
Speaker 3 (28:13):
Is there anything else I should ask you?
Speaker 5 (28:15):
Oh, we didn't talk about instruct lab. So when you
grab a model, you start from the model, but you
need to then customize it on your proprietary data to
create value on top of that. So instruct lab is
giving you a method based on open source contributions to
(28:36):
collectively contribute to improve the base model. So if you're
an enterprise, you can leverage your internal employees to collectively
all contribute to improve the model. And I'll give you
an example of why it matters. Like if you go
to hugging Pace today and look for Lama, there are
(28:58):
about fifty thousand different lama us coming up. And the
reason is because there is no way to contribute to
the base model. If you're a developer, you have to
make a colon of the copy of the model and
finding need for your own purpose. We figure the method
that we call instruct lab to be able to collectively
collect all that information and contribute to the base model
(29:21):
and enhance.
Speaker 6 (29:21):
So that's instruct lab.
Speaker 5 (29:24):
I just wanted to highlight the value of being open
because that's another topic that has been emerging in the
market over the past eighteen months. In particular, I believe
the future of AI is open, and we've been seeing
how the open source markets has been changing, how the
models are accessible to a wider audience, and good things
(29:47):
typically happen when you make technology pieces accessible to a
broader range of community to stress test that, and that's
the direction that we've been adopting with granted, and I
felt like that's really the adoption that the market kit
is gonna emerge to moving forward.
Speaker 3 (30:02):
Yeah, there's this interesting I think, maybe naively unintuitive, but
it makes sense once you think about it, thing that
open source things are safer. You might naively think, oh no,
put it in a box so nobody can see it
and that'll be safer, But like it turns out of
the world. If you let everybody poke at it, the
world will find the vulnerabilities for you and you can
fix them. Right.
Speaker 6 (30:23):
That's exactly what's going to happen.
Speaker 3 (30:25):
Yeah, great, it was lovely to talk with you. Thank
you so much for your time.
Speaker 6 (30:30):
The same here, thanks Jacob, and.
Speaker 4 (30:34):
That wraps up this episode. A huge thanks to Mariam
and Jacob. Today's conversation open my eyes as to how
open technology and AI are intersecting to create more transparent
and efficient systems for enterprises. From the power of smaller,
more targeted models like granted to the importance of trust
and governance in AI, these developments are reshaping how businesses
(30:58):
operate at their core. As we continue to unpack the
complexities of artificial intelligence, it's clear that openness, whether in data,
technology or collaboration, is not just a concept, but a
driving force that can unlock new possibilities. Smart Talks with
(31:18):
IBM is produced by Matt Romano, Joey fish Ground, Amy
Gains McQuaid, and Jacob Goldstein. We're edited by Lydia Jane
kott Or. Engineers are Sarah Brugerier and Ben Tolliday. Theme
song by Gramoscope special thanks to the eight Bar and
IBM teams, as well as the Pushkin marketing team. Smart
Talks with IBM is a production of Pushkin Industries and
(31:40):
Ruby Studio at iHeartMedia. To find more Pushkin podcasts, listen
on the iHeartRadio app, Apple Podcasts, or wherever you listen
to podcasts. I'm Malcolm Glauwell. This is a paid advertisement
from IBM. The conversations on this podcast don't necessarily represent
(32:00):
IBM's positions, strategies, or opinions.
Stuff To Blow Your Mind News
Hosts And Creators
Robert Lamb
Joe McCormick
Popular Podcasts
1. Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
2. Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations.
3. Crime Junkie
If you can never get enough true crime... Congratulations, you’ve found your people.