February 14, 2025 • 30 mins
Last month, no one had heard of DeepSeek. But then it racked up a stunning 16 million downloads in just 18 days — nearly double ChatGPT’s initial adoption.
The China-based AI model has certainly captured the attention of both investors and technologists. But while some see it as the future of AI, others are raising red flags over its origins and potential security risks.
In this episode of Innovation Heroes, we explore what makes DeepSeek unique, how it compares to existing AI models, and the startling data privacy concerns for businesses and governments worldwide.
Should you dive in? Or swim in shallower waters?
Featuring:
- Jack Hogan, Vice President of Advanced Growth Technologies at SHI, leading initiatives in AI, cybersecurity, and emerging technologies.
- Lee Ziliak, Field Chief Technology Officer and Managing Director of Architecture at SHI.
- Jordan Mauriello, Chief Strategy Officer and Head of Stratascale.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jordan (00:00):
We wanted to run the chat bot itself in a contained environment, right?
And so we loaded up virtualizationenvironments, ran it on the Android
system, and then watched all thecommunications that were coming out of it.
We wanted to validate, look at thisourselves, watch what was happening.
There are communicationsgoing directly back to China.
They are in fact goingto China mobile systems.
Right, which is an organization thatis banned from doing business in the
(00:23):
U. S., that the U. S. considers that asmuch of a tie to the Chinese Communist
Party and the military organizationsthere that we're not allowed to invest
in as individuals or businesses, um, andthey're banned from doing business here.
Ed (00:36):
In the world of technology, heroes are everywhere.
They're overcoming disruption, deliveringsustainable outcomes, and fearlessly
forging the future to solve what's next.
Join me, Ed McNamara, as we meet thepeople and businesses driving change
in our constantly disruptive world.
This is Innovation Heroes, apodcast brought to you by SHI.
(01:01):
Matthew McConaughey famously said.
that the number one rule of Wall Streetwas that nobody knows if a stock is
going up, down, sideways, or in circles.
Predicting where the next big thing in AImight come from kind of feels the same.
The most recent market disruptingsurprise in AI came from China,
and its name is DeepSeek.
In January, DeepSeek launched with abang, racking up 16 million downloads
(01:24):
in just 18 days, nearly double ChatGPT'sinitial release numbers, with claims it
was developed at literally a fractionof the cost of other AI models.
That combination wiped out hundredsof billions of market cap on a day on
Wall Street that would have surprisedboth Jimmy and Warren Buffett.
Some are calling DeepSeek the cuttingedge of AI innovation, while others are
raising eyebrows over some big concerns.
(01:47):
Today we're breaking down what makesDeepSeek unique, how it stacks up against
other platforms, and how companiesmight consider governing the risks
and promise of technologies like it.
To help us do that, we'rejoined by three panelists.
Jack Hogan, Lee Ziliak,and Jordan Mariello.
Jack Hogan is Vice Presidentof Advanced Growth Technologies
at SHI International, leadinginitiatives in AI and cybersecurity.
(02:11):
Lee Ziliak is SHI's FieldTechnology Officer and Managing
Director of Architecture.
And last but certainly notleast is Jordan Mariello.
Jordan is Chief StrategyOfficer and Head of Stratascale.
Bringing over 25 years of cybersecurity expertise to the role.
Jack, Lee, Jordan, welcometo Innovation Heroes.
Jack (02:30):
Hey Ed, thanks for having us on today.
Thanks Ed.
Ed (02:33):
Absolutely.
So, gentlemen, today we're talkingabout DeepSeek, but before we dive
in, I want to hear a little bit abouteach of your professional backgrounds.
You all have a mix of experienceand expertise in machine
learning, generative AI.
It and cloud computing and probablyevery other emerging technology
for the past, you know, 20 plusyears, Jack, we'll start with you.
Tell me a little bit about your backgroundand how you got to where you are today.
Jack (02:55):
Yeah, thanks.
And my actually historygoes back about 30 years.
And when I first came out ofcollege, I actually wrote a neural
network program to predict themovement of the Swiss Frank future.
So this is leveraging some ofthe early scale machine learning.
capabilities that we're nowreally seeing at scale with
things like DeepSeek and GPT.
(03:16):
But I spent the bulk of my careeron the, as a customer side co
founder and CTO in a consumer cloudand SAS company went through a
number of the main waves of Web 1.
0, Web 2.
0, early stages of virtualization,storage technologies, and ultimately
dealing with large data and data sets.
And so I spent really a seven year.
(03:37):
Time with, uh, one of my corebusiness partners in that, which
was Pure Storage, where I led, Iwas their VP of technology strategy.
And in that role, I understood theimportance of the physics of, uh,
dealing with data processing at scale.
Um, and I joined SHI, uh, a littleless than a year ago to lead the
advanced growth technologies team.
So from my point of view, I've livedon the customer side, I've lived
(03:58):
on the vendor side, and really now.
At SHI, helping customers navigatethrough some of the bigger challenges
as it relates to next generationtechnologies and the infrastructure
and components that go into that.
Ed (04:10):
Excellent.
And Lee, what about you?
Lee (04:13):
Well, Ed, you were about half, halfway there when you said 20 years.
I've been in the industryfor about 40 years.
Um, starting right out of high school,just like Jack and working my way
through college, um, started, uh,working for the government doing, uh,
various things in such old languagesas like assembler language and COBOL,
but then spent the bulk of my careerat, uh, Verizon, um, where I ran server
(04:38):
storage and mainframe architecture.
Um, led the charge into the, the publiccloud and private cloud at Verizon.
And that's really, uh, what led meto SHI was, uh, you know, SHI was
standing up a cloud business, so Icame over, uh, and started the cloud
professional services business, uh, at,at SHI, um, for enterprise customers.
(05:00):
Um, and then after that, I, um,moved over into our global segment
and then joined Jack, uh, in,uh, a GT in July when he, uh.
Came on board and started thatgroup because, you know, it
was, uh, focused on, uh, AI andlarge data center opportunities.
And that's kind of my sweet spot that I'vebeen in for, uh, most of those 40 years.
Ed (05:21):
Awesome.
And Jordan, how about you?
Jordan (05:23):
Had a similar story here.
Kind of started my career, uh,straight out of high school for me.
I went into the military and working,uh, communication, security, crypto
cyber there, kind of doing the sameexact job for the government afterwards.
Many people know when they moveover to the contractor side, you
do the same work and just get paida little bit better for it and
deploy a little bit less, hopefully.
Um, and then I moved over to thecommercial side after that and I
(05:46):
spent a few years working in financialservices, ended up leading a global
security operations center for avery large financial organization.
Um, and then, uh, decided to stepout and join some partners and jump
into the entrepreneur real world.
And, uh, so did, uh, a few startupbusinesses, kind of spun out two
from one main one and ended upbuilding three separate companies.
(06:06):
But did some platform work,um, and building a security
orchestration platform.
Um, then, uh, built a managed detectionand response business, Critical Start,
fairly well known one in that space.
And through that time, there's kind ofthis intermingling of building software
on machine learning and technology todo security analytics in particular.
(06:27):
So we approached a lot of that from anAI and machine learning perspective.
Um, and whether that's, youknow, developing neural networks
or using a lot of clusteringalgorithms, implementing those.
From an AI perspective intosecurity orchestration platforms
and improving detection.
Ed (06:43):
So, all right, DeepSeek.
Lee, I'll look in your direction to startwith, you know, when you first heard
that news, what were your first thoughts?
Lee (06:52):
Um, well, I thought it was actually pretty exciting, right?
I, you know, if all the claimswere true, you know, what they
did was kind of revolutionary.
Uh, being the, the geek that Iam, I, I soon started pulling back
the covers to kind of understand,all right, what, what in this is
real, uh, and what is, what is not.
And so, you know, I'm not sayingthat the claims are false.
(07:17):
A lot of people have, but if youstart reading the details, like
the claim of 6 million to train.
The model, right?
That that's true, but itwas only the final run.
There's like months and yearsof work that went into that that
everybody else accounts for that thatwasn't in that final cost, right?
They built their model based onwork that other companies have done.
(07:37):
So yeah, of course, it'sgoing to be cheaper.
It's revolutionary, right?
It's a new way to build a model.
So from that perspective,it's super exciting.
I think it's going to revolutionizethe way that, uh, you know, other
companies are looking at buildingmodels, but I think everything
kind of needs to be put in context.
So, um, like I said, excitingstuff, but really just need to
(07:59):
needed to pull back the layers tounderstand it a little bit better.
Ed (08:02):
Jordan, what were your first thoughts when you heard the news?
Jordan (08:06):
I probably a little bit of disbelief I had coming
from the security world.
I probably my first bent is to beskeptical, um, of releases and innovation
announcements of the sort like this.
And we see them a lot in thesecurity world and we've solved
the security problem of X, right?
And then it's our job tobe super skeptical of that.
So I think I dove in prettyquickly to say, okay, well, what
(08:26):
do we think really happened here?
Um, and you know, isthat dollar amount true?
And like Lee was saying, youknow, it kind of seems to be.
Only enumerating an end of, uh,maturity or end of release phase,
specifically a training the model anddoing distillation, but there's things.
Going into that, that led up to that,that probably accrued significant
costs before that, that I don't thinkare accounted for in that number.
(08:49):
However, I also did the same point thatthis is a very well trained model and it
does have, um, some unique innovations,uh, around the reasoning processes is
there that, that are worth knowing.
I think drew attention for a reason.
Um, but then my next immediate thoughtafter that is, okay, what are the
security implications of where thisis housed and who's running this?
(09:10):
And what do we need to beworried about in that world too?
Ed (09:12):
And Jack turning to you, you know, can you, you summarize
what the AI model impact reallyon, on, I'll say on the market?
Jack (09:19):
Well, yeah, I think from a market point of view, it introduced a, well,
I'll say, I wouldn't say, I guess a totalquantum leap, but it, it, it represented
a very big leap forward in the waythat models can actually be designed.
Um, and you know, conceptsthat we'll get into things like
distillation and chain of thought.
You know, those are, those areimportant foundational building blocks.
(09:40):
Um, I think though that when we lookat the market impact of it, because
the model was delivered as an opensource model, it actually created
a lot of positive impact for themarket, despite the actual financial
markets seeing the negativity of it.
The positives is that we all are now ableto look at how the model was put together
(10:01):
and And really deconstruct some of thosecomponents to extract really that, that,
you know, moving models forward or moving,you know, the AI development, those were
some of the moments as we're, you know,continuing to go, you know, grow towards
AGI or some of the broader conceptsof machine learning and artificial
intelligence outdoing human intelligence.
(10:22):
So from my point of view, I thinkfrom a market perspective, it really
created, um, that, that flashpoint of,okay, now we can look at developing
different types of models differently.
And I think it was a large enough, uh,global impact that pretty much everyone's
taking it, paying attention to it.
And I think rapidly turning towardshow do we take advantage of it?
(10:42):
This is actually a great thing.
for the rapid development that'smoving, coming ahead of us.
And, uh, you know, so I thinkthe excitement in terms of
what it means in the long termis, I think, really exciting.
Ed (10:54):
So, Jack, maybe we'll stay with you for a second.
Um, let's talk about some of DeepSeek's AItechniques that are potential advantages
over platforms like ChatGPT or Gemini.
Um, you know, one of thetechniques this model uses is
called Mixture of Experts, or MOE.
Um, can you explain, like, whatthat is and how is it different
from, say, ChatGPT's model?
Jack (11:14):
Well, when you look at the way neural networks are put together and
the different concepts that have tobe, um, trained and, and operating
together, being able to tie togetherthese experts and mix them in one model,
leveraging the distillation of someother models, and then, you know, turning
to a more deeper reasoning response,I think that, you know, one of the
(11:36):
things that's really important aboutwhen you look at bringing in a bunch of
experts into any practical sense, You,you, you learn a lot by having those
experts kind of work with each other.
Similarly in AI model development,that's kind of the practical element
of how a mixture of experts can cometogether and open up new ways to tie
together previously trained componentsor net neural nets within that, you
(11:58):
know, that fact that they relied on alot of the mixture of expert components.
I think where we're, we're lookingat that next phase is how does that
move into the mixture of models, um,you know, different models that are
different type of experts that aretrained to do very specific things.
And I think there's some, there's somereally important elements as we look
at how we move into this concept ofagentic AI, um, and where you want a
(12:21):
less reasoned model to do a very definedresponse, you know, question response.
Um, being able to have mixtures ofexperts and mixtures of models start
to become a really important way that,uh, AI can be applied beyond deep
reasoning or deep learning models.
Ed (12:36):
And Lee, another difference was something called chain of thought or COT.
Uh, can you kind of explain what that isand what maybe some of the advantages are?
Lee (12:45):
So chain of thought is exactly what it sounds like, right?
So the model, um, takes yourquery or the, the problem it needs
to solve and it breaks it down.
Into different steps of what it callsthoughts, and then it starts to reason
through each one of those thoughtsuntil it gets through to the answer.
Um, so it's actually, you know, it'sinteresting to watch in a chat bot because
(13:09):
I joked with Jack when I first saw it.
I'm like, I think this model wastrained by my college math teacher
because it has to show me everythingalong the way to get to the answer.
So that's, I mean, that'sgood and bad, right?
So, but the, the good piece of thatis it's, it increases transparency.
It, uh, increases accuracy, right?
So as it works through these differentproblems, the accuracy level of the model
(13:32):
is actually increased, so it's improved.
Um, and so you can see everything thatthe model goes through to get to its
answer, and it'll, uh, work through fromone thought to the other to increase
that accuracy, so that's, that'skind of what the chain of thought is.
Starting with a problem and here'sall the different steps that I
worked through to get to the answer.
(13:54):
Um, in the, you know, like I said,there's benefits to it in terms
of transparency and accuracy.
There's downsides to it.
Right.
So the, as I said, the outputtends to be very verbose.
And so if you're look, you're lookingfor a very, you know, quick or, or, or
short answer, um, you know, it's notquite as easy to swap out GPT and put in.
(14:15):
Uh, deep seat because the outputis basically completely different.
The other impact of that is going throughthe chain of thought process is also
tends to be more resource intensive.
Um, and so, you know, the hardware thatis required, especially on the much larger
models, um, is going to be impacted.
Now, that's sometimes that'soffset by the, like I said,
(14:36):
the implementation of the MOE.
Um, in the distillation, um, but italso is important because of that to
verify the claims of efficiency thatdeep sea claimed when they put out
the model, they claimed that it wasn'tonly efficient to train, but it was
also that it runs more efficient.
And so that's one of the thingsthat we're doing is we put DeepSeek
(14:57):
into our, um, next gen AI labis we're verifying that impact.
Ed (15:03):
So we've got, um, mixture of experts, MOE, we've got chain of
thought, COT, Jordan, the, the otherthat was mentioned was distillation.
Can you, can you expand on that andmaybe some of the benefits on that?
Jordan (15:15):
Yeah, you know, distillation is not necessarily dissimilar for some
things we're already familiar with, liketraining small language models on LLMs.
It's like.
We do this a lot for security, right?
There's a lot of securityspecific data that the LLM doesn't
necessarily have context about.
So you have a large languagemodel like OpenAI's ChatGBT, you
know, massive data set extensivelytrained to get this high performance
(15:38):
for generating human like text.
And that's what we wanted to do.
But then we can take and work off oftop of that model and use that in what
we call a teacher student relationship.
And then we use the parent or theteacher model to generate Soft
targets, meaning we're asking specificquestions to get specific outputs when
we don't just want that one output.
We want to get some of the otherpossible outputs that were also there.
(16:01):
And this allows us to build a littlebit more targeted, but more efficient
data set that doesn't need to be.
as extensive of a database that'sbeen trained on from the beginning,
and therefore there's an efficiencygain there in the training
process that's fairly significant.
And I think there's substantivedata that supports the idea that
(16:22):
DeepSeek's DeepThinkingR1 model isdistilled from OpenAI's ChatGPT.
And I think it also shows, like Jackwas saying, like, this is a process
that really does work to generate ahigh functioning model that is very
useful for business applications.
Um, and I think it helps to bend thecurve a little bit on the efficiency,
(16:43):
especially from a cost perspective,on what people will need to do moving
forward, provided they had the accessto do this kind of distillation work.
Ed (16:52):
So.
With all of the experience thatwe have in the college, throw a
toss up question out to you guys.
Um, when, when you heard aboutthis, what are some use cases?
Who wants to take that one?
Lee (17:01):
I'll chime in and, uh, you know, I, I think the, especially
around the chain of thought, right?
It lends itself, especially tocertain type use cases around
things like, um, analysis of dataor mathematics or business process.
Um, automation, those type ofthings, because it kind of lays it
(17:22):
out for you, what needs to happen,how it gets to that end point.
Jack (17:26):
Yeah, and I'll jump in on this one too.
I think one of the things that we'reseeing is that DeepSeek itself, uh,
may not necessarily be the rightmodel for all use cases, but the
way that it was constructed can be.
Then there's the other piece, which isthe distillation of really smart experts.
In this case, you know, distillingit from other expert models.
(17:50):
So the use cases we're seeingare not specific to deep sea.
Where I see that is it's creating areally interesting, um, you know, kind
of fork towards what customers aretrying to achieve with AI right now.
We've been heavily going downthis reasoning model, trying
to get to things like AGI.
But the other side is we'realso now trying to turn.
(18:10):
Uh, AI into an agentic responsecomponent bot or something that
can do something very, very well.
And so the way that the model wasconstructed, the way that we can
build small language models off ofthe large language models is, you
know, capable of having what I'llsay, dumber response models, but that
they do something very, very well.
And so I think that that's the,those are the use cases that we're
(18:32):
seeing is that the way that it wasconstructed opens up new ways for us
to work with customers to solve agenticuse cases in a unique way and deep
reasoning use cases in another way.
And so it creates to me, you know,really two paths you can go down, um, for
different business reasons, some that youmay want to have a guaranteed response
(18:53):
that you know that you're going to get.
A specific type of answer set that'sreasoned out and thought about in
an agentic side, or you want tobe able to have much more, um, you
know, kind of deeper thinking andthese deeper learning models in
these, this more rational approach.
Ed (19:09):
DeepSeek is also open source and, you know, some of
the other big players aren't.
And there's debate around the benefitsand drawbacks of open source AI models.
What's your opinion on that?
Jack (19:19):
The fastest way to create innovation is to get everybody
collaborating together.
So, I think, you know, we've kind oftalked about the benefits of being able
to see how deep sequence put together theother side of this is, you know, a lot of
large enterprises are not willing to havea deployed solution, whether it's an AI
large language model or an AI bot or evenjust a standard software operating system.
(19:43):
That's where that ability to have supportfor those is a really important aspect.
And so I think that it's from aninnovation and, uh, getting onto the
bleeding edge, um, having that abilityto kind of tie towards open source is
important, but having actually, um,supportable integrations is where you
(20:03):
have to move more towards a proprietaryintellectual property control model.
And I think we're seeingthis play out live right now.
We've talked a little bit about.
OpenAI is GPT, but, you know, the otheris, you know, Meta and their Llama model.
It's another open source model, and it'sthe foundation for a lot of, uh, of, of
models that are being used in, in certainapplications and use cases that can be
(20:27):
packaged into somebody else's output.
There's, there's really strongbenefits to having open source to
understand the foundation, but inpractical use, you do need to, you
know, really understand the supportmodels behind the open source model.
Jordan (20:40):
You know, having that public scrutiny on a code base and
a model from a security perspectiveis also really important, right?
Cause we never want to fall back into.
You know an echo chamber when it comesto how secure something is we want
to have people looking at it who Aresmarter than the original authors, right?
If we go back to you know, the thepurpose of open crypto models Right goes
(21:01):
all the way back to world war ii andthe enigma machine That the the german
nazis had built to encrypt their dataand they didn't think that the math on
that was breakable Because they weren'tas smart as alan turing right and until
he came around and worked that problemAnd then he broke it right and and the
same thing could happen with any cryptoalgorithm that's out there The same
thing can also happen with any model orany code base that we're working on now.
(21:22):
So having this secure mechanism ofmultiple people, of experts in each
individual topic looking at it, allowsus to provide a more secure basis to
then provide the support models andother things on top of, while still
having the scrutiny of the masses,on the level of security and privacy
that's being implied in the code itself.
Ed (21:42):
And Christopher was the name of Alan Turing's machine, just for
anybody who's on the tip of theirtongue and can't remember it right now.
So he just saved youa Google search there.
Um, One thing we want to talk aboutis the, the kind of the Jurassic Park
question, you know, just because we can,should we, you know, and the ethical
considerations around, around deep sea.
Jordan, sticking with you for a secondfrom a technical standpoint versus,
(22:03):
um, you know, kind of the geopolitical.
You know, aspect of DeepSeek,you know, how, how are customers
and all users supposed to, uh,you know, kind of balance that,
Jordan (22:12):
you know, we did run specifically, we wanted to run the chat
bot itself in a contained environment.
Right.
And so we loaded up virtualizationenvironments, ran it on the Android
system, and then watched all thecommunications that were coming out of it.
We wanted to validate, look at thisourselves, watch what was happening.
There are communicationsgoing directly back to China.
They are in fact goingto China mobile systems.
(22:33):
Right, which is an organization thatis banned from doing business in the
U. S. that the U. S. considers that asmuch of a tie to the Chinese Communist
Party and the military organizationsthere that we're not allowed to invest
in as individuals or businesses, um, andthey're banned from doing business here.
And there are definitely some ties.
between the DeepSeek organization andChina Mobile that we can very clearly
(22:55):
see that both exist in the code,that exist in the communications and
monitoring where the comms are going.
Now, a lot of people will saythat this doesn't necessarily
prove anything outright.
They're right.
But, you know, one of the thingsover years of security research
that I've learned is generally wherethere's smoke, um, there's fire.
And, you know, we've looked for morethan one signal of smoke here, and
(23:17):
we have found it in several places.
Right.
And the implications of that.
And then when we separate out the privacyconcern that separately we look at privacy
and regulatory compliance standards thatwe should be meeting as an organization.
Well, none of those apply to DeepSeek.
And so when we look at personal databeing uploaded there, they don't have
the same standards that need to be met.
(23:37):
And so you look at the standardsthat OpenAI, that XAI, that SONNET
and any other models that we're allinteracting with are having to meet.
And I think people need to be concernedabout what that looks like with their
data, what they're uploading and whatquestions they ask of the system.
Ed (23:56):
So my premise at the top of the podcast was that no one's going to
know where the next big thing comesfrom an AI it's DeepSeek now, it's
going to be something else later.
So Jack, wanted to turn to you, youknow, SHI is in the process now of
completing construction on a new AIand cyber labs facility designed to
be the place where customers can.
Do their evaluation and testing of A.I. And cyber security technologies.
(24:20):
Can you tell how you envision theA. I. And cyber security labs?
You know how customers deal with thismountain of new technologies that
are anticipated to come their way?
Jack (24:30):
Yeah, well, first of all, I like the softball question.
Talk about some of this newcapability that we're bringing about.
But like the practical element of thisis, you know, we have used the RAI
lab and cyber security environmentto be able to securely download the
DeepSeek large language model andoperate it outside of the app or the
(24:51):
chat bot that Jordan just went into.
And so that we had that abilityto Securely bring that in and
use it as a foundational modelin in really kind of validated.
And so having the capacity to be ableto pull the full model in and use it is
really an important capability that wehave that we can bring to our customers
(25:11):
to do things like validate, um, thepractical benefits of a model like this.
While also securing it from any of thatinformation being passed back to China
Mobile or the state itself in China.
So, you know, that is a reallygood example of how we're able
to leverage our AI and cyberlab to do this type of testing.
(25:33):
We don't stop there, you know, wewe're working with pretty much every
large language model that's out there.
We can pull down pretty much every oneof the millions of available trained
large language models that exist in theGitHub libraries and HuggyFace libraries.
And so we have that abilityto use any foundational model.
Um, and as I mentioned before, evenconsider things like a mixture of
(25:55):
models and using different modelsto different aspects of a workflow.
Um, and so I think that it's, it'sa really, um, you know, key time for
customers to need this sort of, um,validation work and confirmation work.
Because, you know, what we're seeingin the practical applications of AI and
leveraging things like generative AIand large language models is that, um,
(26:17):
it really matters, um, how those arearchitected and really matters ultimately
the type of infrastructure that you'rerunning them on to achieve success.
Ed (26:28):
Excellent.
And Jordan, maybe I'llstart start with with you.
What do you think the nearterm impact will be on DeepSeek
on the on the A. I market?
Jordan (26:37):
I think we'll see a lot of very use case driven distillation models
coming out over the next 6 to 12 months.
Um, and, and I think that's actuallyfantastic for, because I think one of
the things like Jack was talking about,a lot of things struggle post POC to
get in and use cases aren't clearlyenough to find, and that is one of the
problems we often face with getting realbusiness application, and so I think
(26:58):
people are going to take this idea ofusing distillation to generate a very
usable model and improve on this, right?
There's definitely some thingswe needed to learn, Right?
There's some problems with, youknow, the safety isn't necessarily
inherited from the teacher model.
Um, accuracy isn't alwaysinherited from the teacher model.
So there's more hallucinations,more jailbreaks.
And I think now we'll begin to tightensome of these things down and get better
(27:21):
at using distillation to create modelsand then get more effective use case
driven models that people are actuallygoing to start adopting at a higher pace.
Ed (27:30):
Lee, what do you think the impact is?
Do you agree and will you add to that?
Lee (27:35):
Yeah, I think, uh, it's going to bring some new innovation to the
market as, uh, as Jordan pointed outand how models are being, um, produced.
Absolutely.
I think, uh, There's going to becontinued pressure really from
a, uh, a market standpoint toget better efficiency in models.
I think all that's good.
I think there's, there's goingto be continued scrutiny of
(27:56):
DeepSeek, um, which I hope is true.
I, I would advise any companyto proceed with caution.
Validate what is being, um,put out from the model, right?
Because.
We're still looking at model bias, right?
Every, every model has bias.
We don't know what that's going to be.
Um, you know, we've seen clear, um,censorship, um, within the model.
(28:18):
Um, and so, you know, it's reallyproceed with caution with respect to
DeepSeek, but the way that the marketis going to go, everybody's going to
be pointing at DeepSeek and askingother producers, why can't you do this?
And I think that's, that's the innovationare going to drive the innovation.
Across the market.
Ed (28:36):
Absolutely.
And Jack, there's going to be alot more conversations about, about
DeepSeek and about every, everyother thing, AI and cyber related.
Um, what's your recommendationto any SHI customers who want
to continue this conversation?
You know, what, what, what's the bestway for them to do that and kind of
get, get in touch with you guys whoare talking to customers every day.
Jack (28:54):
Well, you said, first of all, I'd say it starts with the relationships
that you have with your local accountteams from SHI, and I'll note that.
You know, what, what's really, um,encouraging to me about, you know,
what DeepSeek exposed to the marketis we're going to be pushing the
innovation curve as rapidly as possible.
And we at SHI happened to already bewell in, well ahead of that curve.
(29:19):
Um, I'll, I'll say that if anythingthat DeepSeek proved to us was that
by intuning these models based onthings like distillation, chain
of thought, mixture of experts.
Like, that's where we are, and soSHI is there on the cutting edge.
We have the ability to help guideour customers in ways that, um,
really I don't believe there's anyoneelse in the market that does this.
Ed (29:43):
The industry has its shares of bulls and bears about DeepSeek,
which is sparking both excitement andskepticism, sometimes simultaneously.
Its rapid ascent in the AI sectorunderscores the evolving global landscape.
of artificial intelligence and showsthat the next big thing in AI could
come from anywhere with little warning.
It highlights both the potentialfor innovation and the necessity for
(30:05):
careful consideration of ethical,security, and geopolitical implications.
Is DeepSeek the future of AI,or just another hype circle?
Time will tell.
But one thing's for sure, theAI landscape is evolving fast.
A huge thank you to Jack Hogan,Lee Zeliak, and Jordan Mariello
for sharing their insights today.
(30:26):
Until next time, to the wolves ofwhatever street you're on out there,
For Innovation Heroes, I'm Ed McNamara,and I'll see you again in two weeks.
We wanted to run the chat bot itself in a contained environment, right?
And so we loaded up virtualizationenvironments, ran it on the Android
system, and then watched all thecommunications that were coming out of it.
We wanted to validate, look at thisourselves, watch what was happening.
There are communicationsgoing directly back to China.
They are in fact goingto China mobile systems.
Right, which is an organization thatis banned from doing business in the
(00:23):
U. S., that the U. S. considers that asmuch of a tie to the Chinese Communist
Party and the military organizationsthere that we're not allowed to invest
in as individuals or businesses, um, andthey're banned from doing business here.
Ed (00:36):
In the world of technology, heroes are everywhere.
They're overcoming disruption, deliveringsustainable outcomes, and fearlessly
forging the future to solve what's next.
Join me, Ed McNamara, as we meet thepeople and businesses driving change
in our constantly disruptive world.
This is Innovation Heroes, apodcast brought to you by SHI.
(01:01):
Matthew McConaughey famously said.
that the number one rule of Wall Streetwas that nobody knows if a stock is
going up, down, sideways, or in circles.
Predicting where the next big thing in AImight come from kind of feels the same.
The most recent market disruptingsurprise in AI came from China,
and its name is DeepSeek.
In January, DeepSeek launched with abang, racking up 16 million downloads
(01:24):
in just 18 days, nearly double ChatGPT'sinitial release numbers, with claims it
was developed at literally a fractionof the cost of other AI models.
That combination wiped out hundredsof billions of market cap on a day on
Wall Street that would have surprisedboth Jimmy and Warren Buffett.
Some are calling DeepSeek the cuttingedge of AI innovation, while others are
raising eyebrows over some big concerns.
(01:47):
Today we're breaking down what makesDeepSeek unique, how it stacks up against
other platforms, and how companiesmight consider governing the risks
and promise of technologies like it.
To help us do that, we'rejoined by three panelists.
Jack Hogan, Lee Ziliak,and Jordan Mariello.
Jack Hogan is Vice Presidentof Advanced Growth Technologies
at SHI International, leadinginitiatives in AI and cybersecurity.
(02:11):
Lee Ziliak is SHI's FieldTechnology Officer and Managing
Director of Architecture.
And last but certainly notleast is Jordan Mariello.
Jordan is Chief StrategyOfficer and Head of Stratascale.
Bringing over 25 years of cybersecurity expertise to the role.
Jack, Lee, Jordan, welcometo Innovation Heroes.
Jack (02:30):
Hey Ed, thanks for having us on today.
Thanks Ed.
Ed (02:33):
Absolutely.
So, gentlemen, today we're talkingabout DeepSeek, but before we dive
in, I want to hear a little bit abouteach of your professional backgrounds.
You all have a mix of experienceand expertise in machine
learning, generative AI.
It and cloud computing and probablyevery other emerging technology
for the past, you know, 20 plusyears, Jack, we'll start with you.
Tell me a little bit about your backgroundand how you got to where you are today.
Jack (02:55):
Yeah, thanks.
And my actually historygoes back about 30 years.
And when I first came out ofcollege, I actually wrote a neural
network program to predict themovement of the Swiss Frank future.
So this is leveraging some ofthe early scale machine learning.
capabilities that we're nowreally seeing at scale with
things like DeepSeek and GPT.
(03:16):
But I spent the bulk of my careeron the, as a customer side co
founder and CTO in a consumer cloudand SAS company went through a
number of the main waves of Web 1.
0, Web 2.
0, early stages of virtualization,storage technologies, and ultimately
dealing with large data and data sets.
And so I spent really a seven year.
(03:37):
Time with, uh, one of my corebusiness partners in that, which
was Pure Storage, where I led, Iwas their VP of technology strategy.
And in that role, I understood theimportance of the physics of, uh,
dealing with data processing at scale.
Um, and I joined SHI, uh, a littleless than a year ago to lead the
advanced growth technologies team.
So from my point of view, I've livedon the customer side, I've lived
(03:58):
on the vendor side, and really now.
At SHI, helping customers navigatethrough some of the bigger challenges
as it relates to next generationtechnologies and the infrastructure
and components that go into that.
Ed (04:10):
Excellent.
And Lee, what about you?
Lee (04:13):
Well, Ed, you were about half, halfway there when you said 20 years.
I've been in the industryfor about 40 years.
Um, starting right out of high school,just like Jack and working my way
through college, um, started, uh,working for the government doing, uh,
various things in such old languagesas like assembler language and COBOL,
but then spent the bulk of my careerat, uh, Verizon, um, where I ran server
(04:38):
storage and mainframe architecture.
Um, led the charge into the, the publiccloud and private cloud at Verizon.
And that's really, uh, what led meto SHI was, uh, you know, SHI was
standing up a cloud business, so Icame over, uh, and started the cloud
professional services business, uh, at,at SHI, um, for enterprise customers.
(05:00):
Um, and then after that, I, um,moved over into our global segment
and then joined Jack, uh, in,uh, a GT in July when he, uh.
Came on board and started thatgroup because, you know, it
was, uh, focused on, uh, AI andlarge data center opportunities.
And that's kind of my sweet spot that I'vebeen in for, uh, most of those 40 years.
Ed (05:21):
Awesome.
And Jordan, how about you?
Jordan (05:23):
Had a similar story here.
Kind of started my career, uh,straight out of high school for me.
I went into the military and working,uh, communication, security, crypto
cyber there, kind of doing the sameexact job for the government afterwards.
Many people know when they moveover to the contractor side, you
do the same work and just get paida little bit better for it and
deploy a little bit less, hopefully.
Um, and then I moved over to thecommercial side after that and I
(05:46):
spent a few years working in financialservices, ended up leading a global
security operations center for avery large financial organization.
Um, and then, uh, decided to stepout and join some partners and jump
into the entrepreneur real world.
And, uh, so did, uh, a few startupbusinesses, kind of spun out two
from one main one and ended upbuilding three separate companies.
(06:06):
But did some platform work,um, and building a security
orchestration platform.
Um, then, uh, built a managed detectionand response business, Critical Start,
fairly well known one in that space.
And through that time, there's kind ofthis intermingling of building software
on machine learning and technology todo security analytics in particular.
(06:27):
So we approached a lot of that from anAI and machine learning perspective.
Um, and whether that's, youknow, developing neural networks
or using a lot of clusteringalgorithms, implementing those.
From an AI perspective intosecurity orchestration platforms
and improving detection.
Ed (06:43):
So, all right, DeepSeek.
Lee, I'll look in your direction to startwith, you know, when you first heard
that news, what were your first thoughts?
Lee (06:52):
Um, well, I thought it was actually pretty exciting, right?
I, you know, if all the claimswere true, you know, what they
did was kind of revolutionary.
Uh, being the, the geek that Iam, I, I soon started pulling back
the covers to kind of understand,all right, what, what in this is
real, uh, and what is, what is not.
And so, you know, I'm not sayingthat the claims are false.
(07:17):
A lot of people have, but if youstart reading the details, like
the claim of 6 million to train.
The model, right?
That that's true, but itwas only the final run.
There's like months and yearsof work that went into that that
everybody else accounts for that thatwasn't in that final cost, right?
They built their model based onwork that other companies have done.
(07:37):
So yeah, of course, it'sgoing to be cheaper.
It's revolutionary, right?
It's a new way to build a model.
So from that perspective,it's super exciting.
I think it's going to revolutionizethe way that, uh, you know, other
companies are looking at buildingmodels, but I think everything
kind of needs to be put in context.
So, um, like I said, excitingstuff, but really just need to
(07:59):
needed to pull back the layers tounderstand it a little bit better.
Ed (08:02):
Jordan, what were your first thoughts when you heard the news?
Jordan (08:06):
I probably a little bit of disbelief I had coming
from the security world.
I probably my first bent is to beskeptical, um, of releases and innovation
announcements of the sort like this.
And we see them a lot in thesecurity world and we've solved
the security problem of X, right?
And then it's our job tobe super skeptical of that.
So I think I dove in prettyquickly to say, okay, well, what
(08:26):
do we think really happened here?
Um, and you know, isthat dollar amount true?
And like Lee was saying, youknow, it kind of seems to be.
Only enumerating an end of, uh,maturity or end of release phase,
specifically a training the model anddoing distillation, but there's things.
Going into that, that led up to that,that probably accrued significant
costs before that, that I don't thinkare accounted for in that number.
(08:49):
However, I also did the same point thatthis is a very well trained model and it
does have, um, some unique innovations,uh, around the reasoning processes is
there that, that are worth knowing.
I think drew attention for a reason.
Um, but then my next immediate thoughtafter that is, okay, what are the
security implications of where thisis housed and who's running this?
(09:10):
And what do we need to beworried about in that world too?
Ed (09:12):
And Jack turning to you, you know, can you, you summarize
what the AI model impact reallyon, on, I'll say on the market?
Jack (09:19):
Well, yeah, I think from a market point of view, it introduced a, well,
I'll say, I wouldn't say, I guess a totalquantum leap, but it, it, it represented
a very big leap forward in the waythat models can actually be designed.
Um, and you know, conceptsthat we'll get into things like
distillation and chain of thought.
You know, those are, those areimportant foundational building blocks.
(09:40):
Um, I think though that when we lookat the market impact of it, because
the model was delivered as an opensource model, it actually created
a lot of positive impact for themarket, despite the actual financial
markets seeing the negativity of it.
The positives is that we all are now ableto look at how the model was put together
(10:01):
and And really deconstruct some of thosecomponents to extract really that, that,
you know, moving models forward or moving,you know, the AI development, those were
some of the moments as we're, you know,continuing to go, you know, grow towards
AGI or some of the broader conceptsof machine learning and artificial
intelligence outdoing human intelligence.
(10:22):
So from my point of view, I thinkfrom a market perspective, it really
created, um, that, that flashpoint of,okay, now we can look at developing
different types of models differently.
And I think it was a large enough, uh,global impact that pretty much everyone's
taking it, paying attention to it.
And I think rapidly turning towardshow do we take advantage of it?
(10:42):
This is actually a great thing.
for the rapid development that'smoving, coming ahead of us.
And, uh, you know, so I thinkthe excitement in terms of
what it means in the long termis, I think, really exciting.
Ed (10:54):
So, Jack, maybe we'll stay with you for a second.
Um, let's talk about some of DeepSeek's AItechniques that are potential advantages
over platforms like ChatGPT or Gemini.
Um, you know, one of thetechniques this model uses is
called Mixture of Experts, or MOE.
Um, can you explain, like, whatthat is and how is it different
from, say, ChatGPT's model?
Jack (11:14):
Well, when you look at the way neural networks are put together and
the different concepts that have tobe, um, trained and, and operating
together, being able to tie togetherthese experts and mix them in one model,
leveraging the distillation of someother models, and then, you know, turning
to a more deeper reasoning response,I think that, you know, one of the
(11:36):
things that's really important aboutwhen you look at bringing in a bunch of
experts into any practical sense, You,you, you learn a lot by having those
experts kind of work with each other.
Similarly in AI model development,that's kind of the practical element
of how a mixture of experts can cometogether and open up new ways to tie
together previously trained componentsor net neural nets within that, you
(11:58):
know, that fact that they relied on alot of the mixture of expert components.
I think where we're, we're lookingat that next phase is how does that
move into the mixture of models, um,you know, different models that are
different type of experts that aretrained to do very specific things.
And I think there's some, there's somereally important elements as we look
at how we move into this concept ofagentic AI, um, and where you want a
(12:21):
less reasoned model to do a very definedresponse, you know, question response.
Um, being able to have mixtures ofexperts and mixtures of models start
to become a really important way that,uh, AI can be applied beyond deep
reasoning or deep learning models.
Ed (12:36):
And Lee, another difference was something called chain of thought or COT.
Uh, can you kind of explain what that isand what maybe some of the advantages are?
Lee (12:45):
So chain of thought is exactly what it sounds like, right?
So the model, um, takes yourquery or the, the problem it needs
to solve and it breaks it down.
Into different steps of what it callsthoughts, and then it starts to reason
through each one of those thoughtsuntil it gets through to the answer.
Um, so it's actually, you know, it'sinteresting to watch in a chat bot because
(13:09):
I joked with Jack when I first saw it.
I'm like, I think this model wastrained by my college math teacher
because it has to show me everythingalong the way to get to the answer.
So that's, I mean, that'sgood and bad, right?
So, but the, the good piece of thatis it's, it increases transparency.
It, uh, increases accuracy, right?
So as it works through these differentproblems, the accuracy level of the model
(13:32):
is actually increased, so it's improved.
Um, and so you can see everything thatthe model goes through to get to its
answer, and it'll, uh, work through fromone thought to the other to increase
that accuracy, so that's, that'skind of what the chain of thought is.
Starting with a problem and here'sall the different steps that I
worked through to get to the answer.
(13:54):
Um, in the, you know, like I said,there's benefits to it in terms
of transparency and accuracy.
There's downsides to it.
Right.
So the, as I said, the outputtends to be very verbose.
And so if you're look, you're lookingfor a very, you know, quick or, or, or
short answer, um, you know, it's notquite as easy to swap out GPT and put in.
(14:15):
Uh, deep seat because the outputis basically completely different.
The other impact of that is going throughthe chain of thought process is also
tends to be more resource intensive.
Um, and so, you know, the hardware thatis required, especially on the much larger
models, um, is going to be impacted.
Now, that's sometimes that'soffset by the, like I said,
(14:36):
the implementation of the MOE.
Um, in the distillation, um, but italso is important because of that to
verify the claims of efficiency thatdeep sea claimed when they put out
the model, they claimed that it wasn'tonly efficient to train, but it was
also that it runs more efficient.
And so that's one of the thingsthat we're doing is we put DeepSeek
(14:57):
into our, um, next gen AI labis we're verifying that impact.
Ed (15:03):
So we've got, um, mixture of experts, MOE, we've got chain of
thought, COT, Jordan, the, the otherthat was mentioned was distillation.
Can you, can you expand on that andmaybe some of the benefits on that?
Jordan (15:15):
Yeah, you know, distillation is not necessarily dissimilar for some
things we're already familiar with, liketraining small language models on LLMs.
It's like.
We do this a lot for security, right?
There's a lot of securityspecific data that the LLM doesn't
necessarily have context about.
So you have a large languagemodel like OpenAI's ChatGBT, you
know, massive data set extensivelytrained to get this high performance
(15:38):
for generating human like text.
And that's what we wanted to do.
But then we can take and work off oftop of that model and use that in what
we call a teacher student relationship.
And then we use the parent or theteacher model to generate Soft
targets, meaning we're asking specificquestions to get specific outputs when
we don't just want that one output.
We want to get some of the otherpossible outputs that were also there.
(16:01):
And this allows us to build a littlebit more targeted, but more efficient
data set that doesn't need to be.
as extensive of a database that'sbeen trained on from the beginning,
and therefore there's an efficiencygain there in the training
process that's fairly significant.
And I think there's substantivedata that supports the idea that
(16:22):
DeepSeek's DeepThinkingR1 model isdistilled from OpenAI's ChatGPT.
And I think it also shows, like Jackwas saying, like, this is a process
that really does work to generate ahigh functioning model that is very
useful for business applications.
Um, and I think it helps to bend thecurve a little bit on the efficiency,
(16:43):
especially from a cost perspective,on what people will need to do moving
forward, provided they had the accessto do this kind of distillation work.
Ed (16:52):
So.
With all of the experience thatwe have in the college, throw a
toss up question out to you guys.
Um, when, when you heard aboutthis, what are some use cases?
Who wants to take that one?
Lee (17:01):
I'll chime in and, uh, you know, I, I think the, especially
around the chain of thought, right?
It lends itself, especially tocertain type use cases around
things like, um, analysis of dataor mathematics or business process.
Um, automation, those type ofthings, because it kind of lays it
(17:22):
out for you, what needs to happen,how it gets to that end point.
Jack (17:26):
Yeah, and I'll jump in on this one too.
I think one of the things that we'reseeing is that DeepSeek itself, uh,
may not necessarily be the rightmodel for all use cases, but the
way that it was constructed can be.
Then there's the other piece, which isthe distillation of really smart experts.
In this case, you know, distillingit from other expert models.
(17:50):
So the use cases we're seeingare not specific to deep sea.
Where I see that is it's creating areally interesting, um, you know, kind
of fork towards what customers aretrying to achieve with AI right now.
We've been heavily going downthis reasoning model, trying
to get to things like AGI.
But the other side is we'realso now trying to turn.
(18:10):
Uh, AI into an agentic responsecomponent bot or something that
can do something very, very well.
And so the way that the model wasconstructed, the way that we can
build small language models off ofthe large language models is, you
know, capable of having what I'llsay, dumber response models, but that
they do something very, very well.
And so I think that that's the,those are the use cases that we're
(18:32):
seeing is that the way that it wasconstructed opens up new ways for us
to work with customers to solve agenticuse cases in a unique way and deep
reasoning use cases in another way.
And so it creates to me, you know,really two paths you can go down, um, for
different business reasons, some that youmay want to have a guaranteed response
(18:53):
that you know that you're going to get.
A specific type of answer set that'sreasoned out and thought about in
an agentic side, or you want tobe able to have much more, um, you
know, kind of deeper thinking andthese deeper learning models in
these, this more rational approach.
Ed (19:09):
DeepSeek is also open source and, you know, some of
the other big players aren't.
And there's debate around the benefitsand drawbacks of open source AI models.
What's your opinion on that?
Jack (19:19):
The fastest way to create innovation is to get everybody
collaborating together.
So, I think, you know, we've kind oftalked about the benefits of being able
to see how deep sequence put together theother side of this is, you know, a lot of
large enterprises are not willing to havea deployed solution, whether it's an AI
large language model or an AI bot or evenjust a standard software operating system.
(19:43):
That's where that ability to have supportfor those is a really important aspect.
And so I think that it's from aninnovation and, uh, getting onto the
bleeding edge, um, having that abilityto kind of tie towards open source is
important, but having actually, um,supportable integrations is where you
(20:03):
have to move more towards a proprietaryintellectual property control model.
And I think we're seeingthis play out live right now.
We've talked a little bit about.
OpenAI is GPT, but, you know, the otheris, you know, Meta and their Llama model.
It's another open source model, and it'sthe foundation for a lot of, uh, of, of
models that are being used in, in certainapplications and use cases that can be
(20:27):
packaged into somebody else's output.
There's, there's really strongbenefits to having open source to
understand the foundation, but inpractical use, you do need to, you
know, really understand the supportmodels behind the open source model.
Jordan (20:40):
You know, having that public scrutiny on a code base and
a model from a security perspectiveis also really important, right?
Cause we never want to fall back into.
You know an echo chamber when it comesto how secure something is we want
to have people looking at it who Aresmarter than the original authors, right?
If we go back to you know, the thepurpose of open crypto models Right goes
(21:01):
all the way back to world war ii andthe enigma machine That the the german
nazis had built to encrypt their dataand they didn't think that the math on
that was breakable Because they weren'tas smart as alan turing right and until
he came around and worked that problemAnd then he broke it right and and the
same thing could happen with any cryptoalgorithm that's out there The same
thing can also happen with any model orany code base that we're working on now.
(21:22):
So having this secure mechanism ofmultiple people, of experts in each
individual topic looking at it, allowsus to provide a more secure basis to
then provide the support models andother things on top of, while still
having the scrutiny of the masses,on the level of security and privacy
that's being implied in the code itself.
Ed (21:42):
And Christopher was the name of Alan Turing's machine, just for
anybody who's on the tip of theirtongue and can't remember it right now.
So he just saved youa Google search there.
Um, One thing we want to talk aboutis the, the kind of the Jurassic Park
question, you know, just because we can,should we, you know, and the ethical
considerations around, around deep sea.
Jordan, sticking with you for a secondfrom a technical standpoint versus,
(22:03):
um, you know, kind of the geopolitical.
You know, aspect of DeepSeek,you know, how, how are customers
and all users supposed to, uh,you know, kind of balance that,
Jordan (22:12):
you know, we did run specifically, we wanted to run the chat
bot itself in a contained environment.
Right.
And so we loaded up virtualizationenvironments, ran it on the Android
system, and then watched all thecommunications that were coming out of it.
We wanted to validate, look at thisourselves, watch what was happening.
There are communicationsgoing directly back to China.
They are in fact goingto China mobile systems.
(22:33):
Right, which is an organization thatis banned from doing business in the
U. S. that the U. S. considers that asmuch of a tie to the Chinese Communist
Party and the military organizationsthere that we're not allowed to invest
in as individuals or businesses, um, andthey're banned from doing business here.
And there are definitely some ties.
between the DeepSeek organization andChina Mobile that we can very clearly
(22:55):
see that both exist in the code,that exist in the communications and
monitoring where the comms are going.
Now, a lot of people will saythat this doesn't necessarily
prove anything outright.
They're right.
But, you know, one of the thingsover years of security research
that I've learned is generally wherethere's smoke, um, there's fire.
And, you know, we've looked for morethan one signal of smoke here, and
(23:17):
we have found it in several places.
Right.
And the implications of that.
And then when we separate out the privacyconcern that separately we look at privacy
and regulatory compliance standards thatwe should be meeting as an organization.
Well, none of those apply to DeepSeek.
And so when we look at personal databeing uploaded there, they don't have
the same standards that need to be met.
(23:37):
And so you look at the standardsthat OpenAI, that XAI, that SONNET
and any other models that we're allinteracting with are having to meet.
And I think people need to be concernedabout what that looks like with their
data, what they're uploading and whatquestions they ask of the system.
Ed (23:56):
So my premise at the top of the podcast was that no one's going to
know where the next big thing comesfrom an AI it's DeepSeek now, it's
going to be something else later.
So Jack, wanted to turn to you, youknow, SHI is in the process now of
completing construction on a new AIand cyber labs facility designed to
be the place where customers can.
Do their evaluation and testing of A.I. And cyber security technologies.
(24:20):
Can you tell how you envision theA. I. And cyber security labs?
You know how customers deal with thismountain of new technologies that
are anticipated to come their way?
Jack (24:30):
Yeah, well, first of all, I like the softball question.
Talk about some of this newcapability that we're bringing about.
But like the practical element of thisis, you know, we have used the RAI
lab and cyber security environmentto be able to securely download the
DeepSeek large language model andoperate it outside of the app or the
(24:51):
chat bot that Jordan just went into.
And so that we had that abilityto Securely bring that in and
use it as a foundational modelin in really kind of validated.
And so having the capacity to be ableto pull the full model in and use it is
really an important capability that wehave that we can bring to our customers
(25:11):
to do things like validate, um, thepractical benefits of a model like this.
While also securing it from any of thatinformation being passed back to China
Mobile or the state itself in China.
So, you know, that is a reallygood example of how we're able
to leverage our AI and cyberlab to do this type of testing.
(25:33):
We don't stop there, you know, wewe're working with pretty much every
large language model that's out there.
We can pull down pretty much every oneof the millions of available trained
large language models that exist in theGitHub libraries and HuggyFace libraries.
And so we have that abilityto use any foundational model.
Um, and as I mentioned before, evenconsider things like a mixture of
(25:55):
models and using different modelsto different aspects of a workflow.
Um, and so I think that it's, it'sa really, um, you know, key time for
customers to need this sort of, um,validation work and confirmation work.
Because, you know, what we're seeingin the practical applications of AI and
leveraging things like generative AIand large language models is that, um,
(26:17):
it really matters, um, how those arearchitected and really matters ultimately
the type of infrastructure that you'rerunning them on to achieve success.
Ed (26:28):
Excellent.
And Jordan, maybe I'llstart start with with you.
What do you think the nearterm impact will be on DeepSeek
on the on the A. I market?
Jordan (26:37):
I think we'll see a lot of very use case driven distillation models
coming out over the next 6 to 12 months.
Um, and, and I think that's actuallyfantastic for, because I think one of
the things like Jack was talking about,a lot of things struggle post POC to
get in and use cases aren't clearlyenough to find, and that is one of the
problems we often face with getting realbusiness application, and so I think
(26:58):
people are going to take this idea ofusing distillation to generate a very
usable model and improve on this, right?
There's definitely some thingswe needed to learn, Right?
There's some problems with, youknow, the safety isn't necessarily
inherited from the teacher model.
Um, accuracy isn't alwaysinherited from the teacher model.
So there's more hallucinations,more jailbreaks.
And I think now we'll begin to tightensome of these things down and get better
(27:21):
at using distillation to create modelsand then get more effective use case
driven models that people are actuallygoing to start adopting at a higher pace.
Ed (27:30):
Lee, what do you think the impact is?
Do you agree and will you add to that?
Lee (27:35):
Yeah, I think, uh, it's going to bring some new innovation to the
market as, uh, as Jordan pointed outand how models are being, um, produced.
Absolutely.
I think, uh, There's going to becontinued pressure really from
a, uh, a market standpoint toget better efficiency in models.
I think all that's good.
I think there's, there's goingto be continued scrutiny of
(27:56):
DeepSeek, um, which I hope is true.
I, I would advise any companyto proceed with caution.
Validate what is being, um,put out from the model, right?
Because.
We're still looking at model bias, right?
Every, every model has bias.
We don't know what that's going to be.
Um, you know, we've seen clear, um,censorship, um, within the model.
(28:18):
Um, and so, you know, it's reallyproceed with caution with respect to
DeepSeek, but the way that the marketis going to go, everybody's going to
be pointing at DeepSeek and askingother producers, why can't you do this?
And I think that's, that's the innovationare going to drive the innovation.
Across the market.
Ed (28:36):
Absolutely.
And Jack, there's going to be alot more conversations about, about
DeepSeek and about every, everyother thing, AI and cyber related.
Um, what's your recommendationto any SHI customers who want
to continue this conversation?
You know, what, what, what's the bestway for them to do that and kind of
get, get in touch with you guys whoare talking to customers every day.
Jack (28:54):
Well, you said, first of all, I'd say it starts with the relationships
that you have with your local accountteams from SHI, and I'll note that.
You know, what, what's really, um,encouraging to me about, you know,
what DeepSeek exposed to the marketis we're going to be pushing the
innovation curve as rapidly as possible.
And we at SHI happened to already bewell in, well ahead of that curve.
(29:19):
Um, I'll, I'll say that if anythingthat DeepSeek proved to us was that
by intuning these models based onthings like distillation, chain
of thought, mixture of experts.
Like, that's where we are, and soSHI is there on the cutting edge.
We have the ability to help guideour customers in ways that, um,
really I don't believe there's anyoneelse in the market that does this.
Ed (29:43):
The industry has its shares of bulls and bears about DeepSeek,
which is sparking both excitement andskepticism, sometimes simultaneously.
Its rapid ascent in the AI sectorunderscores the evolving global landscape.
of artificial intelligence and showsthat the next big thing in AI could
come from anywhere with little warning.
It highlights both the potentialfor innovation and the necessity for
(30:05):
careful consideration of ethical,security, and geopolitical implications.
Is DeepSeek the future of AI,or just another hype circle?
Time will tell.
But one thing's for sure, theAI landscape is evolving fast.
A huge thank you to Jack Hogan,Lee Zeliak, and Jordan Mariello
for sharing their insights today.
(30:26):
Until next time, to the wolves ofwhatever street you're on out there,
For Innovation Heroes, I'm Ed McNamara,and I'll see you again in two weeks.
Popular Podcasts
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com