Rafi Qumsieh

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
Greetings and welcome to the United States Transhumanist Party Virtual
Enlightenment Salon. My name is Jannati Stolieroth the second and
I am the Chairman of the US Transhumanist Party. Here
we hold conversations with some of the world's leading thinkers
in longevity, science, technology, philosophy, and politics. Like the philosophers

(00:22):
of the Age of Enlightenment, we aim to connect every
field of human endeavor and arrive at new insights to
achieve longer lives, greater rationality, and the progress of our civilization. Greetings,
ladies and gentlemen, and welcome to our US Transhumanist Party
Virtual Enlightenment Salon, streaming on Sunday, September seventh, twenty twenty five.

(00:48):
Today we have a fascinating conversation in store for you
about agentic AI. How many of us have been faced
with the predicament that we have so many tasks that
we would like to accomplish and only so much time
in which to do them. I know this is a
constant predicament for me and probably for many of you. Thankfully,

(01:10):
there are some impressive agentic artificial intelligence technologies that are
coming online to help us with that situation. So joining
us today we have our panel of US Transhumanist Party officers,
including our Director of Visual Art Art Ramon Garcia, and
our Director of Scholarship, doctor Dan Elton. And our special

(01:34):
guest today is Rafi Kumcia. He is the founder and
CEO of Breakthrough Labs and Breakthrough Labs is specializing in
the development of agentic AI. So Rafi welcome. You have
a presentation to share with us, so please proceed.

Speaker 2 (01:52):
See yes, sir, thank you for having me.

Speaker 3 (01:55):
And I do love the values of the transfer Humanist
Party stuff. So yeah, as Janadi said, I'm the founder
of Breakthrough Labs and can.

Speaker 2 (02:07):
You see my slides by the way, yes we can, okay.

Speaker 3 (02:10):
So our machine in a nutshell is to accelerate scientific discovery,
preferably using AI of course, and specifically using agentic AI
because of its capabilities or at least it's coming up capabilities.
We're based in Austin, Texas. Here's my email, here's the
website if you want to go and take a look
at what we do. We you know, how we see things,

(02:33):
our vision, things like this, So let's go ahead and
get started. So you know, AI agents have a lot
of definitions, right and you know, as the field evolves,
were still figuring out, you know, what it means to
be an AI agent.

Speaker 2 (02:50):
But here's a working definition that I think a lot of.

Speaker 3 (02:52):
People in the field that actually build AI agents can
agree with. So an AI agent contains the following compos So,
first is a system message, which you can think of
it as just like a text that you set and
prepen to the agent. You basically give it its persona, you
tell it you know what the agent is, what is
capable love how to think about things, And think of

(03:15):
it as like the main algorithm that the agent is
going to use to kind of figure out tasks as
you as you throw tasks at it, and you're going
to specify there how the agent is going to choose
and handle tools depending on you know, its incoming information.
And then you have the LM of course, which is
the brain of the agent, right, it's basically it's thinking

(03:37):
or reasoning engine.

Speaker 2 (03:38):
These are you're typically you know, GPT, Claude.

Speaker 3 (03:42):
Whatever, you you know, all these large language monegs. And
then very importantly are the tools that the agent can use.
We basically these are the possible actions that the agent
can take. Do you want the agent to read files.
Do you want it to be able to write fast?
Do you want it to search the internet? Do you
want it to access in you know, a database, or
make a call to a service like.

Speaker 2 (04:06):
You know, a.

Speaker 3 (04:06):
Journal or something to pull some papers, research papers, things
like this. So there are so many tools, and there
are website where we can actually.

Speaker 2 (04:16):
Browse tools built by people, which is really cool.

Speaker 3 (04:18):
And you know, you can pick and choose which tools
you want to add to your agent. And of course
there are some other capabilities like planning and memory, and
these can be framed these tools too. You can say that, hey,
like here's a service that you can call to make
a plan or write a plan or create a plan,
or you can simply write a plan to a file

(04:39):
and then you know, refer to that plan later. And
of course memory is a big deal for agents. We'll
talk about this later, but memory is the next frontier
in my opinion, and I think even open AI, if
you you know, follow you would say that, you know,
Sam Adman's also mentioning that memory is the thing they're
going to be working on.

Speaker 2 (04:57):
And we'll talk about this later. Okay, So just to
kind of.

Speaker 3 (05:01):
Give you an idea of what an agent is And
since I can't see you, guys, if you have any question,
please feel free to stop me and ask. Okay, I'll
be happy to answer. So here's a typical research process
or a pipeline. Right, you got the idea generation in
literature review. Then you go into planning and experiment design.

(05:21):
Then you prepare the experiment and you execute it and
then you get the results. Then you do some data analysis,
and then you write the report and you know some
people review it and that you know. That's basically a
pipeline of how research is done. And of course AI
agents can help us along every step of the way,
right almost, So I can tell you that for an

(05:44):
idea generation, for example, and literature review, you could have
an AI agent go out and collect all the relevant
papers to the topic you're interested in, right, Or maybe
you have a research problem and you want the AI
agent to you know, the AI agent has been trained
on so many data, so a lot of data, almost
all the internet, right, so it can it can bring

(06:05):
some connections maybe you haven't thought about, maybe knowledge gaps
that it can see that.

Speaker 2 (06:11):
Maybe it's hard for one human too, you know.

Speaker 3 (06:15):
See, so this is one thing that agents can help
in in research planning and experiment design. Of course, you
could tell it you know what the objective is. It
could go ahead and like create an experiment, define all
the tests you need to do, and all you know
the data analysis you will need to do.

Speaker 2 (06:31):
Right, These are things that I think AI in.

Speaker 3 (06:33):
General is capable of with some guidance, of course, and
then experiment execution, well, that depends.

Speaker 2 (06:40):
Is your experiment a wet lab.

Speaker 3 (06:42):
We're not still there yet, even though some companies are
using robotics and stuff to do this and automating the process.

Speaker 2 (06:48):
But if your experiment is more of like an experiment that.

Speaker 3 (06:51):
Involves code or things like this, definitely AI agents could
speed up the process, right because they could write the code,
run the test, and report to you things like this.
Analysis also something that they can help in because they
can write code obviously, and you know, if you have
access to the data, you can write good code. You
have a good understanding of statistics on all these mathematical

(07:14):
models and techniques, you can definitely write code. But this
is something that I want to expand on because when
I say you know, they could do things, it comes
with caveats, right, and they're still not perfect.

Speaker 2 (07:25):
But I'm just giving the.

Speaker 3 (07:26):
General idea of what they can do, and then you
have a report writing. Of course they can write you
know this depending on you know what you feed them.
They can write good stuff and then pay per review. Well,
they can definitely critique and give you some things. But
we still can't take anything they do for granted, and
we'll talk about why later hopefully. Okay, So this is

(07:50):
kind of an overview of a typical research process, right, so,
and how agents can help you in each each part. Okay,
So I want to talk about the work we're doing
that right through labs. So we're focused on two main
avenues of this process. The first one is hypothesis generation
and the second one is data analysis. And I think

(08:11):
for hypothesis generation, we want the AI to brainstorm truly
novel and high quality hypothesis to research problems. I think
for me personally, the most difficult problem for humans is
actually coming up with a really high quality solution to
a problem or hypothesis that you can test, right, because
it takes thinking outside the box. Now, we as humans

(08:34):
are good at this, but can we even speed this up?
Can we even enhance it? Can we collaborate with AI
on you know, doing this back and forth forth, the
AI could come up with something really nice and you know,
we could say, ah, this works. This not I feel
like this is a really good place for AI to
come in and help us. And the second one is

(08:57):
data analysis, and ideally we want the AI to be
able to successfully run thorough analysis of complex data sets.
And I say successfully because that is the main problem.

Speaker 2 (09:09):
Right.

Speaker 3 (09:09):
So AI can write code, but a lot of the
times the code is you know, there's a bulk or
something and the AI cannot debug it and humans have
to step in and actually do the work. And this
is not good from an automation perspective because we want
now we still want the human in the loop. We

(09:29):
want the AR to report to the human, but we
don't want the human stuck in debugging small you know,
error problems or our library that's not updated or things
like this. Right, So this is a big area that
AI agents need to improve in, which is actually writing
successfully writing code death rons without intervention human intervention. So

(09:50):
these are the two things that we are interested in,
and we feel sure because these are the things that
can be automated with AI in AI agents generally, right,
And I'll show you kind of how we're working on
this basically, does anyone have any questions?

Speaker 2 (10:05):
So far? Are we good?

Speaker 1 (10:09):
I think we're good.

Speaker 3 (10:11):
Okay, cool, thank you. Okay, So just a bit about
me found of breakthrough left. You know this, My education
is in math and physics, and you know, I'm very
interested in solving the mysteries of the universe. It's it's
a really fascinating place.

Speaker 2 (10:30):
It's insane.

Speaker 3 (10:31):
And after I graduated, back then, AI was not what
it is now, but you know, you could start seeing
glimpses of these systems were you know, you just feed
them data and they learn from the data and they
pick up the pattern and they start generalizing, and you know,
you could ask them something and they could predict something
that was out of the data distribution.

Speaker 2 (10:51):
And I just thought, this is really fascinating.

Speaker 3 (10:53):
And you know, when the day comes and we can
use this in science, imagine how much we can progress science.
So that was kind of the thing, and I kept
this with me as I went through my career. So
I after that, I taught as a college teacher, teaching mathematics.
I worked as an aim A scientist at General Motors,
and I'm also an independent researcher on AI. In addition

(11:17):
to that, I tried some startups outside AI and science
that were just fun and this is basically kind of
where you know, where I spent most of my time.
But when these lllms became commercially available via APIs, that
was my time to say, Okay, this is the time
to start actually building something on top of them.

Speaker 2 (11:36):
Right.

Speaker 3 (11:37):
So I'm uh as there's a there's a channel on
YouTube two minute papers.

Speaker 2 (11:43):
I don't know if you.

Speaker 3 (11:44):
Guys know about it, but the guy's signatures think that
he says, what a time to be alive, right, because
we're really seeing some fascinating stuff. So yes, So this
is about me, about my passion and for AI and longevity.
I have a few beliefs here. First, that AI can
help us solve our most complex problems. I have no

(12:06):
doubt about this. It's going to happen. I mean, it's happening.
But how it happens there so many ways. But I
do believe that. I believe that the study of AI
is fascinating and it is a study of nature itself
because even humans, right, because we're learning about the world
and we're kind of condensing and storing this, you know,
what we observe in this nature and universe in our

(12:28):
heads and somehow what in the models inside our heads
are actually a reflection of reality right in some ways.
So this is really fascinating. And then the belief that
existence can be beautiful, right, and it's our collective duty
to ensure that it is for everyone in everything that
it is beautiful.

Speaker 4 (12:49):
Right.

Speaker 3 (12:49):
So, and I feel like this probably connects with the
transhumanist beliefs, right, because we can do that. We're strong enough,
we're smart enough too, so and now we have beautiful
tools so we can't complain which is going to do it?

Speaker 2 (13:07):
So okay, so this is about me a bit. But
now let's go back to.

Speaker 3 (13:13):
Talking about what we do at Breakthrough Labs, which is
accelerating hypothesis generation in data analysis, right, and so we'll
begin with hypothesis generation.

Speaker 2 (13:22):
So the main thesis.

Speaker 3 (13:23):
Here on why this would work, Why would AI help
us generate new hypotheses and things that are actually out
of the box. Well, the main thesis here is that
these language large language models are informational golden minds because
they've seen a lot of concepts and connections between them, right,
They've seen so many things on the Internet, and they're

(13:43):
not simply just dumbly storing information. There's actually a lot
of abstractions happening inside, right, a lot of analogies just
based on the architecture itself.

Speaker 2 (13:54):
Right.

Speaker 3 (13:55):
And then they are generative, which means they can generate
new ideas. Now, when we say generative, right, they can
generate all kinds of things, So they're not necessarily brute
forcing all knowledge. They are actually picking things that are plausible.
But they still have their own problems, as you know, hallucinations,

(14:15):
and so we need to kind of put guards against
these hallucinations. And it's still an active area of research
how to mitigate hallucinations. We don't know if we can
actually do it with the current le and ms, but
they're always around it. Okay, so yeah, we still have
to review ideas for now. Okay, so how can we

(14:39):
mind these models to extract insights?

Speaker 4 (14:41):
Right?

Speaker 3 (14:41):
How can we model them to extract new hypotheses? Well,
you can do all kinds of prompting techniques. You can
even employ some searching optimization concepts because you can think
of it as you know, we're searching the internal model
or the internal knowledge of these large language models. To
find you know, golden nuggets or whatever. Now for data analysis,

(15:09):
that rationale is really simple. Well, there's way more data insights.

Speaker 2 (15:13):
Right.

Speaker 3 (15:14):
If you think about the amount of data that exists
in this world, it's insane. I mean, it's not that
existing in this world that is actually stored unused.

Speaker 2 (15:22):
It's insane.

Speaker 3 (15:23):
And there must be ways to kind of extract inside from.
So this is a lot of these data and you know,
and especially in biofarm and stuff like this, it's really expensive.
And so it would be a way just to kind
of have the data sitting. We need to kind of
mind them as much as possible, right, and AI agents
would be ideal for it, because you released an AI agent,

(15:44):
it doesn't get tired, it just keeps going hopefully if
it's functioning properly, would really get you a lot of
good insights. And so we kind of want to get
go from this data. You know a lot of data
more than insights to kind of data to insights, not
a one to more. Obviously, some data won't contain size
what you get the point. Okay, So for us, we

(16:10):
want to apply them in what our applications that we
can do. Well, we are personally interested in these biopharma
obviously a lot of potential. You can save lives, you
can remove pain, minimize suffering, all these nice things that
you can help with fusion. We need clean energy, we

(16:32):
need good energy, and you know, we want to move
away from coal and you know all these gas and
whatnot oil, So fusion would be really nice if and
you know, it's a very active area of research fusion,
so AI could step in and help us generate new material,
new designs even for reactors and stuff, maybe to optimize

(16:54):
certain even AI could be used to kind of decide
how to configure the fusion chambers to maximize you know,
the probability of having a fusion or something. Longevity is
a big thing that I love. Obviously, we all want
to live longer and healthier. And I think, as as
you guys agree, death is horrible and it's tough, and

(17:18):
if we can push it further, that'll be really cool.
Consciousness studies Consciousness is fascinating and it gets really meta
and philosophical, so it's really nice. It's kind of think
about consciousness and how it works and stuff and philosophy
of course, coming up with cool ideas and things like that,
and I'm talking about hypothesis generation with philosophy and consciousness mostly. Okay,

(17:44):
so this is the current focus for our lab. So
spend a lot of time thinking about Okay, so we
have these cool systems.

Speaker 2 (17:52):
Where do you want to go with them? Which kind
of problems you want to solve?

Speaker 3 (17:55):
And I think repurposing drugs is something that actually currently
passionate about for two main reasons. Because it does address inefficiencies. Right,
there's a lot of already approved drugs. They can be
used for different indications, but we don't know maybe you know,
no one spend.

Speaker 2 (18:14):
The time sit down. I think there's a lot of
people doing it.

Speaker 3 (18:17):
But maybe AI could step in and show us connections
or mechanisms of actions or targets that we didn't even
think about. Right, because the AI is seen so many examples,
so many ideas to many thoughts, so many concepts and connections.

Speaker 2 (18:29):
Between them and what now.

Speaker 3 (18:31):
And because it actually saves lives and minimizes suffering, so
especially especially in rare diseases because the monetary incentive for
biofarm is low and rare diseases because the amount of
people that have these diseases are is low, and so
they don't have incentive to spend so much research doing it,
and so if you can repurpose the drug and you know,

(18:53):
and help these people, that would be really awesome.

Speaker 2 (18:55):
You know.

Speaker 3 (18:56):
So we're working on repurposing drugs right now. And I
have to say I'm not an expert in biofarmer. I'm
learning this as I go, but I think that based
on what I'm seeing that there's a lot of potential there.
So what's the progress right now for our product? Well,
we have aristotele is in data and GAUS is under development.

(19:20):
So Aristotle is a hypothesis generation platform to generate novel
solutions to research problems, and I'll show you I can
show you a demo of it at some point. And
you have GAUS, which is the AI atoms AI Data
Analyst agent that analyzes complex biofarma clinical data. Right, so

(19:40):
when you have clinical data, you need to kind of
extract insights from it and stuff you would release GAS
on it, and hopefully GAUS well, you know, you know,
report some nice insights for clinical trial managers and vps
and stuff like this. Okay, so just to show you

(20:01):
kind of quickly what Aristotle looks like, you define a
problem kind of give it, you know, the problem statement
if you want to give it an objective and then
a criteria of what the solutions you expect to be,
what the solutions should be to your problems, or you
know what kind of solutions you're looking for, and then
here's the problem definition, and then you run, you run

(20:23):
Aristotle and you start providing you solutions. For example, this
is a solution to this, this is something about hair loss,
but this is the solution gives you a score, rationale process,
and you can do so much with the solution. You
can interact with it, you can expand on it, you
can talk to the AI about it, you can see
the detailed score on There's a lot you can do.

(20:45):
So I don't want to go into the details. Maybe
if we do a demo, I can show you it
looks like, but it's really cool. I use it myself
and I enjoy it. And we have researchers from Mayo
Clinic using it for a rare disease that we can
talk about little too. Okay, ah, so yeah, just a
famous plug as well to say, if you're interested or

(21:06):
anyone watching and you know, trying aris total for research problem,
feel free to reach out.

Speaker 2 (21:11):
Of course, I'm not charging anything.

Speaker 3 (21:12):
I just want to provide value and see that actually
ours total is bringing some cool hypotheses or solutions to
problems that would make my day. So as for gals,
it's still under development, of course, but if you want
to kind of think about what it does, well, you
would feed it, like, you know, some clinical trial data

(21:36):
for diabetes. It would go out and analyze the data,
go through the whole process, and then it would generate
a list of repurposing opportunities that you can explore if
you want.

Speaker 2 (21:46):
And this helps WI Farma kind of recoop some of
the losses that.

Speaker 3 (21:51):
You know, if the failed clinical trial happens, it costs
like a billion or something around the billion for a film,
So if you can recoup of that, it would be
really helpful. And so what are the challenges now, Well,
there's a lot of challenges, but I just kind of
put a few here. So for hypothesis generation, hallucinations is

(22:15):
a big problem.

Speaker 2 (22:16):
Right.

Speaker 3 (22:16):
So when I first started working with some researchers, the
AI would come up with facts that actually don't even exist,
and you know, the researchers were frustrated. We kept iterating
and you know, building guards against hallucinations, and I think
we have a fairly nice like a stable version now
that doesn't do that.

Speaker 2 (22:36):
But yeah, they're.

Speaker 3 (22:37):
Basically a hallucinaist facts and using them as premises in
other arguments. And then novelty, it's really hard to judge
the novelty of a solution, right, And also novelty is
one of these tricky things like it cannot exist in literature.
But how much are you okay with?

Speaker 2 (22:58):
Right? How much are you okay.

Speaker 3 (23:01):
With maybe someone thought about it in one paper, would
you discover that completely? Well, it would be useful for
another research up to see because they won't otherwise see it.
So novelty is a big problem, but we're working on
it to make sure that the solutions are truly novel
and helpful at the same time. For data analysis, some

(23:21):
of the challenges are, of course, I told you coding,
they still write bugs and then we have to come
in manually and fix them. Well, that doesn't work because
it takes a lot of time from the researchers that
should be doing You're spending their times doing something maybe
higher level, you know, and then memory agents for forget
what they are doing, and then they go into infinite loops.

(23:43):
I know people who build agents probably feel this. It's
a real pain because basically the problem becomes okay, well
what goes into what goes into the prompt?

Speaker 2 (23:53):
Right?

Speaker 3 (23:54):
How do you make sure that the agent doesn't forget
what it was doing, especially for a long complex task, right,
because most of our tasks are complex as humans, but
we can mentally track Oh I was here, I was
doing that.

Speaker 2 (24:05):
I remember this.

Speaker 3 (24:06):
Well, agents can do that if it's out of the context,
game over. So you need to figure out how to
kind of manage the context right. And planning, of course,
agents do not handle long term plans. Well, there are
a lot of hacks that people are doing, but I
think there's still a lot of work to do in planning.

(24:28):
And this is also an active area of research for
US at break through levels. Okay, so this is just
I was just talking about us mainly, and you know
the problems we face and stuff. But since Janet do
you asked me to kind of mention a bit about
the state of AI agents, I thought i'd conclude a

(24:50):
few slides kind of just to show you. So there's
a lot of projects actually, and a lot of academic
projects that are working on AI for science, which is
really fascinating, but it everyone's doing it separately and it's
not being pushed like in a direction where it's actually
becoming more systematic maybe or including industry in it, or

(25:14):
we need a way to kind of make sure that
everyone's pushing towards the same thing in a way, not
just separate individual, you know efforts. So I can tell
you that Google is doing great works. So they have
the core scientists agentic like flow that you can go
and look at it. But basically it's an agentic system.

(25:37):
It's a bunch of AI agents that you give it
a research problem and it does literature review.

Speaker 2 (25:44):
It comes up with hypotheses.

Speaker 3 (25:46):
It even has like a tournament of hypotheses where they
matge against each other and they discuss it the agents
and then they choose one and then the ratter report
blah blah.

Speaker 2 (25:54):
So it's really nice. It's really nice. And actually they
are so this I think they're releasted this year.

Speaker 3 (25:59):
If I'm not mistaken. And what they're doing is they're
working with researchers.

Speaker 2 (26:03):
It's Google.

Speaker 3 (26:04):
They have access to a lot of you know, institutions
and stuff, and they did come up with a repurposing
opportunity that of course they still need to test in
it clinical trials. And actually one cool thing that they
did was they were trying to figure out a mechanism
for antibiotic resistance, right, and so they work with researchers.

(26:25):
The researchers already have good like a good hypothesis that
they tested, a good theory that's called it about.

Speaker 2 (26:35):
Anti sorry antibiotic resistance.

Speaker 3 (26:41):
But then they give the AI that doesn't know about
this because it wasn't published information. They gave the AI
the same problem and the AI ended up on the
same conclusion as these guys. So we know that it
works for something, right, it does work, it's just knowing
how to use it and things like this. So this
was really cool by Google cosse scientists. You can read

(27:04):
on their website. You have Future House too, it's a
nonprofit that's within the platform for AI agents to automate
the scientific discovery. And then the AI too Institute, Allen's
Institute that's one of the founders of co founders that Microsoft.
I think they have an AI for science nonprofit too
that's building stuff. And if you ever use semantic scholar,

(27:26):
if this is familiar. It's like a search engine for
research papers that's by them. They have code scientists. But
I still think that all these systems still fail, you know,
and you will need an intervention, which is a problem.
And there's many others. I'll show you in the next
side how many others are. So you can see here
and these are just from this source. I'll include the

(27:48):
source at some point. But they did like a literature
review of all of them. But you can see in
chemistry and biology, material science, you don't see there's so many,
rightntil we kind of combined efforts into one. It might
be hard to see, you know, what actually works and whatnot.
And most of them, you know, you still have to

(28:09):
kind of go and test them in the lab. So
this is something that we're kind of lacking. Yeah, So
one thing that if you read the papers around agentic
AI for science, they mentioned literature review. And I'm not
sure why literature review is a problem. If I were

(28:30):
to guess, maybe because you know, we're still relying on
systems like retrieval augmented Generation RAG to kind of you know,
manage the knowledge of these AI models. And I do
think that RAG is not enough for us to kind
of you know, do you know do all these memory

(28:50):
storage and retrieval for these agents?

Speaker 1 (28:53):
Right?

Speaker 3 (28:53):
They need a more sophisticated memory and I think we're
going to be seeing this soon hopefully. I think these
are muchlides actually, so I can stop here.

Speaker 2 (29:05):
Maybe if you have a question or anything, please let
you know.

Speaker 1 (29:10):
Yes, thank you very much for that overview. I think
this was a great presentation, and we definitely share the
vision that you have for the future of AI to
help accelerate scientific progress, including of course in the longevity field,
and to improve human well being. I'm going to show

(29:34):
all of us here for the moment I wanted to.
I wanted to ask you what are some of the
ways that you have been able to innovate in in
terms of finding ways around the hallucinations, Because you mentioned
there are approaches that could reduce them or minimize them.

Speaker 3 (29:59):
What are some of those The christis, so before I
talk about what I do, I think hallucinations are kind
of intrinsic to llms because of the way we sample
the responses. Right, when we sample a response, we don't
associate an uncertainty level with it, right, So when you

(30:20):
know when you.

Speaker 2 (30:20):
Say their cat and then it getses jumped.

Speaker 3 (30:23):
Right when we select that, we don't include a metric
of how uncertain the model is. We just say it
is what it is, right, And this is kind of
a problem because you know, this is one problem. The
second problem is that they are probabilistic models and something's
just because something is probably doesn't mean it happens all

(30:47):
the time, right or something like this, like factuality and
probability are not necessarily one hundred percent one to one.

Speaker 2 (30:56):
Right.

Speaker 3 (30:57):
So these are problems that I think they're just intrinsic
to the system. So it's okay, But what we can
do to mitigate that is we can put God right.
So let's say you have an agent that generates ideas, well,
you can have an agent that kind of affects jet
as much as they can. Or maybe go and you know,
do a literature review or yeah, like just go and

(31:18):
find things that you know it could critique the idea
and then you'd have to determine, you know, yes or
no on that. So that's one way to do it
that I think is really easy, and it's likely a
low hanging food.

Speaker 1 (31:30):
Basically, yes, thank you for that answer, And I'm curious also,
you mentioned literature review being one of the main challenges
for AI agents, and I've noticed for just ordinary large
language models, if one can call them, that it's possible

(31:53):
to get hallucinated citations. So that has been a pitfall
for many users who try to, for instance, get some
sort of outline of published research on a subject, and
then there are hallucinated citations. There has even been a
situation where a legal team tried to argue a case
before a court and the AI hallucinated the legal precedents.

(32:18):
So what are some of the ways around that problem
that you're aware of so that you know that your
AI agent is always citing research that actually exists.

Speaker 3 (32:33):
Right, So with the RAG, if you're familiar with drug
or retrival augmented generation, you so it's not very efficient.
In my opinion, it's not the way to go. But
you can include the you know, the link that the
thing came from. But the problem is, you know, with DRAG,
you're basically, let's say you take a paper, you chunk

(32:54):
it into paragraphs, and then you store each paragraph and
then you know, when the user are something, you try
to find the paragraphs that match.

Speaker 2 (33:01):
The query the most.

Speaker 3 (33:03):
The process is inefficient, but It does address the hallucination
part for the most part, because you're literally.

Speaker 2 (33:10):
Including the link in the database.

Speaker 3 (33:12):
So it's not even an AI process, it's like just
a database management process, right, So if the AI pulls
a chunk, it's going to include the link with it.

Speaker 2 (33:20):
Now, is it relevant or not?

Speaker 3 (33:23):
Maybe not, But it does kind of mitigate that reference problem, right,
the reference at least. But the problem with drag and
literature review in general is that, you know, the context
is you know, let's say one hundred thousand tokens, right, so,
but the amount of literature that exists out there is

(33:46):
is so huge, right, and so the question becomes, well,
how do you how does the agent have access to
all this information? And how do you decide what information
to pull for the agent to process at any given time.
So it's really tricky problem because the way we do it,
you know, now they're using like something called.

Speaker 2 (34:05):
A knowledge graph. I don't know if you're familiar with.

Speaker 3 (34:07):
It's more sophisticated tools to kind of like store the
information of all the literature papers out there, but the
agent still has to retrieve them and reason and rationalize
about them. And so that's still a tricky process, right,
so I think that's why literature review is really difficult,
because there's just so much information more than the agent

(34:27):
can fit in there. You know, memory or context will do,
and we have to do the design as humans of
what to bring.

Speaker 2 (34:36):
In at any given time for the agent. Yes, I
don't know if that made sense. I hope it made sense,
but it.

Speaker 1 (34:43):
Does make sense to me now in terms of what
we bring in. My next question is about prompting. So
one aspect of various large language models that I have
encountered is that if you ask them in just a
basic way to multiply two large numbers, they will give

(35:05):
an answer, but the answer will be approximate. It will
be accurate to about three significant figures. And it seems
like every major LLLM does this. But if you then
prompt them to give a precise answer, then they engage
some additional resources to do the precise calculation. So they

(35:26):
have the surface level heuristic answer that they give you
based on a surface level prompt. But if you tell
them to do something that is more accurate, they will
actually engage more resources to give that to you. And
of course, for scientific discovery scientific calculations, you want the

(35:47):
most precise and the most accurate answers possible. So are
there prompting techniques that could get around kind of the
rough and messy first approximations that the all Alams will
sometimes get.

Speaker 2 (36:02):
Yeah, yeah, but just to be precise a bit.

Speaker 3 (36:06):
I think with numbers they probably use code to figure
that out. I don't think they can, Like, maybe they
can do some numbers, but when it gets crazy, I
just don't think because of the way they tokenize inputs
and stuff, I don't know if they can actually do numbers.
But maybe with these I haven't tested it. With like
these new models, you can ask it to think more,
and you know, it starts like doing more, you know,

(36:27):
spending more resources, like you said, to figure that out.
But I think generally when when you ask it to
do numbers, maybe what's happening behind the scenes is that
it's calling a tool which is like, hey, I'm going
to write some code or I'm using a calculative tool
to kind of input these numbers and right run some
you know, some Python code or whatever and bring you
the answer.

Speaker 2 (36:46):
And that's why it's very precise. I don't think they
can do it.

Speaker 3 (36:49):
This is this is besides the point though, I think
what you were saying is that, yes, prompting is I
think when when first the language models came out, I
think there was a lot of research that was I
would say, maybe cooler. And then now now it's like,
since you know, things are more stable, people are trying
to figure out how to use them and stuff.

Speaker 2 (37:09):
But before it was.

Speaker 3 (37:10):
Like, you know, them prompting it, you know, think step
by step, and that's what released you know that, that's
kind of that was the inception of these reasoning models.
After they told the think step by step, it started
like producing the reasoning traces and that improved the answer, right,
And you know, you guys probably know one of them
was like, oh my grandmother, what is it? My grandmother's

(37:31):
gonna die if you don't think this through or something
like this, and it improved.

Speaker 2 (37:36):
I think it's fascinating.

Speaker 3 (37:38):
You know how much we can get from finding the
right prompts. I do think, so, I think, and there's
a lot to uncover, right. I even think it's like
more like psychology too, because it kind of thinks like
a human right, So I feel like whatever can but
maybe this is something we can use the AI to
bridgetone we can ask it about nice prompting techniques that

(37:59):
we can use. Maybe it would give us something surprising, right.

Speaker 1 (38:04):
Yes, yes, indeed, and it seems that learning how to
create good prompts is going to be one of the
major skill sets for success in an economy and in
a research ecosystem that has these AI agents, because we

(38:24):
need to understand how they work from a conceptual standpoint,
even if we are not ourselves building them as users.
It's important for us to have this high level understanding
in order to get them to fulfill our objectives. So
it's a very important subject to consider. Now, Stan Elton,

(38:49):
do you have any questions for RAFFI?

Speaker 5 (38:53):
Yeah, I think that's very interesting. I might likely be
getting into this sort of work myself pretty soon. And
I was wondering if you've considered having teams of agents,
because there's been a couple papers utilizing teams of agents

(39:18):
where each agent has a specific role. Yeah, there was
one that you may have seen where they designed like
covid antibody body so nice.

Speaker 3 (39:33):
Yes, yes, So I think all the experts kind of
agree that having especially and if this is even true
in the economics rights, having specializations kind of help the
whole economy or whatever. So I do agree that having
specialized agents helps, you know, giving each one a role
kind of just helps the agent focus on one thing

(39:55):
and do it really well. Right, So you kind of
limit the tools that it can use, limit the scope
of its work, and it kind of becomes a specialized
agent or specialized expert in that in that thing. So
I definitely agree with you, and I think that's what
Google is doing.

Speaker 2 (40:08):
And all these you know, Harvard is working on this
like all of them.

Speaker 3 (40:13):
But I do think that we need to improve the
agent itself. I think regardless of which agent you pick,
I think agents need. Every agent should have a great memory,
should have great planning skills because these are things that
you will need when you do, you know, any any task,
especially in the science scientific domain. Yeah, but yeah, teams

(40:33):
of agents definitely.

Speaker 6 (40:37):
Yeah.

Speaker 5 (40:37):
The other way, I also wanted to hear more about
how the AI agent will do data analysis, because, you know,
is it essentially because I I've used tools like Cursor, yeah,

(40:59):
which basically allow the AI to look at files, explore
the structure files, and then write code to analyze those files.
So I was wondering if your if your system is
able to do stuff like that, and also kind of
what what are you able to actually you know, are

(41:22):
the agents actually able to do kind of exploratory data analysis?

Speaker 2 (41:29):
Yeah, so it depends.

Speaker 3 (41:30):
I mean, if you just give it an excel sheet,
I think a lot a lot of even I think
open r CHGPT you can upload. You can upload like
an Excel sheet and ask you questions and it will do.

Speaker 2 (41:43):
A nice job. Right, It's okay. But when when you
start talking about like.

Speaker 3 (41:48):
Let's say clinical trials data, you have so much First
the amount of data is and seen, it's different kind
of data, different sources.

Speaker 2 (41:56):
You know.

Speaker 3 (41:57):
Now your agent has to do like has to and
big plans before it can do that. And you have
to remember, like the agent is not going to look
at every data point just like a human does it. Right, So,
if you're a human, any any data scientists could tell you,
like when they look at the data, they do some
exploratory work. Maybe they summarized stuff, maybe they called skin

(42:18):
through some you know, some data just kind of to
get an idea of what it's about. But you won't, like, ok,
at the whole data or something. And so if you
ask me, there's a lot of problems that are going.
That's why you know the GAOS is taking longer than
as total, because there's a lot of technical challenges that

(42:38):
you have to overcome to actually get something useful.

Speaker 2 (42:42):
So the AI agents are not.

Speaker 3 (42:44):
There yet when it comes to really good data analysis, right.

Speaker 5 (42:49):
And I was actually wondering about what your experience has
been with clinical trials data because I've heard that it's
very heterogeneous, like if trials were conducted in multiple sites. Yeah,
it's not necessarily all put into one nice database. There
might be you know, the data might be spread over

(43:10):
many different files in different formats. And what's your experience been, Like,
you know, it's to the extent you've gotten experience with
that actual.

Speaker 3 (43:20):
Yeah, exactly, so I still haven't gotten Like so I'm
still developing toy data setsges because you need to build
the agents. But exactly so, if the way I think
about it is, I think about how a human would
approach it, right, and I think if we can mimic that,
we're kind of successful. Maybe at some point AI agents
will becoup, will be smarter than us, But for now,

(43:42):
like if we want them, we should give them the
things we would do, right, And so you could have
a team of agents. Like Dan said, what you you know,
you have the orchestrator, and then you have a guy
who does just analysis. You have the guy who goes
and grabs, you know, data or finds the data sources.
Like you can do this specialization and I think this
is the way to go, but in the end you

(44:02):
still have to kind of design the system around it. Understanding,
knowing where things are, knowing how to pull things. As
long as you can put it, put all this information
into the context of an agent, I think it does
a decent job into coming up with, you know, a
good next step.

Speaker 2 (44:19):
Or something like this.

Speaker 3 (44:20):
Right, it's just kind of like managing the overall plan
a complex plan at this point, Like it's really complex,
right if you're talking about clinical clouds, data managing writing
code clearly without bugging out, because if you bug and
you know, the agent can't go look at it gets
really messy really quick.

Speaker 2 (44:40):
With the coding.

Speaker 3 (44:41):
So I would say coding just you know, memory, knowing
where things are, knowing what to pull, knowing where to
find things, and planning.

Speaker 2 (44:49):
I think these three things, if.

Speaker 3 (44:51):
You kind of like really come up with high quality solution,
I think we'll have awesome agents in every aspect.

Speaker 5 (44:59):
You know, well, one of the one of the interesting
things about this paper on using the AI agents for
designing the antibody was, you know, I haven't I haven't
gotten to read the whole thing, but basically they they
started with their researcher having a pretty long back and
forth with the sort of main agents in this virtual lab.

(45:22):
And I think what they were doing was really nailing
down the plan back and forth. So instead of putting
in like one prompt and then just letting the AI go,
there was an extended back and forth and the you know,

(45:43):
and I think I think working together, the human and
the AI developed a plan for how they were going
to design these or how they were going to search
through different antibodies. And also so I think that that
might be how you can you know, that might be

(46:05):
better than just prompting if you're you're trying to get
the agent to do like a multi step plan.

Speaker 2 (46:12):
Right, I do agree with you.

Speaker 3 (46:16):
I think I think the next step, actually, like the
next logical steps is to start kind of augmenting and
working human AI collaboration.

Speaker 2 (46:24):
Actually, you get a lot of.

Speaker 3 (46:26):
Training data from this process, right, because you know, it
shows you basically the human preference or the human thoughts.
And if you collect this data, you can even find
you and the agent to become better because you know,
they got feedback from the human right, And yeah, I
definitely agree with you.

Speaker 2 (46:42):
That would be a rational next step for me personally.

Speaker 3 (46:45):
I've got interested in AI in general, and so I
just love thinking about these things, you.

Speaker 2 (46:52):
Know, But I think that you know, I just like
with programmers.

Speaker 3 (46:56):
AI didn't replace programmers yet, right, that people are coding
with like you said, genetic cursor and stuff like this
and so, and it's already like boosting the productivity and
things like this.

Speaker 2 (47:10):
So yeah, even for researchers, this would be the next step, right,
good point.

Speaker 5 (47:15):
Yeah, yeah, And just kind of one other point I
guess I could make is you know, AI and the
LMS now can deal with very heterogene in heterogeneous data sets.
So in the world of like electronic health records, there
have been very big initiatives to try to standardize all

(47:38):
the data and to standardize all the coding systems. Like,
for instance, they have the you know, the way diabetes
if a patient has diabetes, you know, that could be
coded in many different ICD codes. It could be coded
in i CD nine, ICD ten, it could be coded SNOWMED.
There's all these different coding systems, so there's a lot

(48:00):
of different people pushing to kind of get everything standardized
into one system. But actually with l MS now it
can actually work with multiple different systems. And so like
if you say, you know, go into all these records
and find all the patients with Alzheimer's, it can work

(48:20):
with many different types of many different nomenclatures, many different
coding systems, and so the need to standardize is actually
not as great, and so that's, uh, you know, that's
kind of a big advantage.

Speaker 2 (48:35):
I think, yes, it's awesome.

Speaker 5 (48:38):
But I as far as as far as the you know,
I've seen and I've seen like I've seen that actually
you know, work in practice, But as far as like
the data analysis, I've seen it work in a small

(48:59):
scale with with like looking you know, you know, asking
asking its model to do some kind of replicator some results,
But I don't know, I haven't seen it really at
a larger a larger scale yet, Like for instance, if
you have, I'm wondering how you would approach like working

(49:24):
with electronic health records and kind of doing hypothesis generation
and using like a large electronic health records database.

Speaker 3 (49:40):
Right, right, Well, this is actually the challenge, right, it's
this is precisely the challenge that you know, you could
upload an Excel sheet. It's could because it's like a simple,
you know table with a few columns. It can like
attempt to understand it, come up with, you know, certain
numbers that it could you know, somebody statistics whatever and

(50:02):
summary figures and show you. But when it comes to
bigger things, we still we still have to figure that out, right,
It's it.

Speaker 2 (50:11):
Can do it.

Speaker 3 (50:12):
It's just about managing that huge data set and knowing
what to bring to the AI at any given point,
and having the AI kind of have a plan around it.

Speaker 5 (50:24):
Yeah, what you could do is you could have it
go through and look for like statistical associations, right, But
then it also has to kind of do that in
a rigorous way where it's it's rigorously tracking you know,
like the false discovery rate p values and all that. Right,
So it has to be Uh, it seems seems doable,

(50:47):
but you know that it could look for interesting statistical associations.
But but I have not seen anyone pull it off yet.
It would require a lot of a lot of compute.
You know, if it's writing code, that's because if it's

(51:08):
if it's if it's writing code basically to analyze, to
pull down data from a database and then analyze it,
and then I'm doing that hundreds of times to check
for different statistical associations. Like you know, that's gonna be
there's probably going to be a pretty big compute cost too, right,
So absolutely are your thoughts.

Speaker 2 (51:29):
Oh, I absolutely agree with you.

Speaker 3 (51:31):
It's an expensive process, and I agree with you.

Speaker 2 (51:34):
It's a difficult problem and.

Speaker 3 (51:36):
No one has needed it yet, which makes it a
really nice challenge because I mean, I hope we agree
that analyzing these data sets is not important and could
be useful, right, or can be important? Let me say,
and so we need the agents to kind of know
how to analyze data for us. We could just have

(51:56):
humans do it, but if you're released agents, it's it's
probably more efficient to just give going and do that.
The costs are one hundred percent with you, I'm with you.
I think there's a trend towards having small language models
being the agents. You kind of fine tune them a
bit too specific tasks to do them really well, and
then they're small so they're not that expensive you can

(52:18):
run them. But if you're talking about computing on the cloud,
you know running you know, certain operations and stuff. I
agree with you, there's there's some cost, but I feel
like the value we may get justifies the cost right hopefully,
and we can always optimize once we figure out how
it works, we can say, oh, this is a bottleneck,

(52:41):
let's try and reduce it.

Speaker 2 (52:42):
Right.

Speaker 3 (52:43):
That's something I think humans are good at in general.
But my question to you, do you see the value
of like automated data analysis, because I feel like there's
just a.

Speaker 5 (52:52):
Data there, absolutely, because I hear it that, you know,
I was, I'd been working at the nih I'm probably
going to be leaving in about a month to start
at a new job. But I've been at the NIATION.
I've been looking at this data set called all of
US data Set, which is a large collect like biobank,
which includes like electronic health records, genetics data, and surveys.

(53:18):
So it's a massive data set. And if you look
at a lot of the papers that have been published
on this data set, a lot of them are just
looking at a very specific question and then doing some
statistical analyses. For instance, you know, I'm looking right now,
so that there's been hundreds of papers published. Like one
of them I'm looking at right now, just randomly is

(53:41):
looking at an association between low vitamin D and alopecia.
You know, they lent into electronic health records and they
looked at all the patients who had blood tests for
vitamin D and then they correlate that with alopecia. And
there's tons and tons of papers like this. Another one

(54:01):
you know, I saw recently was looking at the wearables
data as well and looking at the correlation between number
of steps and recovery after surgery. There's a lot, a
lot of these different papers I see, Like I thought,

(54:23):
is like a lot of these papers, I think they're relative.
There's a relatively simple template they're following where they're like
defining the cohort, you know, isolating that cohort within the
e h R data, and then looking running a bunch
of statistics. And I've been thinking, you know, like AI
is almost at the point where it could basically do

(54:47):
a lot of these analyzes.

Speaker 2 (54:49):
Right, Yeah, I agree with you. Actually, you know, in.

Speaker 5 (54:52):
This sort of research, there there is sort of a
there is sort of a lot of these papers, like
I just said, they follow a sort of template, right,
A lot of these like observational studies, right, So yeah, yeah,
and and so I'm thinking like in the future, you know,
you you could even you could just have AI, you know,

(55:14):
do the entire study for you, right, And and.

Speaker 2 (55:17):
Exactly, I just like test, you know, test these ideas.

Speaker 3 (55:20):
You know that people because you know, maybe the researcher thought, Okay,
there's a relationship between Batman D and deficiency and alpicia,
but it turned out to be wrong.

Speaker 2 (55:28):
Well you had to run the test, right, right.

Speaker 5 (55:32):
It's like I'm I'm looking at like there's another one here,
association of e cigarette use and psychological distress. You know,
there's all these different papers, and I think, you know,
I think AI should be able to do this. And
with the understanding that it's it is still an observational study,
so you know, there's probably going to be confounding variables.

(55:53):
It's not as not as definitive as an actual rc T.
But these sort of observational STU studies can can be
very I think can be very useful for hypothesis generation
and then you know, finding a signal that can then

(56:14):
potentially motivate you know, more rigorous studies.

Speaker 2 (56:21):
Yeah, yeah, yeah, So I completely agree with that.

Speaker 5 (56:24):
I think it's I think it's amazing and I think
there's a lot of low hanging fruit there and you know,
right yeah yeah, as far as uh, the other topics,
you know, I think I think this is you know,
I don't know, but definitely within EHRs and and observational analysis.

(56:46):
I think a lot of it can be automated even
with today's AI.

Speaker 3 (56:52):
I do want to ask you, actually, so the ni H, like,
are they willing to pay for anass like this?

Speaker 2 (56:59):
Right?

Speaker 3 (56:59):
Because usually with anything, especially for science, you have to
worry about funding and stuff like this.

Speaker 2 (57:05):
So I'm just wondering, if you know anything.

Speaker 3 (57:07):
About it, like are they willing to pay for this
kind of work, you know, automating the analysis or something,
because I know they have a lot of.

Speaker 2 (57:14):
Data, right.

Speaker 5 (57:18):
Well, there's a lot of data in all of all
of us data set. It's I think I think that
we can talk about this offline like a little more,
but they it's sort of the data is all in

(57:44):
the cloud right now, and it's hard to actually link
an LM and get give the LM access to that
data because and researchers are not allowed to download any
of the data. Oh okay, and so the only way
to access the data is through this researcher work bench,
and it's sort of a walled environment where it's hard

(58:05):
to actually get you know, I don't know of any
way to link in l M or yeah. So but
I think so that's one big issue right now. But
as far as whether they're willing to pay, I would
say probably yeah, they would be. They already pay for
a lot from Microsoft, Like recently they they started providing

(58:33):
licenses for Microsoft AI Copilot and co Pilot Studio, which
is mostly for like summarizing documents that you have on
your on your one edrive and stuff like that. But
they're definitely willing to They've definitely been paying for AI

(58:55):
from Microsoft, and I know they pay for a lot
of consultants and contractors to come in to sort of
do like advisement work. So so yeah, I think potentially,
but I'm not sure that. I'm not sure how how

(59:20):
honestly not sure if a lot of NIH researchers are
really aware of really what can be done at the
cutting edge right now. I see, Yeah, probably a bit
of a gap in just knowledge and understanding makes sense.

Speaker 3 (59:33):
Actually, right, you're right if they're catching up to it, right.
But yeah, generally these big companies get you know, the
big contracts, which is kind of sad because you want
to kind of like give small businesses the chance to
you know, do their thing. But I know the government
has programs for small businesses to to be fair, but
generally the big ones get the big fish. But yeah,

(59:55):
thanks for all the in for that's really nice to know.

Speaker 5 (59:58):
Yeah, yeah, yeah, yeah.

Speaker 6 (01:00:04):
I feel like it's just a select number of scientists
right now that are really trying to do this sort
of stuff, but more people are catching on to what's possible.

Speaker 3 (01:00:19):
That's awesome, man, I mean it helps all of us
if everybody's on board, you know.

Speaker 1 (01:00:23):
And I will point out, of course, often innovation does
come from the small businesses or the individual researchers who
are willing to do something unconventional and explore paths that
have not been sufficiently explored. So that is where I
think our Virtual Enlightenment salons bring value because we help

(01:00:45):
to identify some of these innovators and help them get
their message out, as we're doing right now now. Dan,
you shared three articles and two of them you touched on.
So this one is from Scientific Reports. AI design mutation
resistance broad neutralizing antibodies against multiple stars COVI two strains.

(01:01:10):
And here's another one from Nature. The Virtual Lab of
AI agents designs new SARS COVI two nanobodies. And interestingly enough,
I have spoken for about two years about the need
to create a way to defeat multiple COVID variants with

(01:01:30):
the same treatment or the same vaccine, the same antibody,
whatever it may be. And in two thousand and three
there was actually a study that discovered a very promising
antibody called E seven that worked against multiple COVID variants.
But it seems that that study was not then taken

(01:01:50):
further to develop a clinical treatment. But it seems there
are other approaches, other possible antibodies that could be discovered
or synthesized, including through the assistance of AI, and that
I think is going to be necessary to get rid
of this scourge of COVID that continues to affect us.

(01:02:13):
We're in the middle of another COVID search right now.
Even though a lot of people prefer to ignore it,
the fact is that virus is not ignoring us. And
then Dan shared another article, and Dan, perhaps you could
comment on this one further. This was from Stanford Medical School.
It's entitled Researchers Create Virtual Scientists to solve complex biological problems. So, Dan,

(01:02:38):
what else do you want to say on this one?

Speaker 5 (01:02:44):
Yes? Yeah, so that's the that's the AI agents I
was talking about earlier that helped design the covid antibody
or nanobody. If if I could share my screen, actually
we can. We can discuss it a little bit more.

Speaker 2 (01:03:07):
H you let me do what?

Speaker 5 (01:03:09):
No, I'll just.

Speaker 2 (01:03:14):
I think I can just turn on.

Speaker 5 (01:03:15):
So can you guys see this? Yes, yeah you can,
so you can see. They had They had five different
agents principal investigator, immunologist, machine learning specialist, computational biologists, and
scientific critic. And the first this is what I was

(01:03:37):
saying before the first part of it was just going
over the project specification with an extended back and forth dialogue.
You know, I think they actually the human actually talked
to these agents for I think a couple hours, really
nailing down what they wanted to do. Then they a uh.

(01:04:04):
They had these different tools that the agents could use
like UH alpha fold for protein folding UH and studying
the effects and mutations on the folding, and then Rosetta,
which which was just like docking software. And then there

(01:04:26):
were more meetings and then finally, finally the system was
allowed to to run. The entire team of agents was
allowed to run autonomously. And I think they have another

(01:04:46):
figure here. Yeah, it was allowed to run autonomously and
basically design designed uh a bunch of the these antibodies
and then ah, in the end they tested some of

(01:05:09):
them in the lab. So yeah, here you can see like, uh,
some of the tools they were using, like Alpha fold, Rosetta.
It's pretty cool.

Speaker 2 (01:05:28):
It's so cool.

Speaker 3 (01:05:29):
It's so cool, and I just want to comment, like,
the nice part is that if you put the combinations
of you know, different combinations of different agents with different
tools gives you different you know, ideas or different processes
or different approaches. It's like, it's it's crazy the possibilities, right,
and the tools you give it. You know, the choice

(01:05:51):
of tools makes a difference. If you give it access
to alpha food and you tell it this is what
it does, well, the agent imagine the possibilities of what
they It's just in the end, even if they're not
smarter than us, like if they can automate and do things,
you know, just do more things for us that we
just don't have the time, you know, we're that's just

(01:06:11):
I think it's enough on its own at this.

Speaker 2 (01:06:13):
Moment, right.

Speaker 5 (01:06:15):
Yeah. And then the other link I just shared is
to a researcher named daria Yea nut MutS, And he's
been he's been one of the people kind of at
the forefront of all this really pushing AI because I
feel I think a lot of scientists got skeptical of AI.

(01:06:37):
There were there were you know when when you know
people when when chat GPT came out, I think there
was a lot of there was a lot of hype
around using AI for science and a lot of I
think what happened was a lot of scientists tried it
and then they were running into the issue of hallucinations.

(01:06:58):
And then there was also I believe Meta had an
open source model called Galactica or which really was a
big flop, doomed to remember that. Yeah, and they had
this big AI for science model, I think it was
called Galactica, and hallucinating stuff like all over the place.

(01:07:24):
So I think, you know, a lot of scientists had
just become very skeptical based on prior experience, but they
don't realize that the newer generation of models have really
advanced a lot to the point where they're able to
access they're able to check their work, and I feel like,

(01:07:45):
you know, with the deep thinking, the models are much
more careful about what they say, and they often will
if they're not sure about something, they'll do web searches
to kind of check, you know, the literature. And I
feel like that, you know, the issue of lucinations has
really declined a lot to the point where I've I've

(01:08:07):
almost never run into it personally, and uh, I think
maybe if you're getting into very esoteric topics, you know,
it could happen. But generally speaking, the models have gotten
really better, and you know, a lot of the uh,
you know anyway, I think, uh, people just just need

(01:08:33):
to try them out and and and uh, keep keep
experimenting with them, because they are getting better every year.
It's not it's not plateauing really, despite what a lot
of people think. So yeah, De Daria has really been
at the forefront really pushing forward AI for science. And

(01:08:57):
in this in this Twitter post that I shared, he
talks about how GPT five he basically gave a very
large data set to a GPT five which was a
series of metabolites from patients with the chronic fatigue syndrome,
and the model basically replicate a lot of the work

(01:09:23):
that they had done, even finding some associations in the
data that took them months to to discover. So this
was He's got a he's got a couple of posts
on this, so, you know, not it's worth going through

(01:09:49):
his feed and just reading some of his his posts.

Speaker 2 (01:09:54):
Nice.

Speaker 5 (01:09:57):
Yeah, very very interesting.

Speaker 2 (01:10:00):
Yeah, for sure.

Speaker 3 (01:10:01):
I mean I do love there for this. I mean
he's just challenging everyone's like just try stop, you know, here,
here are some results, bringing some receipts.

Speaker 2 (01:10:09):
That's really cool. And I do want to.

Speaker 3 (01:10:10):
Say even open ai had him on on their YouTube
channel one of the videos featuring him.

Speaker 5 (01:10:18):
Yeah, I think he's like I think he's like an
advisor to open ai now or something like that.

Speaker 3 (01:10:23):
So yeah, yep, so he has stakes in it, but
I mean we can't say it's you know, just purely
monetary or something.

Speaker 2 (01:10:30):
Maybe he just believes it.

Speaker 5 (01:10:32):
So actually he says he got an open ai pro grant,
So he got a he got a grant from open
AI two for free chat cipt pro.

Speaker 2 (01:10:48):
That's really cool.

Speaker 5 (01:10:49):
But I also think he might have some other relations
a small so yeah, always aware conflicts of interest, But
but I don't I don't think he's like I think
he's been He's been talking about this stuff long before
he got this, so.

Speaker 3 (01:11:09):
It's been consistent. Yeah, it's always good to Yeah, that's
really good.

Speaker 1 (01:11:14):
Yes, I've seen Dario Nutma's posting on X for a
number of years, and yes, it seems to me his
messaging regarding the promise of AI systems has been consistent. Now,
I will echo what Dan said about the importance of

(01:11:37):
continuing to try out the new AI models and not
just assume that they are stuck at some level of
capabilities that they might have had a few years ago
or even a year ago, because there has been significant
progress since that time. Now, art Ramon, do you have

(01:11:57):
any questions for RAFI?

Speaker 2 (01:12:02):
Not so much.

Speaker 5 (01:12:03):
Questions to some docs.

Speaker 4 (01:12:05):
I remember, very early on Reiitler, before rags became a feature,
I remember looking up parksfecs for seat belt harness in
my Nissan. And normally this automotive data technical data is
behind a service, some sort of pay service that you

(01:12:26):
have to give the technical data, so it wasn't really
out there, so it was generated a lot of sollucinations
on the parkspecs of this seat belt harness quote. And
I think I did some research first, and then finally
RAG started coming out and I told you so on
the internet. Searched for it and it didn't find it
on the internet because someone had posted this proprietary information

(01:12:50):
on a forum that out there. And finally I was
able to give the correct parks FECs for quote. So
I remember when that happened, you know, hallucinations and then
RAG comb out and it's a lot better getting information
off the Internet. And one of the things I've done

(01:13:10):
is used projects the project stature of open AI, and
you know, you tell it to act like a certain
type and I have like this investment AI act like
a professional start plan or whatever, and then give it
certain information and I always use that project for my queries.

Speaker 5 (01:13:30):
You know, it's like, you.

Speaker 4 (01:13:31):
Know, I've got X amount of money to invest, what
should I buy? And it gives me. It's led me
to some very interesting ets that I never even heard
of before. So I've been swing training those and it's
done pretty well.

Speaker 1 (01:13:47):
At that.

Speaker 4 (01:13:48):
Another thing I've done was a while back. I don't
remember exactly what it was, but I did a queried
open eye. It gave me some information, but I wasn't
an expert in that particular field, so I didn't understand
if it was true or false or hallucinations. So I
fed it into Claude. The results, I said, that's true,

(01:14:10):
and it pretty much confirmed that. You know, yeah, everything
looks right. So that's when I first started using two
different ais check each other and that I do that
every now and then now, but tragic, he's got a
lot better.

Speaker 2 (01:14:26):
I don't have to do that as much. So nice yeah, yeah,
even Google. Do you guys use Google?

Speaker 3 (01:14:33):
It's like the AI stuff you know when you're type now,
it gives you that AI feature thing.

Speaker 2 (01:14:38):
It's been really nice to.

Speaker 3 (01:14:39):
It gives links and everything so and it's three two
so and it's really quick, surprisingly so quick though.

Speaker 4 (01:14:48):
Actually, uh yeah, I've mostly used Google a lot. I
used to use the Google feature that it would do
a keyword search for you, and I think it would
like check for it daily.

Speaker 2 (01:14:58):
And I used to.

Speaker 4 (01:14:59):
Use that to track my name or or certain things
I was working on, and that was like an automated process,
but I've done something similar with the newer agentic AI
in open AI. So I wanted to know, you know,
when the Google Hub Max, which is a little device

(01:15:20):
that you put in your kitchen and it has a
little screen and you could answer the front door with
it and do conference and do some basic search, well,
I thought the product wasn't very good, but I mostly
used it for the doorbell answering, and I was like,
you know, I gave the Ajai searched the net on

(01:15:41):
a regular basis for when Google plans on upgrading this
with Jemmykin and it's been checking weekly and finally it
gave me a positive result. They're coming out in October
with you know, they're going to redo the whole operating
system in the in the Hub Max and great Gemini
with it. So it's gonna be a lot more functional

(01:16:02):
than it has for uh when they do that, because
gem Andi. I use it every now and then, but
I pay for open AIS chats and fatigue. I don't
use it as often other than to check sometimes or
I put in the same question into several different AIS
and see who has the better result. I remember one

(01:16:24):
time I put in an acronym of something I've heard
and only Claude was able to figure out this this acronym,
which was interesting not much. Yeah, and yeah, I'm trying
to use the agentic part more. But that else couldn't

(01:16:45):
needed something interesting for it to do. At the times,
I could just do it real time, you know, do
querity road time. But I'm trying to find more things
where I can use the agenic AI to do things.

Speaker 3 (01:16:59):
For Yes, yeah, I mean it's it could be an
overkill honestly for a lot of things. If you want
to spill effect or something and a simple search like
you don't need to worry about this, but you know,
some things do require this agentic quilt. And I don't
know if you guys have seen, but they have these
new browser now AI right companions they want they kind

(01:17:22):
of want to just to tell it what you want
and it would do things in your browser too, So
this might you know, help split things up for yourselfing.

Speaker 5 (01:17:31):
But yeah, yeah, they're saying that. I think I read it.
Someone was saying that computer use agents are going to
be coming out probably in early twenty twenty six. So,
like I said, this whole AI wave is not plateauing.
I mean, you're just going to keep seeing more more

(01:17:57):
agentic capabilities and yeah, more ability to do longer and
longer tasks. There was a study from the the what
was the one of those places in Berkeley, the Meter,

(01:18:28):
I believe it's called MATR and Modeling and Threat Research
Model Evaluation and Threat Research. They have a study where
they looked at the ability of AI to do tasks
of varying lengthening, so like task that would take a
human one minute, task that would take a human thirty minutes,

(01:18:53):
and then they looked at how likely it was to
be able to do each of those different tasks, and
what they found was that the you know, as the
task get longer and longer, the probability that the AI
will be able to complete it and drops off. But
what they also found is with each new iteration of

(01:19:16):
AI model, like the the task that's the longest task
it can do with like a fifty percent success rate
is basically increasing exponentially. So you know, with like chat
GPT two it could do like basically tasks that could

(01:19:36):
take a human like ten seconds, but then when you
got into longer, more complex tasks, it would fail. But
now it's getting to like with GPT five, it's getting
up to where it can do longer and longer tasks. So,
like I said, it's all the signs are showing that
it's the scaling is continuing, we're moving towards using more

(01:19:58):
and more compute, and the development is really not plateauing.
There there's always a lot of news articles coming out
saying AI is plateauing, But when you look at the data,
when you look at the benchmark data, when you look
at the data from metam etr, it's not plateauing. But oh,

(01:20:22):
you know, while we're waiting for the next GPU generation
to come out, the next generation of GPUs, but over
multiple you know, it's it's really just progressing on an
exponential curve basically, like kind of like curs Wild predicted
a little a little bit different, but because we're moving

(01:20:43):
more towards compute rather than larger and larger models. But basically, yeah,
basically an exponential curve.

Speaker 4 (01:20:50):
So that's where I've seen YouTubers say that it's it's
it's going to replace a lot of a positions because
you can have that hi AI do all this incre
level easy stuff. So a lot of level programmers aren't
going to get highly now because that's agentic AI is

(01:21:13):
going to do.

Speaker 5 (01:21:14):
All that work. It's very tough for people that are
just graduating college right now. I read that there there's
a record record unemployment among people who are graduating with
c S majors. If you have some experience in data
science or computer science, it's it's you know, you might
be able to get a job. But it's very tough

(01:21:35):
right now for and yeah, people are going to have
to adapt.

Speaker 3 (01:21:43):
Yeah, But I mean I feel like it's more on
the policy side, people like, how are we going to adapt?
Because eventually AI will replace even blue collar jobs. I mean,
there's a lot of robotic innovations happening. So I feel
like that the conversition should be on the polity politician policymakers, right,
they need to start thinking about this. And I feel

(01:22:04):
like these guys have no idea what's going on. It's
like they don't represent us really, right.

Speaker 1 (01:22:09):
Yes, and this is where the Transhumanist Party comes in,
because we actually have some ideas about how to help
people deal with an environment that is quickly being reshaped
by AI and automation, including, for instance, solutions like a
universal basic income or a decoupling of meaning from what

(01:22:35):
one does for a living, trying to cultivate a more
flexible skill set that will persist even with advances in technology.
Our twenty twenty four presidential candidate Tom Ross made this
a centerpiece of his campaign, essentially helping people to discover
their unique human skill sets that cannot be as readily

(01:22:59):
coded or automated away. And I think this is actually
a good bridge to my question for you. What do
you think will be the human skill sets that will
persist in an environment of widespread AI agents. Let's say
we have AI agents in our browsers. Let's say we
have AI agents that could quickly perform various tasks like

(01:23:25):
hypothesis generation or of data analysis. What do you think
will be the human contribution to interacting with these AI
agents that is not going to go away or become
obsolete anytime soon.

Speaker 3 (01:23:45):
If you're asking me do you know? I would say,
I mean, it's honestly hard to tell. And the answer
is that I don't know, but I can give you,
like hypothesis or whatever you're going to call them. I
would say that emotional part, maybe with the emotional intells,
that emotional connection, maybe the creativity would still be even

(01:24:08):
though I really don't know I'm saying it, but I
really want to think about it. I'm not sure because
I don't know what creativity is right, but I do
agree with you that we should decouple.

Speaker 2 (01:24:19):
You know, existence from what you do.

Speaker 3 (01:24:21):
Like just because you exist, you have you know, you
have the right to a nice life and a decent life,
and you can enjoy it without having to be good
or create at something.

Speaker 2 (01:24:34):
And you know, maybe the meaning is just to explore.

Speaker 3 (01:24:37):
Life and live it and see what's out there and
be curious and you know, maybe SPST trivel hopefully someday
live longer, be with your loved ones. You know, maybe
we can we can provide all these things, but we
have to talk about also if robots become sentient, how
we I'm sentent? Also a bit what conscious if you

(01:24:59):
want to say, if they become conscious, how would we
handle this?

Speaker 2 (01:25:03):
Like it's kind of complicated.

Speaker 1 (01:25:04):
Tool, Yes, absolutely, and we do have these kinds of
conversations as well. In the Transhumanist Party, we formulated the
Transhumanist Bill of Rights Version three point zero, which outlines
a framework of essentially rights and how do we all
get along if there are multiple types of conscious beings.

(01:25:28):
If artificial general intelligence emerges and become sentient, or if
we have even say, uplifted animals, or we encounter extraterrestrial
life forms, how do we have a framework of rights
that allows all of us and humans and augmented humans
to coexist peacefully and to benefit from that intellectual diversity.

(01:25:53):
I think that's very important to consider. And having ais now,
and I think ais that exist now and are going
to exist for a while, are still going to be tools.
But the way they think, the way they generate ideas,
is a bit different from the way the humans think.
And this will be good practice for us to become

(01:26:15):
accustomed to what some have called alien minds, essentially because
in some ways they are already far beyond our current capabilities.
But in other ways, perhaps we can go along certain
paths more easily than they can. And there's a complementarity there,

(01:26:36):
But there needs to be an understanding of both the
capabilities and the limitations.

Speaker 3 (01:26:42):
Great points, great points, And I do love the parties
just progressiveness right, just like looking forward. I think I
do love this and appreciate it, and I think there's
a lot to be done, and actually it's an increasing movement.

Speaker 2 (01:26:56):
Right.

Speaker 3 (01:26:56):
You probably know about Brian Johnson, right, Yes, so, yeah,
Brian Johnson is very bullish on this, and I think
he probably agrees with most of the values, if not
all of them, and so he would be a good
person to kind of help push the agenda forward, right,
And even he was meeting with other celebrities. Now I

(01:27:18):
hate the conditions Cardassians, but he met with them and Mike.
You know, they help if they say something, people listen.
So it's nice to see the movement movement, right, But
there's a lot of other issues we have to worry
about as humans. This is the brutality of a lot
of humans that the way you know, you see these boardes,
these things, like, it's kind of crazy that we have
all the stack and we're still talking about these things.

(01:27:42):
You know, it's kind of depressing a bit, But I
have faith in humanity.

Speaker 1 (01:27:47):
Yes, And actually the stream that we held last week
was a compilation of my presentations from twenty twenty three,
twenty twenty four and existential risk, the idea of the
Great Filter, the problems with all of these wars, and
other kinds of let's say political or geopolitical nonsense that

(01:28:09):
we're experiencing right now. We're discussed and the conclusion that
I came to is it's essentially up to forward thinking
people like us to innovate our way out of this,
to provide a vision for a better world, provide for
tools to help humans get there, to help humans realize

(01:28:32):
that all of these age old destructive ways of functioning
are not serving us, and there are better ways to
approach the future.

Speaker 5 (01:28:43):
Of our species.

Speaker 2 (01:28:45):
Lovely, thank you, thank you.

Speaker 1 (01:28:48):
So now I think it would be a good idea
to see a demo of Aristotle if you have the
ability to share your screen again, I think.

Speaker 3 (01:28:59):
Absolutely yes, and it would be my pleasure actually to
show you guys, get your feedback to let me know
when you can see my screen.

Speaker 2 (01:29:10):
So this is Aristotle, right let me know if you
can see it.

Speaker 1 (01:29:16):
Yes, we can see Aristotle right now.

Speaker 2 (01:29:18):
Perfect.

Speaker 3 (01:29:19):
So just basically here you have the discover mode or feature,
and then the chat if you want to chat with
different ars. You know how you guys were saying like
sometimes like switch between ais, Well you can do this here.
But the discover part, well, you can define a problem
like we say, saw before. And these are some of
the problems I define, right, So let's look at this one.

(01:29:41):
For example, I'm asking the Aristota to brainstorm solutions for
this problem, which is a design for a wireless charger
receiver that efficiently converts received electromagnetic works into electricity.

Speaker 2 (01:29:53):
Basically, I'm saying.

Speaker 3 (01:29:54):
I want you to create like a wireless charger where
I have my phone here, let's say, and it's being
charged from a distance. Now we have the magnetic ones, right,
but these are short range, so I'm talking about long
range using electromagnetic waves. So I give it the objective
and I was like, this is the criteria that I
want you to So every solution you produce must have

(01:30:16):
this criteria.

Speaker 2 (01:30:17):
Right.

Speaker 3 (01:30:18):
If it's not, then I'm probably not going to look
at it, or maybe give it.

Speaker 2 (01:30:22):
A low score or something. And so you can see
some of the things.

Speaker 3 (01:30:25):
It must be compact, lightweight, and suitable for integration, blah
blah blah. And then you know, you tell it, hey,
go ahead and generate like brainstore and you can choose
the model that you want. But if you open so
it generated some solutions. I guess this one generated one solution,
but you can see the solution in generated, right, So
this is the dietle and this is a description of it.

(01:30:48):
This is the score things based on the criteria, how
much it matched the criteria, and then a rationale behind it,
and then a process. Right, And as you guys said,
as the models improved the quality of the thing, the
quality of the solutions improves. And of course let's open
the solution so we can actually look more into things.

Speaker 2 (01:31:08):
So for example, we can read about it.

Speaker 3 (01:31:10):
We can see for every criterion that I said, it
gives the score and a rationale for the score. So
you can see as a researcher, now you're doing things
a bit differently. You're actually like interacting with the AI's
idea and kind of evaluating it.

Speaker 2 (01:31:25):
At the same time.

Speaker 3 (01:31:26):
And then you can go down here and be like, oh, well,
I don't understand what you meant by meta material and
how would meta material do this? So it could be
like what are meta materials? So it's another way to
kind of research things, and then you send it with answer.

Speaker 5 (01:31:39):
You So.

Speaker 2 (01:31:42):
So not just that, you can also do a lot
of things with a solution. You can create a detailed description.

Speaker 3 (01:31:48):
Like a report. If you like it, you're like, I
want more details on it. You can evaluate it but
it already did. You can refine it. Let's say it
looks for knowledge, gaps, issues, inconsistencies, and improves it. You
can expand on it, which means, you know, you can
take the concept and generate basically like variance of it. Right,

(01:32:10):
you can google scholar test too basic. Here is another
thing you can do. You can refine it. But you
can add addition to criteria. And let's say you like it,
but it did something you know, you think, you know
you want to exclude. You can even refine it further.
And so these are some of the things you can do.
So I wanted to show you too that I was

(01:32:31):
working with one of the researchers in Mayo Clinic trying
to brainstorm repurposing opportunities for this disease. This is a
very rare, rare disease, and it's deadly and it's painful too.
So that's the problem with it because what happens is
that whenever you have a trauma or injury in some location,

(01:32:52):
bones grow. Imagine how painful this is, so the person
ends up with bones outside their skeleton. It's it's horrible,
and so they're trying to find repurposing opportunities, and so
we ran the AI and actually it came up with
a lot of them, and based on the feedback, these
solutions are not like the rationals are good. So last

(01:33:14):
thing we checked was that some of them the researcher
heard of. So that's the novelty part, Like you still
need it to be more novel, but terrationals were good,
which is a significant improvement from the previous iterations where
it was hallucinating all kinds of facts. Right, So yeah,
I mean I think this is what our total is.

(01:33:35):
You can define your own problems if you like, and
I wanted to make it more social where you can
actually like comment, so different people can comment have discussions
this way, like it's more of a social network than just.

Speaker 2 (01:33:48):
You know, just you interacting with the AI. But yeah,
basically that's it.

Speaker 3 (01:33:54):
If anyone is interested in using it, just let me
know and I will give you a token where you
can actually like an account for you can actually use it.
But you can see the AI actually gives very decent information.
And I think one of these drugs is actually something
the researcher is going to discuss with their coworkers, so

(01:34:15):
maybe they will take it to clinical trials. That would
make me so happy, you know, if this is the case.

Speaker 1 (01:34:22):
Yes, yeah, absolutely so for those who are watching on YouTube,
if you have any interest in trying out Aristotle, please
make a post about that in the chat and I
will get your information over to RAFI. That way we'll
have more beta testers for this very fascinating system. Now

(01:34:46):
I noticed it seems to be possible to choose an LLM,
like there was a version of Claude available. What are
the options for the llms that can be used?

Speaker 3 (01:34:58):
So I keep adding, this is the choices, These are
the choices that you have. GPT five has been really nice, actually,
especially with brainstorming and hypothesis generation.

Speaker 2 (01:35:09):
I found it to be for some reason. I don't
know what they're doing.

Speaker 3 (01:35:11):
It's really nice. Opus is good. It's pretty expensive though,
but it's really good. So on it is decent. Gemini
with brainstorming has been fifty fifty. It's very critical of
ideas to the point where you know, nothing passes basically,

(01:35:32):
But yeah, I mean you have different and even for
the chat. I use it to do chat, and you know,
I'm very with security. I'm very iffy. So this doesn't
store any information right, So that's kind of cool. I
have to worry about my data. You can download also
the chat. So yeah, it's just I'm trying to give

(01:35:53):
you tools to kind of use AI to you know,
boost your research efforts and to add to this some
of the ideas that I had before about let's say
I'm researching AI. I'm trying to come up with a
way to improve AI, and I asked THEI to brainstorm ideas.
Some of the ideas that I had were actually part
of the brainstorm. So you know, it might take you

(01:36:14):
two months to bristom something and then the AI just
throws it at you. I mean, you feel good and
bad at the same time, right, Yes.

Speaker 1 (01:36:25):
Well, it's really interesting that you can put in your
criteria for what would constitute a good solution, and you
can put in these parameters to guide the AI to
essentially think in the directions that you would consider to
be promising. But yes, I can see how you can

(01:36:46):
put in an idea or the beginnings of an idea
that you have had and have the AI take it further,
expand upon it, provide more support or perhaps slightly different
directions for it, exactly. But yes, this could really augment
human thinking.

Speaker 3 (01:37:06):
One hundred percent. And I can tell you that. So,
for example, this is a fusion thing. I was working
with some of the researchers. They wanted for example, like
you know, they're always trying because with fusion the chambers
get super hard, especially when you're using plasma stuff, and
so they want to come up with innovations on the
wall itself because the wall starts getting damaged, right, so.

Speaker 2 (01:37:25):
What do you do?

Speaker 3 (01:37:26):
And so you can ask so they asked a I
didn't follow with them after but just because you know,
they get busy and stuff. But it did come up
with interesting ideas. Actually that was like, what could this
actually work? You know, it's just interesting.

Speaker 5 (01:37:47):
And these models are mostly just using they're basically just
using all the internal knowledge they have right there. I
was wondering how that works. Is it? Are they mostly
using internal knowledge or are some of these models, like
GPT five doing web searches and that sort of thing.

Speaker 2 (01:38:07):
That's a great question.

Speaker 3 (01:38:08):
But yes, off the shelf, I haven't done any fine tuning,
which is another step that you can do if you
want to. Actually, if you want to go into this,
you can actually find tun it on the domain.

Speaker 5 (01:38:18):
If you're going into a very specific subject, like I'm
actually very interested in the drug repurposing application. We can
we can talk about that later, but definitely you're going
into something very specific like that. I imagine it would be
useful if if the model either does a Google scholar

(01:38:41):
search or Google search to find papers to read, or
if you upload a bunch of papers and then it
can go through and look at you know, specific like
binding affinity values and stuff.

Speaker 3 (01:38:58):
Absolutely, and actually, and actually I want to say, this
is a good use for an agent. So if you
give it the right tools, like you you shared with
us then earlier the alpha fold tool, right, So imagine
giving an agent like the the right tools to go
and in like research deeper into let's say it picked

(01:39:18):
an idea from here as a high score right from
this FOP and yeah, like you said, it could actually
go and start doing the actual research because this is
right now, it's just an idea. Yes, you have a
you have a rationale and maybe it makes sense. You
still have to go in and like prove it, you know,
using whatever you can find out there. So that's that's

(01:39:40):
where using tools would Actually one tool would be like Okay,
a surgeon putment. Another tool would be you know, looking
at whatever medical or like you know, pharmaster tools that
are out there.

Speaker 2 (01:39:54):
Basically, but yeah, you let the AI you figured that out.

Speaker 5 (01:39:58):
Yeah, yeah, that's that that that's basically what you're describing
is is yeah, yeah, the kind of like I think
where where things are going in the future.

Speaker 2 (01:40:11):
So that's hopefully yeah, yeah, honestly.

Speaker 5 (01:40:18):
And uh and then eventually, like I said, we're going
to get computer use where basically you don't even need
an a p I Like right now these you have
to have a well defined a p I. H right.
But but you know a lot of soft older software
and different software doesn't have an API. But with with
computer use, the agent will just be able to you know,

(01:40:40):
use a virtual click, yeah, go through.

Speaker 4 (01:40:44):
I just have a user memory like everything all your
past queries and conversations that's going to have access to
all of that.

Speaker 2 (01:40:54):
Which one, uh, your model, that's one. That's one. Yes,
I mean I actually thought about this.

Speaker 3 (01:41:01):
I feel like this is the product that people are missing,
but it's being added, right, But where do you see
the value exactly of like having.

Speaker 2 (01:41:11):
Like a memory of what you do? Because I'm curious
to know if it should be added.

Speaker 4 (01:41:17):
It's let's say, like with me using a chat tipt,
I could basically say, tell me about myself. It already
knows a lot about myself because of the past conversation,
and it was to make a pretty good summary of
who I am from the past conversation that wasn't president
in the earlier persons. So but but yeah, that could

(01:41:42):
be I would call it that security uh, privacy concert.

Speaker 2 (01:41:47):
Yes, but it's nice.

Speaker 3 (01:41:50):
It's it's really nice right now, no memory, but you
actually bring a good I guess direction for researchers because
I'm building for researchers or right, or people who like
to learn and stuff. Doesn't have to be like, you know,
a researcher at an institute, but what you could do
so it could you could whenever you give it a

(01:42:11):
link or a paper.

Speaker 2 (01:42:12):
Let's say you have a chat and you can add links.
Maybe it learns about every paper it's sol so.

Speaker 3 (01:42:17):
That it kind of have a database of everything you've
fed it about your research. Right, So this I feel
like would be really useful for researchers, some kind of memory.
But it's not about you and your preferences necessarily. It's
more about your research and what you've shown it, because
you know you always have this paper. You're like, oh,
I've read a paper about this, but I can't get

(01:42:38):
to it. Well, how could would it be if this
model could just help you, you know, serve everything you've
seen in chat rept.

Speaker 4 (01:42:45):
You could search through all your past conversations you've had
with it, so that makes it convenient to find. You know,
I've already I've already figured this out in a previous conversation,
but you know, going through which one is a lot
to go through. So I could just do quick search
it It'll come up with all the conversations.

Speaker 2 (01:43:03):
That's nice.

Speaker 4 (01:43:05):
Even without using that search feature, I could ask it, hey,
go through all past conversations and we're buying me or
summarize what we resolved in the past regarding this subject,
and it could sort of do.

Speaker 2 (01:43:20):
So, I mean, it is worth it.

Speaker 3 (01:43:22):
Definitely, Maybe that would be a good project for me
to integrate memory anyway, because I do want to integrate now.
I think memory is a big deal, right, It's the
next innovation in these system So maybe for sure.

Speaker 1 (01:43:34):
Yes, And I would say in terms of memory, having
some sort of retention of prior papers or even prior
hypotheses that we're generated would be quite helpful. I'm reminded
of Google's notebook LM, which is different from other lllms

(01:43:57):
in the sense that it mostly operates on the materials
that you feed to it. But you can have separate
instances of notebook LM which are essentially independent of one another,
and they can accommodate up to fifty sources each, and
those could be research papers, they could be pages from
a website, they could YouTube videos. So you could already

(01:44:18):
get a lot of interesting connections among the sources if
you have up to fifty. But what if you had
a system like that which also remembered past instances of
materials that you fed into it, and over time you
could have hundreds or thousands of sources in there and

(01:44:39):
it would be able to make connections among them. I
could see how for researchers that would be immensely helpful,
especially if they are specialists in a particular discipline, so
they know about a lot of the literature there and
they want to feed that literature into the model. But
maybe the model could discover some additional sources or have

(01:45:01):
some additional sources that would also be helpful and would
interact with the sources for the researcher put.

Speaker 3 (01:45:07):
In absolutely, I mean I definitely agree with this, and
I think these are things that the AI, like these
models shine in because you know, they can they can
make connections between things.

Speaker 2 (01:45:19):
If you tell it.

Speaker 3 (01:45:20):
Sometimes it's messages, sometimes it's wrong, but they can definitely
connect things that I wouldn't say it's hard for a human.
It's just I would say it's not possible because of
how quick they do competitions versus us. Right, they're doing
it on a chip. They have all these huge ditch centers.
You only have this brain, which is fascinating tool, but

(01:45:42):
it's still limited in that sense. They can see a
lot of things that bring.

Speaker 2 (01:45:45):
It to us, right, So that's really cool. Yes.

Speaker 1 (01:45:50):
Absolutely.

Speaker 5 (01:45:52):
Some of these models like Clode are supposed to have
very long context windows, so you can actually use the
context window as a memory. But I think the way
it works actually is that it actually sort of summarizes
some of it and just condenses it down. Because probably yeah,

(01:46:14):
and I think is supposed to have like a million
token context window. Yeah, you just implemented that directly. You
wouldn't it would You wouldn't be able to do it,
so they're using some tricks under the hood sort of.

Speaker 3 (01:46:30):
And there's a lot of research that it deteriorates the
longer the context, the lower the quality of the response.

Speaker 2 (01:46:39):
I think there's a lot of research on it because
it's just hard to you know.

Speaker 3 (01:46:43):
I mean, I respect what they're trying to do and
it helps a lot of users who want to feed
the PDF for something. But there's a lot of research
that's been done that you know, it fails a lot
of tasks, basic tasks.

Speaker 2 (01:46:55):
The longer you know that, the more context feed it.

Speaker 3 (01:47:00):
Basically, Yeah, and I think, yeah, yeah, you need to
be careful about how you know what you feed into
it and you manage that properly and stuff like this.

Speaker 5 (01:47:09):
Yeah, but there's a simple way of imploying memory is
just you know, having a persistent context. But yeah, it's
absolutely I think there in the future there will be
more sophisticated ways of doing memory for sure.

Speaker 3 (01:47:28):
Absolutely. Absolutely. There's a lot of big players invested in it.
So it's just amount of time.

Speaker 1 (01:47:36):
So what do you think it is that leads to
this outcome? It seems counterintuitive, like one would imagine that
the more information one feeds into a model or the
more context that has, the better it would perform. But
there does seem to be some model decay, and in
part it could be let's say, an accumult of errors.

(01:48:01):
If the model hallucinates a little bit in one response
and then it carries that forward and maybe amplifies it
because it doesn't know that it's a hallucination and the
user hasn't corrected it, that could be a form of
model decay. But is there something else? Is there something
more fundamental as to why a larger context could actually

(01:48:21):
lead to poorer model performance.

Speaker 2 (01:48:24):
I would say, actually, that's a very good question.

Speaker 3 (01:48:28):
I would say, just like on top of my head,
maybe it has to do with a transformer architecture, which
is what these models are built on.

Speaker 2 (01:48:37):
Right, So every token gets mixed with every other token basically, right.

Speaker 3 (01:48:43):
So that's and I say mixed because literally it's called
the transformer because these like the representation of each token
is being transformed, right, It keeps getting transformed as it
passes through these layers until the end.

Speaker 2 (01:48:58):
And so imagine having like you know, one word.

Speaker 3 (01:49:01):
Or token, the representation of it or the vector of
it or whatever is going to be affected by one
hundred thousand more other variables are things right now? Of
course there are attention with it's not going to pay
attention to everything, but imagine how many mixing is going
to happen in between.

Speaker 2 (01:49:19):
So maybe they reach a point.

Speaker 3 (01:49:20):
Where everything's just so mixed up, like, oh, every vector
is so transformed to eleven world it can distinguish between things.

Speaker 5 (01:49:28):
This is where I was kind of getting at before,
is the transformer has a context window. Traditionally, you know,
it's something like ten twenty four tokens, and so as
you're generating texts and stuff, it's only able to see
the last like ten twenty four, or it can only
see back as far as its context window. I see,

(01:49:51):
what is the amount of computation scales quadratically with the
size of that, so you know, as ends weird. And
so there are models that have like quote a million
token context text. But if you were to do that
in a traditional way, you wouldn't be able to do it.

(01:50:11):
You wouldn't have enough compute. So they're actually when they
say it's a million token context window, that's actually a
little bit misleading. Yeah, we're doing some sort of magical
tricks under the hood, like context Yes, So what they
must be doing, is they must be distilling, distilling that
down into smaller bits.

Speaker 2 (01:50:34):
That's fairy.

Speaker 5 (01:50:36):
What you're what you were alluding to before is like
if you feed in one hundred page pdf and it
reads all the way through, and then you ask something
about like what was on way back on page one,
it's not to be able to answer that as well
as when if you ask something about like what was
on like the last page it read. I see, I

(01:50:57):
think I think that might be kind of true.

Speaker 2 (01:51:01):
That's a good point. But that's a good point.

Speaker 5 (01:51:03):
The fact is we don't really know exactly how they're
implementing this ultralong context, but.

Speaker 6 (01:51:12):
It's not.

Speaker 5 (01:51:16):
They're distilling it down basically. Yeah.

Speaker 3 (01:51:18):
No, I mean that's what humans do it though, right,
That's how humans do it, right. We kind of read
that passage we extract when we move on extract extract, right,
So got to make sense.

Speaker 5 (01:51:30):
Yeah, yeah, yes.

Speaker 1 (01:51:33):
So, as we are approaching the end of our virtual Enlightenment, Salon,
I have one more question for you, Rafi, and then
I'll invite you to make some concluding remarks. And that is,
given the proliferation that we're seeing of AI agents, including
Aristotle and gal Switch, you're developing, but also the many

(01:51:56):
others that you've mentioned in various disciplines. It would seem
to be fascinating to have AI agents in the future
communicating with one another and interfacing using their distinct architectures,
their distinct ways of looking at scientific problems, just like

(01:52:17):
you could have different human scientists, different specialists from different
disciplines interacting with one another collaborating on research. Where do
you see the possibilities for that? How soon do you
think we're going to have teams of AI agents working
together on scientific discovery?

Speaker 3 (01:52:37):
I mean, great question, And I do want to say
that Google is actually looking into this. And you know,
you know, if you've heard of the MCP protocol, which
is it's just yeah. So basically Anthropic came up with it.
They said, hey, like this is how we think, you know,
you should define tools for the agent. It's a standard
on how to define tools and define agents in general. Well,

(01:52:58):
Google came up and said, okay, well now we need
an agent to agent protocol, right, and this is a
way of you for you how to define your agent
expose it to the world so that if anyone wants
to talk to your agent, they would how to query it,
et cetera, like authentication and stuff like this. So definitely
is something that you know should be happening in twenty

(01:53:20):
twenty six seeing agents communicate with each other, and you're
going to see people publishing their own agents, right, Even
I thought, like you said, Aristotle could be exposed via
an API or an a to agent to agent framework
where someone could query Aristotle and get the solutions right
or GAUS in the future, they could give gaos point
them to the data sets and GUS would do all

(01:53:42):
these fascinating things and maybe their agent is calling, right
the agent would be calling my agent basically, So yeah,
very forward thinking. Do I think you hit a very
good point?

Speaker 1 (01:53:55):
Yes, thank you very much in twenty twenty six, that
is coming very soon. So we are soon to be
immersed in this fascinating world of AI agents communicating with
one another, contributing to scientific research. Let's hope they help
us solve many of these pressing problems, cure some diseases,

(01:54:15):
help us live longer and better. So thank you very much, ROTHI,
and any concluding thoughts for our audience today.

Speaker 3 (01:54:26):
It's been a fascinating chat, guys, really nice and you know,
like I said, I am a big proponent of the party.

Speaker 2 (01:54:33):
I read the values, I think I agree.

Speaker 3 (01:54:36):
It's the progressive movement we need, and you know, we
should stay in touch. And it's been really fun.

Speaker 1 (01:54:42):
Absolutely, and we look forward to seeing how your efforts
progress with Aristotle and soon with Gauss. Let's hope that
they can be deployed to benefit as many people as
possible so that we can all live long and prosper
and prosper.

Speaker 2 (01:55:02):
Absolutely

All Episodes

Episode Transcript

Popular Podcasts

Stuff You Should Know

Dateline NBC

The Joe Rogan Experience

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}Rafi Qumsieh

Episode Transcript

Popular Podcasts

.css-r6mb8g{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;}Stuff You Should Know

Dateline NBC

The Joe Rogan Experience

All Episodes

Rafi Qumsieh

Stuff You Should Know