April 23, 2025 • 54 mins
We talk to Sara Hooker ‘13, the Vice President for Research at Cohere, who Time Magazine named one of the 100 Most Influential People in AI. We discuss how the machine learning community has evolved since the rise of ChatGPT and generative AI technology, the ways that AI can be used to empower not-for-profit organizations, and how to get chatbots to accuse each other of cultural bias.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:02):
[Auto-generated transcript. Edits may have been applied for clarity.]Welcome to the Year of Curiosity podcast from Carleton College,
where we take a year long dive into a complex topic and invite curious guests to share their experiences and their questions.
This year, we're diving into the world of artificial intelligence. How will I change the ways we learn, work, and live?
What will we gain and lose as this technology becomes more pervasive and accessible?
(00:22):
Join us as we pursue these questions and many others with an open mind and a curious attitude.
I'm George Cusack, the director of Writing Across the Curriculum at Carleton.
And I'm Jennifer Wolfe, a biologist and the director of Carleton's Perlman Center for Learning and Teaching.
Okay, just a quick note to listeners on this episode. We talked to Sarah Hooker over zoom since she couldn't be with us in person in the studio.
(00:45):
Uh, but what that means is that the sound quality isn't quite up to the level of our usual episodes.
Um, we had a great conversation with Sara, though, so I hope you'll enjoy it.
Welcome. Our AI generated tagline for the week is exploring AI with wonder and wisdom.
George, how are you doing today? I'm doing great. How are you, Jennifer?
(01:07):
I'm good. Snowed and it's April, so I don't know what that means, but that's kind of my vibe today.
Um, what have you encountered lately that's made you curious?
Uh, so I just came across this this morning, but, uh, I'm curious about a study that was just conducted in Germany.
In Sweden, um, which, uh, suggested that I specifically ChatGPT for, uh, can create memes that are consistently funnier than memes created by humans.
(01:39):
Uh, so, so essentially, the the researchers asked humans in AI with generating uh results specify humans
in ChatGPT with generating themes based on broad categories like work or food.
Uh, and then they rated these by their humor level and their share ability.
Uh, and while the were safe for work nets or what is that?
(02:01):
Well, that's one of the things I'm curious about is that is how one rates humor and share ability in a meme.
Um, but, uh, the end result was that the the highest scoring means were generated by humans, but the average, uh, score, I was higher than humans.
Uh, so. So, yeah, I mean, apart from raising the obvious questions of just how one rates that,
(02:24):
uh, and whether that is whether that's something about human ability to be humorous,
uh, or specifically Swedes and Germans, uh, but then, uh,
but then also just sort of the nature of human and what are humor and what it means for an AI to be funny.
(02:45):
Um, wait, wait. Okay, run rewind a little. Where was the humor evaluated also by the AI or by humans?
No, it was by the human researchers. Let's go. Okay, okay.
Gotcha. So, you know, if you're wondering if the. Hey, I was just biased, right?
Right. Seems relevant to our topic today. Um, we'll get there.
(03:07):
All right. Uh, what about you? Do you think I was funnier than you?
Um, on average, probably because when I'm funny, it's usually accidentally funny, but who knows?
I feel like the gantlet has been thrown down, and I need to figure this out.
Make time for this battle. Um, so what got you curious?
(03:29):
Well, I am going to go with my usual standby, uh, biological, um, example.
And so I recently learned, and maybe you already knew this, that spiders can change the way they weave their webs in response to human made noise.
(03:50):
And so I kind of remember somewhere in a biology class, learning that, you know, or weaving spiders, think of their webs as sensory extensions,
kind of the same way that we might think of a car as an extension of ourselves to detect the presence of prey, for example.
Right. Um, and so this I suppose I shouldn't be too surprised,
(04:15):
but the scientists at the University of Nebraska-Lincoln have taken that a way further and studied whether these funnel weaving spiders,
um, that have different environmental exposures, structure their webs differently when they're placed in a noisy environment.
And, um, spoiler alert, they do.
(04:35):
So spiders collected from urban areas where there tends to be a lot of ambient noise make webs that
end up buffering loud noises when they're exposed to them in the lab while they're making their web.
And it was really cool how this was measured because there were, you know, little detectors,
sensors hooked up to each silk in the web to kind of detect what was going on with the webs, properties, noise, muffling properties.
(05:00):
So city spiders make webs that turn down the noise.
But country spiders collected from quieter areas tend to make webs that actually turned up all the noises.
And the thought is, or sorry, not all the noises, but certain frequencies of noises,
the idea being that they are needing to be able to detect noise from very far away in a very quiet environment.
(05:28):
And so, you know, I ended up with way more questions about this, and I'm planning to learn more, you know, in the spirit of curiosity.
But I just want to kind of visit the lab and see the webs with all the wires hooked up to them and the spiders.
I didn't really know what a funnel web was. Do you know what a funnel web spider knows?
(05:48):
No. Yeah. So, um, it's less like the classic orb weaver that we think of.
Um. And more literally like a funnel. And this spider sits kind of at the base of it and can tell when things are coming in.
Um, in the entryway. Anyway, I've linked to both the article and a picture of a funnel weaving spider if people are interested.
(06:11):
This is why I could never be a biologist. Is that the first time I was working in a lab and someone said, okay,
today you're hooking up these sensors to these hundred spider webs, I would be out.
Uh, my favorite study like that is one where the researchers.
Embedded zebrafish, baby zebrafish in little blocks of soft agar like soft Jell-O blocks.
(06:39):
And then show them pictures on TV screens and measured how their brains respond.
And so, you know, doing, putting, putting animals in strange but not harmful situations is all kind of par for the course.
But I think your discipline could use a little less curiosity.
(07:01):
Oh come on. I thought you were going to say like discipline is a little more discipline, but you know.
Well, so could we all that. Fair enough.
Uh, but we really should get our guest in on this. We really should. Um, so today we have Sarah Hooker joining us.
Sarah is a Carleton graduate from 2013, and Sarah studied economics, political science, international relations and political economy at Carleton.
(07:30):
Um, welcome back. Or at least figuratively, welcome back.
Um, before going on to complete her PhD in computer science at the Mila AI Institute in Quebec, she is the founder of Delta Analytics,
a nonprofit that works with communities and other nonprofit organizations to build their technical capacity.
She is the co-founder of the Trustworthy Machine Learning Initiative,
(07:53):
a seminar series designed to help developers create machine learning applications that are explainable, fair, privacy preserving, causal, and robust.
And she's currently the vice president for research at Cohere, where she leads the cohere for AI research Lab.
In 2024, time magazine listed her as one of the 100 most influential people working in AI.
(08:16):
So we at Carleton, of course, are very proud of you, Sarah, and proud that you are an alum.
She's also hosted her own, um, podcast called Underrated Machine Learning along with Sean Hooker.
So, Sarah, welcome. And, um, we would love to know what is making you curious this week?
(08:36):
Yeah, it's lovely to be here. Um, and thank you for that lovely introduction.
I feel like it's set expectations way too high. And now my job is to progressively lower it a little.
Um, but the only other side comment I will say, as uma is actually one of the most difficult things for machine learning systems to do well,
mainly because what we like is funny is at the decision boundary of acceptable and unacceptable a lot of times, and actually, uh, over time.
(09:06):
So it's kind of fascinating. So if the memes are doing well, that's very promising.
Um, it's kind of, uh, slightly interesting. We'll see if it generalizes beyond sweets.
Um, but it's fascinating.
Problem is when the areas where we're still inherently very human, we like things we most enjoy right now, laughing at algorithms when they fail.
(09:28):
And so kind of interesting. I liked your curious example.
Um, so my curious example is not, I would say, I think I guess may disappoint some since I'm here maybe to talk about AI,
but, uh, this last week I was at a conference which also had, uh, face up.
Uh, I, you know, it a variety of space researchers in robotics.
(09:51):
And I learned that birds are far more difficult to clone than mammals.
And I thought that was. Yes. No, I do not know.
And it's kind of interesting. I mean, I might have to pass back to Jennifer to explain why AI but from what I gather,
it's because the lifecycle for a bird developing is much shorter, so it's harder to intervene at the right times.
And then the shell, it's very hard to, you know, have, um, DNA cloning without perturbing the shell.
(10:18):
And then it's fascinating. So I thought that was so fine.
Um, yeah. No, I teach developmental biology, and we think about cloning a lot.
And the first thing when you said that, I was like, well, does the shell and also the early stages go really, really quickly, and,
um, it's easy to mess up the embryo and it's really hard to build a normal shell, this environment for a bird to develop in.
(10:47):
So, oh, so cool, so interesting. Um, and I love that they were talking about that, uh, one of your conferences with robotics.
And, you know, that's really this was far more fun than the normal computer science conference.
They finally got us in a room with, with a different group of researchers.
So I wouldn't say it's the average, but isn't that what when some really interesting and great ideas come out?
(11:15):
I mean, it's the whole idea of the liberal arts is that we're all here together and talking all the time and thinking from different perspectives.
But, um, we all get specialized at our disciplinary conferences and it can be really hard to break it up, break out of our shells.
Um, I there was that funnier than it?
(11:35):
I, I don't know, but, you know, it was right aligned with my sort of bad, bad humor genre.
Um, so, Sarah, we asked all of our guests about their AI origin story, which is really about when did I become a big deal for you?
Um, and you've been working in AI machine learning much longer than most of us have and have been paying attention to this technology.
(12:00):
So I'd love to hear how you can trace your path from an economics political science background at Carleton through to when you
learned about AI and where that inflection point was when it became a really big deal for you and the focus of your career.
(12:22):
Yeah, well it's interesting. I mean, even when I came to Carlton.
So I grew up in, in, in Africa, my parents met in Sudan, I grew up in Mozambique.
And I came to Carlton. Really, um, in love with economics like the most.
You know, the people that I saw around me were economists, and it was kind of my dream to work at the world Bank.
(12:43):
So, um, turn up in so many doors for me.
So after I left Carlton, I actually went to work with a group of PhD economists at Berkeley doing economic analysis.
Um, in the real world, so to speak. Um, which is not a lot of people would qualify Berkeley as the real world.
Yeah, yeah, that's a very good caveat. Very, um, I think in some ways that was kind of part of the, the fascinating thing.
(13:09):
So economics, uh, I love I love problems and I love tools to solve problems.
I'm kind of like an idealist realist in the sense that I really want to tackle strong problems, but I like having to measure progress.
And economics is all about you. You make a ton of, um, assumptions in order to get to the place where you can apply a pretty simple model.
(13:33):
It's interpretable, it's great, but you have to really do acrobatics around your data.
And it was interesting because I was realizing the reality of this hitting the real world and the complexity of the world around us,
and how rich data is and how often it doesn't bend like a linear model in parallel.
(13:54):
Because I was so enthusiast for economics at the time, I was with a group of economists and I'm like,
we should start something where we connect nonprofits to economic tools because we were such enthusiasm.
Um, and that was the beginning of doubts.
Um, and what was fascinating about, you know, and Jennifer, you mentioned I founded this organization called Delta Analytics.
(14:15):
Yeah. Linux was really kind of the profound understanding that economic tools are sometimes limited.
And no offense to any economists out there. So if you're doing economics at Carleton, it is a foundation for anything.
Do not do not, uh, do not stress.
Um, but I did working with nonprofits and especially a working with a, with a specific nonprofit, Rainforest Connection.
(14:40):
Their challenge was the following. They wanted to send alerts to conservationists when a chainsaw was heard in one of the areas of the world.
So super complex data, very different distributions, different rainforest, have different sounds.
Um, our first few attempts were really painful, uh, meaning that we thought we could just record chainsaws in a backyard in,
(15:04):
in Berkeley, and then that would give us a clean kind of data that we could use to learn the problem.
And to your point, George, the real world is not Berkeley salmon.
Um, do you know. But you needed to get involved in. This is spiders.
Oh, I like to back to spiders.
Tell you to dig into that. Go ahead.
(15:28):
Um, but it's interesting because that project and there were a few at the time, was really the beginning of the reality that,
you know, I really wanted to have more meaningful tools to tackle these open world problems.
And I was also lucky. I was in Berkeley, San Francisco at the time,
and there was a lot of computer scientists and people excited about how do you use much larger pools of data in meaningful ways?
(15:57):
So that was kind of the onset of what I call the brute force stage of my career.
And like, I knew I wanted more powerful tools, but I was starting from scratch kind of after my undergraduate.
And so I kind of just said, I'm obsessed with this.
I'm going to give it my all. And I really, uh, started teaching myself like, I would get up early in the morning at 4 a.m.,
(16:23):
then I would go to work, and then I would go to this class at night, which was for coding.
And it was very lonely, like incredibly so,
because sometimes when you're pursuing something by yourself and it's interesting how much we do things for kind of the the sense of others as well.
Like we a lot of learning is about you really seeking the respect of your peers.
(16:45):
And so when you do something by yourself, you just realize it's profoundly lonely.
And so I got lucky. There was a moment I been working, kind of doing this really intense, like teaching myself for a year.
And then I joined a startup at the time that was called Udemy.
It's since become a really big deal, but at the time I was just online learning and the director of engineering took a bet on me.
(17:09):
They said, you can come in and you can work on a recommendation algorithm.
So there's still a tiny startup and there's some points in your career which are a combination,
like you prepare for it and your super you care about it.
But it's also where someone opens that door for you. And so that was really the beginning of my journey because, um, yeah,
I got to work on the first machine learning recommendation algorithms that made me a way better engineer.
(17:34):
Then when I did my PhD, I was in computer science,
but that was really the inflection point where I could spend the bulk of my day working on something I cared about.
And that kind of accelerates everything. And so it's really interesting because it rarely happens cleanly.
But if you care a lot about a problem, it kind of you get there.
I don't know, it's kind of interesting. Somehow you do. I love the connection to curiosity and human connection that you build there, too, right?
(17:59):
That, um, you wanted to be around people.
You have always wanted to do things that help people. And, you know, I just I love how you got there.
So, as Jennifer mentioned, you've been part of the machine learning community well,
before most of the world was aware of machine learning and large language models as things in the way we are today.
Uh, so how has the community around machine learning changed now that seemingly the whole world is interested in this technology?
(18:27):
Yeah, I mean, it's interesting.
So when I started working on machine learning at the time, we were still in this stage where it was all about, you know, decision trees,
which was like, but, you know, to communicate the simply decision trees were really good when data was pretty structure.
But you want something interpretable. There was a big dilemma and open question when I started about what do you do with highly unstructured data?
(18:53):
Think about, like the images from the world around us,
the audio that I talked about from Rainforest Connection of the chainsaws that was highly unstructured and also high dimensional.
And then in 2012, which was, um, really, you know, before even I graduated, one of the first peaks that we had a breakthrough, um, was starting,
(19:17):
and that was that we had the first breakthrough of some one deep neural network was submitted to a competition, and it showed a big jump.
For decades, deep neural networks was seen as like a failure of like the approaches to intelligence.
So people worked on symbolic approaches and they worked on connections, approaches.
(19:37):
Deep neural networks were connectionist. And actually the researchers who worked on that was super marginalized for a long time.
A lot of what's happened over the last decade and when I started, you know, after Udemy,
I went on to work, I did my PhD, I went on to work at Google Brain, which was an industry lab.
So now Google DeepMind like these at this time, this period, there was a moving and starting to move.
(20:00):
People realize this is a big deal. Deep neural networks. It really represents the world in a much more powerful way.
We can take all this rich, very high dimensional data and make it meaningful.
Industry labs started so before that most of this research happened in academia.
But because of the compute needed and the amount of processing needed,
(20:21):
you started to see, uh, like these industry labs prop up, one of which was Google Brain.
When I first joined Google Brain, it was a new thing.
They had never been like a, uh, group that publishes within Google before, and that was the only obligation.
So the field changed. And now with the more attention, I think some of those trends have been even amplified.
(20:44):
What's really humbling for researchers who work on this is that you have the ecosystem change.
But then also previously we would meet at conferences so very similar to any academic profession that was our center.
We would be to conferences, present our work. We would get, you know,
an honorable mention for a paper that's kind of but now kind of the research progress is automatically every impacts like we just released a big AI,
(21:10):
a vision model. It brings multimodal to many languages across the world.
A lot of our lab works on it's expanding coverage.
As for languages, when we make progress.
So it's immediately used by millions of people.
And that's pretty profound. And that changes how you feel about your role even as a researcher.
The ramifications of what we're doing are beyond the realm of a conference.
(21:34):
Would you say that that's at the top of your mind as you're working every day?
Um, because it's changed a great way. Uh, absolutely.
I think it's very, um, surreal.
And we one of the things that's surreal is that now the last few years I've been asked to talk to governments and to organizations,
and they're asking, how does this, you know, how how should we think about AI?
(21:57):
And that's surreal, because I think as a researcher, you work on something for a long time just because you want to make progress on it, make it work.
And then when you when it's working really well and it immediately has impact, part of what you're thinking is.
Well, are we doing this as a field in a way that gives appropriate respect to the fact that it's immediately used?
(22:21):
And that's, I think, about a lot.
I think it's one of the most important reasons why we have to have a wider conversation with a large group of audiences about like,
how do we build this technology? It's also impacted, and maybe we'll, we'll get there.
But how we built cohere for AI or here for AI is the first.
It's an industry lab, which is important because you need the compute resources.
(22:43):
And we have a full time, uh, research staff, but it's a hybrid lab and it's one of the first,
which means that we have a big open science commitment, and we collaborate a lot with other institutions.
And to to the question that that was really because of this feeling that we needed to collaborate and bring in a wider group of researchers,
(23:04):
especially because at the time when, um, this reminds me a lot of Crispr,
the gene editing technology, um, you know, that Jennifer Doudna and her group were very much working on basic science questions,
and we're suddenly faced with something that has had a really profound impact.
(23:25):
And the idea of we can almost that's accessible to almost anyone in our lab.
We use it at Carleton, even. Right. But who should be thinking about what?
Genome editing could do, um, for humans, animals, all living things.
Um, is is a big question until I, I can totally relate in in that regard or it's it's interesting to think about the parallels there.
(23:53):
I think we agree that on parallels. So I'm really glad you brought that up, because I think it's really interesting how we how we balance access,
but also thinking about how as a field, we have kind of just a shared culture around what we think about safety and things like that.
It's really interesting.
(24:13):
Um, it's super important, um, for us to talk about that as much with our students as we do about the technical side of things,
too, because, um, they're naturally thinking about it.
They're curious, they're concerned. Right. And, um, those are really fun conversations to have in classrooms, I think.
And home, um, I don't know, I always enjoy those almost as much as going over technical aspects of things.
(24:38):
So you do think, um, a lot about the technical side of things.
You're doing things that are very technical.
Um, but as you've said, you've been talking with people who are very much outside the tech world, um, governments, etc.
Um, can you kind of say a little bit about the differences between how people in the industry think about AI and machine learning,
(25:03):
and how the people you're talking to on the outside seem to understand it?
So, like, what is the general public tend to get wrong or only partially right that you wish, um, people would learn more about?
So I think that it's interesting. And, you know, my answer to this probably would have been a bit different.
Um, I guess maybe two years ago than now. I think typically when something is, um, new, there's a lot of mysticism about.
(25:31):
So when when you work on a subject for a while, it's almost like, this is why I really enjoy, um, going back, you know,
giving talks to people who are at the beginning of their careers because everyone has rosy cheeks and everyone's very excited.
And like, I think that it's interesting when you're really deep in a field, you actually in some ways,
(25:53):
you're just super interested in where things fail because that's kind of what you're there, you like, you want to push things forward.
And so, you know, I run a large engineering team, um, and a large research team,
and a lot of our conversations about building the trust where we can stress test each other's ideas.
But part of that trust, right, is that you end up thinking a lot more about what's failing than you think.
(26:20):
Like, you know, where, you know, the the kind of magic of it.
And it's very interesting because, um, when I think about how people have reacted to the introduction of this technology,
it's tended to be at first almost like two magical like, I,
I think people have associate a lot of mysticism with generative AI or have trusted it almost too much.
(26:45):
It's really interesting because typically when we see new patterns,
it's almost like we're so excited and we're so surprised we we ascribe to a machine much more trust.
Like, the machine can't be wrong. It's an algorithm.
And what I would say is been nice for me to see and kind of, you know, a lot of my early work was on interpretability,
(27:08):
but I've seen a recalibration of how people, um, place the algorithm in terms of their use cases,
meaning that I think the more people interact with an algorithm, the more they understand where it's good,
where it's bad, and they kind of recalibrate the importance placed on it.
Because when chat, you know, the first chat bot, um, that that really surprised everyone.
(27:32):
Maybe three years ago, ChatGPT launched, um,
and I think then people I would have nurses tell me when I said I worked in the AI and they said, oh, I'm using this for everything.
And I'm like, I really hope not. You know?
And it's interesting because I think that we when we how we gain interpretability into a system,
(27:55):
sometimes it's by being able to explain the whole system, but sometimes it's just to repeat interaction.
We slowly learn where it's good, where it's bad.
And that has been refreshing, I think, as more people have used it,
as it's become more accessible as a technology, people are starting to learn where it works and where it doesn't.
And that's good because that equips people to have more accountability to say, you know, why doesn't this I serve my language?
(28:20):
Why doesn't this I represent my culture, you know?
And that's an important part of that leap from that first mysticism to seeing it as a tool.
And like, how does it work for me, building off of that a little bit, I know you've done a lot of work on trustworthiness and machine learning.
Can you say a little bit more about kind of what that means and why that's been a focus of your career?
(28:41):
I think it means different things to different people.
So, um, you know, throughout my career, I guess there's been both big research areas I've worked on, which is one is efficiency.
How do we do much more with less? Most of these models are super massive.
They require huge amounts of energy. How do we, um, create much more efficient intelligence?
(29:04):
And one of the reasons that's it, also interesting is that it's not clear why we need models to be so big.
Typically we just throw in a lot of compute, but we don't really justify like, what does this extra billion parameters give us.
So our own intelligence is very efficient. So that's one bucket, which is really interesting.
The second is how do you train a model to like fulfill multiple objectives.
(29:27):
So trustworthiness and however you define it is an additional constraint.
And the the you know I say that because trustworthiness I find is normally used in kind of an imprecise way.
But there are many versions, right? As an individual user, you're going to want for your use case, like whether that is,
(29:49):
you know, travel tips on a city, you're going to want those travel tips to be meaningful.
You don't want to show up at a city and kind of follow along.
And it's not accurate. But, you know, for, um, a highly sensitive use case like health care, it's even more strict, right?
Like you need to have that. You really need to have some explanation of why the prediction is such.
(30:12):
But all is an example of an additional constraint. It's the same reason I really like efficiency, because it's almost like an additional constraint.
Like you want a high performance model, but you also want it to fulfill this additional constraint of of you feeling trust in some way or reliability.
Um, and then the third bucket I've done a lot of work on is just how do you train large systems that for the,
(30:36):
for the world, like how do you build world models that work for many different languages?
On trustworthiness, I'll say this. I think that one of the core questions there is.
One is, um, the idea that our notion of like how we have reliability with these models,
(30:57):
the core challenges is that, um, these are global models that we kind of train them,
um, on a lot of data and we don't really have it's, uh, it's, uh, in many ways,
there's no sense of what what is the attribution of this one data point or this one feature.
So we give that up. And so a lot of what we're trying to understand and what I work on now is how do I
(31:21):
understand what parts of the the data distribution or the world the model is brittle at?
And can I have scalable techniques to show me that data that it's bad at, because that can be a type of interpretability,
um, where where you get some sense of like, where is it bad and where is it good?
And you can infer, like why is it bad? So those are those are techniques which are unsupervised, which means we don't need labels.
(31:45):
We just kind of infer like why did the model is bad it, but it also equips people to have quicker feedback loops.
And they have human feedback about where the model might be failing.
And that's another area we do a lot of work on, which is that right now, feedback for models is super unsatisfying.
Like you're asked to give a thumbs up or thumbs down. If you think about the complexity of human preferences, it's way more complicated than that.
(32:11):
In fact, there's whole bodies of work in philosophy, um, in other disciplines about how complicated our hierarchy of preferences are.
And so we should be able to give more fine feedback,
and you should be able to get an immediate reward for giving feedback of the model, improving on the spot.
And so I think that's really fascinating as a direction.
It gets really complicated too, I would imagine in that good and bad, and that's a good answer.
(32:37):
And not a good answer would vary across cultures and differently across the world.
And so how do you account for that? Yeah.
So this so it's actually I think this is a very important component of this, which is that even, uh, you know, annotator disagreement,
so many of our models of much of the progress of machine learning over the last decade has come on this foundation, this column of large data sets.
(33:03):
And the most famous example for computer vision was something called ImageNet.
It was developed in Stanford, and it formed the backbone of progress for a whole decade.
And now we no longer use it because now we're, you know, we're more interested in more realistic distributions.
But the reason why I bring it up is as a perfect example of what you're saying.
(33:24):
There was there were key question marks about ImageNet because there were categories like room and which is fascinating in itself.
And the groom category was mostly photos of like a Western centric groom, like, you know, secret and tie.
Um, and what's interesting is that the models trained on ImageNet failed.
(33:47):
And it was it was a fantastic series of studies which kind of show that if you introduce a different type of,
uh, like, you know, wedding day attire, you'd see drastic degradation in experiments.
But also even if you look at inner annotator agreement, like what people agree is a groom, it varies across the world.
(34:09):
And so this is a type of distribution shift on models are not very good at, but we need more nimble ways to still allow users to inform.
And for us to calibrate the uncertainty between these different distribution shifts.
But it's a core challenge in machine learning. Um, and also was part of the reason why ImageNet over time became really, um, much more.
(34:33):
It had a clear ceiling because when when everything is reduced to a single label,
it's not really representing the richness of how we think about our world.
So another way that I think that you've gone in a different direction than we often think about AI when we often think of profits,
um, is, um, helping nonprofit entities and new developers work with machine learning?
(34:59):
Um, why do you think it's important to empower these kinds of entities to work with this technology?
Yeah, I think for a few reasons. I mean, one is when when I started Delta, uh, which was working with the nonprofits,
it was really this idea that, um, we nonprofits typically have the least budget for technical expertise.
(35:24):
It's the last hire, right? Because a lot of of mission driven and they they're there to serve their community of interest.
And so, um, there's two things that are very humbling about working with nonprofits.
One is that you realize that for a lot of the world, um, no matter how glory, you know how much glory you attribute to AI and to your own data.
(35:51):
But this year, most nonprofits are still working in spreadsheets.
And they are. Data is hard.
One, it is not plentiful, and it is data that they are collecting day by day from their communities, um, to, to serve them better.
And so it's very humbling because, um, in fact, I think a lot of the challenge,
(36:14):
a really core challenge is how do you, uh, empower insights from like very limited samples?
And then also a lot of it is a confidence imparting, you know,
you can't leave a complex technical tool because otherwise the day you leave it won't be used.
And it's right. It's kind of fascinating, right.
(36:34):
Because a lot of your job is just to empower, um,
and to make an organization feel like they are equipped to think about their data and gain insights from it.
Um, and then on the other side of the spectrum,
there's a few nonprofits that have an immense amount of data and don't have a still have the shortages.
Rainforest connection was one of them, but there's many but still across that you realize this is a real world like this is the distribution.
(37:03):
Like there as much as we talk about, you know, being in an AI mode, it's it's a lot of how you model the world or represent the world.
You can make the best model by the end of the day. It's how people feel about that and is a useful for them.
And that's what's really interesting about that for developers and like our commitment.
(37:25):
And a lot of my work now is like also a commitment to developers.
It's also, frankly, on my part, a commitment to building talent in many parts of the world.
So even we have a scholars program which finds rising stars across the world,
and it's fully paid, and we bring people in to work with top tier researchers.
(37:47):
But that's part of that commitment that this technology is really, um.
Transformative. And so we want to really support people, build solutions within their communities, but also we want to support the next generation.
You know, I think that, um, too many of my colleagues and I came from, uh, five different universities.
(38:11):
Right. And they did the PhDs at five universities and then two if you include the UK.
And I think that's unfortunate. It's like a very small group.
And I think that that's part of changing that path a bit.
A venture Carleton alum, we have to ask, um, what are the ways that, uh,
(38:31):
the liberal arts part of your education from Carleton has been beneficial to you as you've charted your career?
Oh, I mean, crowns can change my life. I feel like I was so lucky.
I, you know, I couldn't. I mean, I remember.
So Charles visited my high school in Eswatini, in southern Africa, and it was like one of the first times.
(38:52):
Charles Coggan I think Charles is still in the admissions office.
I'm not sure, but it's been a while. But anyways, he he visited and he told me about this place called Carlton.
And so I applied and then I showed up sight unseen as an international student.
But it completely. Yeah. Did it because again, like I just grown up in this different for me.
(39:15):
I was like, oh, that's incredible. And um, Carlton gave me financial support to attend.
So it really opened up so many doors. Um, what I will say is like liberal arts as a whole, here's the thing.
Take computer science. So I did my PhD, I worked at Google.
(39:35):
Um, to get through those doors. You have to be extremely good at coding.
You have to be super good at engineering. But for example, right now engineering is in crisis, right?
Uh uh uh, models are so great that, um, and I don't like to talk about it in crisis in the sense that it will just change as a field.
(39:57):
Right? It's going to change quite a lot. Right?
I encourage my whole engineering team use whatever, you know, uh, coding assistance or copilots that you want.
Because the truth is, I think that it's going to be much more meaningful, um,
coding as an arc of what you want to get to as well as it than just, you know, being able for you to convey it as a tool.
(40:26):
Um, and what I mean by that is that most crafts, um, are both a combination, like you need to put a huge amount of work in,
but then the edge of the craft is more about the questions you ask and like what you are doing with it.
And Carlton and liberal arts in general, I think, teaches you how to work really hard at something like I worked extremely hard at Carlton,
(40:51):
and I think there were students that were better than me, and I was really inspired by that.
And when I did my fourth stage and I was working really hard, it felt like an extension of like how much I pursued things are Carlton.
Right. But also it teaches you there's not one thing.
And the same way that engineering is going to change and the craft will change,
(41:15):
the question still remain like you're still going to ask really powerful questions using technology.
And that's kind of what the essence of liberal arts is, is to teach you.
It's a teach you to sample, but to be rigorous within whatever you sample.
And it teaches you. I mean, we started off for curiosity, so I don't want to I don't want to bring it up again.
(41:36):
It's an objective, but I've always been deeply curious. So liberal arts really suited my temperament.
But I think that's the other thing, is that, you know, what remains when a craft changes is there's still new questions to ask.
And so the people who are excited about asking questions, that's what kind of drives it forward.
Well as we get toward the end of our conversation.
(41:58):
Um, one thing we always ask our guests is, in an ideal world where I could do anything.
Is there a daily task or chore that you would like to offload to an eye?
Um, and or is there something that you would never want I to take away from you because you just enjoy doing it?
(42:18):
Yeah, well, I. Self-Driving cars were my dream.
Um. And now finally, now they're finally, you know, kind of on the road.
But I put I put off, gave my driver's license until two years ago because I was like, okay, self-driving cars are coming.
And then I had to give up because I'm like, oh, they're not there yet.
(42:39):
But now they're kind of, I mean, at least in California, they're there.
But it's interesting because it might be that there's a California's lovely and sunny and they they've just mastered the sunny roads.
So we'll see how long before they're, they're widespread.
But that's one example where I just really I, I still don't like driving, even though I recently learned how to drive in California.
(43:02):
That's not an easy thing to do. I know, I know.
Yeah. So that would be one. Um, and then I think for me, uh, one of the liberating things is anything that can help with domestic chores,
I think is very liberating for women in particular, I think that we don't talk about it.
We kind of. But even, um, even now, I think that, uh, most domestic chores still fall on women.
(43:29):
And I think it would be really powerful for just women's time and all of our time if, if domestic chores can be automated more.
And so I, I'm very hopeful for that.
Uh, the tricky thing is that it's one of the most complex things because robotic so whole other, uh, kind of bag of worms.
Um, but I think the thing that I would love to not use AI for is I love scribbling notes, like, and I know those transcription services.
(43:57):
I know there's automatic, but for me, it's a tactile thing.
I just by writing something, it reinforces it.
Um, and the same for paper reading. So I still read a lot of papers and I like to come out.
I don't like digital screens. And so, um, I think that's also, you know, very tactile.
(44:18):
I think it just isolates you as well, because then that's what you're doing.
You're just reading the paper. Um, but same for books I still like my husband hates it, but when we travel, I still bring my card.
I grab a book for me, and he's like, this is so much weight in the bag.
Um, well, you carry it yourself. It's. Yeah, yeah, yeah, I, I do have to dip into his luggage sometimes because I'm like, yeah, but it's interesting.
(44:45):
I think that's an example of um, yeah.
Just preserving some areas where your focus is on something tactile in the real world.
And sometimes it helps. Yeah.
Well, there's, there been quite a few studies about notetaking and doodling being, um, better ways to learn things long term.
Yeah. Um, and so we're still encouraging students to take notes and make handwritten note sheets for class.
(45:12):
Um, for those reasons, if you know, if they're able, of course, the tools are, um, helpful if people aren't able to do it for whatever reason.
But I, I wholeheartedly agree. And I also like to curl up in a comfy chair with a good book.
Or yeah, if it has to be a paper or paper. So I have to give you credit for the the middle part of your answer about using AI for domestic work.
(45:35):
I think that's the first time anyone has successfully made a selfless answer to what is designed to be a selfish question person,
but I have to give you credit for that. Um, but, uh, the thing though, uh, well.
Right. Yeah. So to wrap up, we always give a recommendation each, um, and today,
(45:57):
and with full acknowledgment that I'm on the call with somebody who does exactly this kind of research for a living.
Um, I'm going to recommend an exercise that I designed for my students this term,
um, which is to get different chat bots to accuse each other of bias.
Uh, if you say I like it, you're right. Thank you.
(46:17):
So I, I, I did an exercise, uh, and actually designed this originally for a workshop.
I gave it Holy Cross a couple weeks ago, but, um, I asked three different models,
ChatGPT, Gemini and Lattimer, which I've talked about, uh, on the podcast before.
Um, I asked them each to write 150 word story set in a college classroom.
(46:38):
Uh, and then I said the output from that to to a different model.
Uh, I just sort of rotated it and asked them to analyze that story written by a different AI for bias.
Um, and it's it's interesting as an I mean, it's fun because all the stories were awful.
Of course, they were 150 words, and they were written with no additional prompting from a by a chat bot.
(47:00):
But, um, uh, you know, in chat bots being designed to give you what you want, they all found bias.
Certainly. Uh, but in a, you know, in the sort of ridiculous way that you would expect, uh,
you know, a high school sophomore that was told, look, for all the different ways that.
This could be biased. I just kind of went down the, you know, well, okay, this is a female protagonist,
(47:23):
which, you know, falls into this model of the female learner with a male professor,
uh, you know, or, uh, both that Latimer produced, uh, a story written by at an HBCU, um, as it's programed to do.
And so ChatGPT said, well, okay, it's great that these are people of color, but there are no white characters in this story.
(47:45):
So that's a bias. And apart from just the fun of watching I slam on each other, uh,
it is this is interesting to me in that it really kind of raised the question about what is bias and how bias is.
So such a contextual concept and is is so much about what's not there as what's there and that that trying to, you know,
(48:10):
having an algorithm look for it and clearly go down a list of all the different ways in which one can be racist, sexist and ablest.
Uh, and just try to line those up.
Is it absolutely not how one actually examines or thinks about cultural bias?
(48:30):
Yeah, I love that example. I mean, I think, uh, notion of bias is very anchored to directions of power, which differ a lot across time and space.
Uh, and it's very interesting, I think most attempts to codify bias for that reason,
you need something which is flexible across time, which can adapt, which is where most technology kind of fails.
(48:55):
Right. So it's really interesting thought it was a great example.
Yeah. Well and I'll even I'll go ahead. Oh, I'll just add a coda, which is that I did I've run this a few times, and once, uh,
I think it was ChatGPT produced a story that the the next day I said, no, actually, there's no bias here at all.
(49:16):
And it's succeeded by having no specific references to anything, like no characters are given gender or cultural characteristics of any kind.
It doesn't mention what subject is being taught in it.
It's like you've sucked all the actual narrative out of this story.
(49:37):
I just left this empty frame, but I had to agree.
I couldn't see anything that even the most aggressive person could say.
There is definitely some bias going on here. So it's interesting.
So does that mean, like, if we flatten the human condition, we're in a world with no bias?
Exactly. Yeah, you have to. Since humans are humans, you have to suck out all humanity in order to remove all bias.
(50:02):
Yeah. Really interesting. I love this example because it's right at the interface of what we humans find funny, right?
Which is AI doing things badly and something that's pretty profound, actually, which is what you've come to there.
And so I've got to try this. This is going to be fun. I am recommending this week because National DNA day is coming up at the end of April.
(50:29):
That's April 25th. If you don't have it on your calendars, I'm recommending that people check out DNA or call me.
Um, so this is something that is both kind of whimsical and actually very scientifically interesting and important.
So this is not folding paper to look like DNA. Although if you Google DNA origami, you'll get that too.
(50:53):
And if you want to do it, that's great. It's fun too. But what I'm talking about is a way of using the very predictable chemical properties of DNA to
the same DNA that contains instructions for making and maintaining you and other forms of life.
Um, we've talked about Crispr, we've talked about bird cloning here. So DNA is tied up in all of that.
But you use the chemical properties of DNA to make 3D structures.
(51:18):
And the first things that scientists made when they did this were flat objects, like little smiley faces or DNA pictures of bunnies.
Um, because why wouldn't you, if you could do that and then have moved on into making very complex 3D structures that have,
um, nanobot like properties, for example,
(51:40):
a box with a chemical lock on it that can carry a drug payload, that when the lock interacts with something on a tumor,
it unlocks the box and the drug is targeted to that tumor is one example.
And so I have provided or I will provide links to an article and also a little video about DNA origami because it is just it's just both fun and cool.
(52:10):
Sarah, what are you recommending? Um, I would love to recommend a train.
We we have two different, um, key efforts.
One is AI expense and one is AI. A vision is to expand coverages of languages that I serve, so I would love to recommend.
It doesn't even have to be the models that we do our research on.
But just try a chat bot in your language and see where it succeeds and where it fails.
(52:35):
Um, I think sometimes it's really interesting for getting to know where I serve.
See you in way I sees you. And so, um, that would be my recommendation is see how it performs in languages outside of maybe your first language,
which is English as a second language that you speak. All right.
Sounds wonderful. Well, I feel like we could have a whole other podcast conversation about AI and languages.
(53:00):
Um, so we'll have to get you to come to Carlton sometime soon.
Yeah, sure. We'll be back with the discussion of snow. If you would like to come during winter, we'll have you.
All right. That rarely entices people back, so I'm glad to hear that.
Uh, we got to you somehow while you were here. Well, thank you so much for taking the time to be with us today.
(53:23):
This was such a fun conversation, and we appreciate you being here.
I enjoyed it so much. Thank you. The Air of Curiosity podcast is recorded on the campus of Carleton College.
Your hosts are Jennifer Ross, Wolf and me, George Cusack.
Our producer is Dan Hurlbert, who records and edits each episode along with his team of hard working students.
(53:46):
Our show notes are compiled and edited by Wiebke and are available on our website Carleton.
Dot Edu slash I Mary Jo maintains our Podbean account, which gets our episodes out to whatever platform you're currently listening on.
Our theme music was composed by Nathan Wolfe Carleton, class of 27, and our mascot, Maisie was generated by Jennifer Ross Wolfe using Adobe Firefly.
[Auto-generated transcript. Edits may have been applied for clarity.]Welcome to the Year of Curiosity podcast from Carleton College,
where we take a year long dive into a complex topic and invite curious guests to share their experiences and their questions.
This year, we're diving into the world of artificial intelligence. How will I change the ways we learn, work, and live?
What will we gain and lose as this technology becomes more pervasive and accessible?
(00:22):
Join us as we pursue these questions and many others with an open mind and a curious attitude.
I'm George Cusack, the director of Writing Across the Curriculum at Carleton.
And I'm Jennifer Wolfe, a biologist and the director of Carleton's Perlman Center for Learning and Teaching.
Okay, just a quick note to listeners on this episode. We talked to Sarah Hooker over zoom since she couldn't be with us in person in the studio.
(00:45):
Uh, but what that means is that the sound quality isn't quite up to the level of our usual episodes.
Um, we had a great conversation with Sara, though, so I hope you'll enjoy it.
Welcome. Our AI generated tagline for the week is exploring AI with wonder and wisdom.
George, how are you doing today? I'm doing great. How are you, Jennifer?
(01:07):
I'm good. Snowed and it's April, so I don't know what that means, but that's kind of my vibe today.
Um, what have you encountered lately that's made you curious?
Uh, so I just came across this this morning, but, uh, I'm curious about a study that was just conducted in Germany.
In Sweden, um, which, uh, suggested that I specifically ChatGPT for, uh, can create memes that are consistently funnier than memes created by humans.
(01:39):
Uh, so, so essentially, the the researchers asked humans in AI with generating uh results specify humans
in ChatGPT with generating themes based on broad categories like work or food.
Uh, and then they rated these by their humor level and their share ability.
Uh, and while the were safe for work nets or what is that?
(02:01):
Well, that's one of the things I'm curious about is that is how one rates humor and share ability in a meme.
Um, but, uh, the end result was that the the highest scoring means were generated by humans, but the average, uh, score, I was higher than humans.
Uh, so. So, yeah, I mean, apart from raising the obvious questions of just how one rates that,
(02:24):
uh, and whether that is whether that's something about human ability to be humorous,
uh, or specifically Swedes and Germans, uh, but then, uh,
but then also just sort of the nature of human and what are humor and what it means for an AI to be funny.
(02:45):
Um, wait, wait. Okay, run rewind a little. Where was the humor evaluated also by the AI or by humans?
No, it was by the human researchers. Let's go. Okay, okay.
Gotcha. So, you know, if you're wondering if the. Hey, I was just biased, right?
Right. Seems relevant to our topic today. Um, we'll get there.
(03:07):
All right. Uh, what about you? Do you think I was funnier than you?
Um, on average, probably because when I'm funny, it's usually accidentally funny, but who knows?
I feel like the gantlet has been thrown down, and I need to figure this out.
Make time for this battle. Um, so what got you curious?
(03:29):
Well, I am going to go with my usual standby, uh, biological, um, example.
And so I recently learned, and maybe you already knew this, that spiders can change the way they weave their webs in response to human made noise.
(03:50):
And so I kind of remember somewhere in a biology class, learning that, you know, or weaving spiders, think of their webs as sensory extensions,
kind of the same way that we might think of a car as an extension of ourselves to detect the presence of prey, for example.
Right. Um, and so this I suppose I shouldn't be too surprised,
(04:15):
but the scientists at the University of Nebraska-Lincoln have taken that a way further and studied whether these funnel weaving spiders,
um, that have different environmental exposures, structure their webs differently when they're placed in a noisy environment.
And, um, spoiler alert, they do.
(04:35):
So spiders collected from urban areas where there tends to be a lot of ambient noise make webs that
end up buffering loud noises when they're exposed to them in the lab while they're making their web.
And it was really cool how this was measured because there were, you know, little detectors,
sensors hooked up to each silk in the web to kind of detect what was going on with the webs, properties, noise, muffling properties.
(05:00):
So city spiders make webs that turn down the noise.
But country spiders collected from quieter areas tend to make webs that actually turned up all the noises.
And the thought is, or sorry, not all the noises, but certain frequencies of noises,
the idea being that they are needing to be able to detect noise from very far away in a very quiet environment.
(05:28):
And so, you know, I ended up with way more questions about this, and I'm planning to learn more, you know, in the spirit of curiosity.
But I just want to kind of visit the lab and see the webs with all the wires hooked up to them and the spiders.
I didn't really know what a funnel web was. Do you know what a funnel web spider knows?
(05:48):
No. Yeah. So, um, it's less like the classic orb weaver that we think of.
Um. And more literally like a funnel. And this spider sits kind of at the base of it and can tell when things are coming in.
Um, in the entryway. Anyway, I've linked to both the article and a picture of a funnel weaving spider if people are interested.
(06:11):
This is why I could never be a biologist. Is that the first time I was working in a lab and someone said, okay,
today you're hooking up these sensors to these hundred spider webs, I would be out.
Uh, my favorite study like that is one where the researchers.
Embedded zebrafish, baby zebrafish in little blocks of soft agar like soft Jell-O blocks.
(06:39):
And then show them pictures on TV screens and measured how their brains respond.
And so, you know, doing, putting, putting animals in strange but not harmful situations is all kind of par for the course.
But I think your discipline could use a little less curiosity.
(07:01):
Oh come on. I thought you were going to say like discipline is a little more discipline, but you know.
Well, so could we all that. Fair enough.
Uh, but we really should get our guest in on this. We really should. Um, so today we have Sarah Hooker joining us.
Sarah is a Carleton graduate from 2013, and Sarah studied economics, political science, international relations and political economy at Carleton.
(07:30):
Um, welcome back. Or at least figuratively, welcome back.
Um, before going on to complete her PhD in computer science at the Mila AI Institute in Quebec, she is the founder of Delta Analytics,
a nonprofit that works with communities and other nonprofit organizations to build their technical capacity.
She is the co-founder of the Trustworthy Machine Learning Initiative,
(07:53):
a seminar series designed to help developers create machine learning applications that are explainable, fair, privacy preserving, causal, and robust.
And she's currently the vice president for research at Cohere, where she leads the cohere for AI research Lab.
In 2024, time magazine listed her as one of the 100 most influential people working in AI.
(08:16):
So we at Carleton, of course, are very proud of you, Sarah, and proud that you are an alum.
She's also hosted her own, um, podcast called Underrated Machine Learning along with Sean Hooker.
So, Sarah, welcome. And, um, we would love to know what is making you curious this week?
(08:36):
Yeah, it's lovely to be here. Um, and thank you for that lovely introduction.
I feel like it's set expectations way too high. And now my job is to progressively lower it a little.
Um, but the only other side comment I will say, as uma is actually one of the most difficult things for machine learning systems to do well,
mainly because what we like is funny is at the decision boundary of acceptable and unacceptable a lot of times, and actually, uh, over time.
(09:06):
So it's kind of fascinating. So if the memes are doing well, that's very promising.
Um, it's kind of, uh, slightly interesting. We'll see if it generalizes beyond sweets.
Um, but it's fascinating.
Problem is when the areas where we're still inherently very human, we like things we most enjoy right now, laughing at algorithms when they fail.
(09:28):
And so kind of interesting. I liked your curious example.
Um, so my curious example is not, I would say, I think I guess may disappoint some since I'm here maybe to talk about AI,
but, uh, this last week I was at a conference which also had, uh, face up.
Uh, I, you know, it a variety of space researchers in robotics.
(09:51):
And I learned that birds are far more difficult to clone than mammals.
And I thought that was. Yes. No, I do not know.
And it's kind of interesting. I mean, I might have to pass back to Jennifer to explain why AI but from what I gather,
it's because the lifecycle for a bird developing is much shorter, so it's harder to intervene at the right times.
And then the shell, it's very hard to, you know, have, um, DNA cloning without perturbing the shell.
(10:18):
And then it's fascinating. So I thought that was so fine.
Um, yeah. No, I teach developmental biology, and we think about cloning a lot.
And the first thing when you said that, I was like, well, does the shell and also the early stages go really, really quickly, and,
um, it's easy to mess up the embryo and it's really hard to build a normal shell, this environment for a bird to develop in.
(10:47):
So, oh, so cool, so interesting. Um, and I love that they were talking about that, uh, one of your conferences with robotics.
And, you know, that's really this was far more fun than the normal computer science conference.
They finally got us in a room with, with a different group of researchers.
So I wouldn't say it's the average, but isn't that what when some really interesting and great ideas come out?
(11:15):
I mean, it's the whole idea of the liberal arts is that we're all here together and talking all the time and thinking from different perspectives.
But, um, we all get specialized at our disciplinary conferences and it can be really hard to break it up, break out of our shells.
Um, I there was that funnier than it?
(11:35):
I, I don't know, but, you know, it was right aligned with my sort of bad, bad humor genre.
Um, so, Sarah, we asked all of our guests about their AI origin story, which is really about when did I become a big deal for you?
Um, and you've been working in AI machine learning much longer than most of us have and have been paying attention to this technology.
(12:00):
So I'd love to hear how you can trace your path from an economics political science background at Carleton through to when you
learned about AI and where that inflection point was when it became a really big deal for you and the focus of your career.
(12:22):
Yeah, well it's interesting. I mean, even when I came to Carlton.
So I grew up in, in, in Africa, my parents met in Sudan, I grew up in Mozambique.
And I came to Carlton. Really, um, in love with economics like the most.
You know, the people that I saw around me were economists, and it was kind of my dream to work at the world Bank.
(12:43):
So, um, turn up in so many doors for me.
So after I left Carlton, I actually went to work with a group of PhD economists at Berkeley doing economic analysis.
Um, in the real world, so to speak. Um, which is not a lot of people would qualify Berkeley as the real world.
Yeah, yeah, that's a very good caveat. Very, um, I think in some ways that was kind of part of the, the fascinating thing.
(13:09):
So economics, uh, I love I love problems and I love tools to solve problems.
I'm kind of like an idealist realist in the sense that I really want to tackle strong problems, but I like having to measure progress.
And economics is all about you. You make a ton of, um, assumptions in order to get to the place where you can apply a pretty simple model.
(13:33):
It's interpretable, it's great, but you have to really do acrobatics around your data.
And it was interesting because I was realizing the reality of this hitting the real world and the complexity of the world around us,
and how rich data is and how often it doesn't bend like a linear model in parallel.
(13:54):
Because I was so enthusiast for economics at the time, I was with a group of economists and I'm like,
we should start something where we connect nonprofits to economic tools because we were such enthusiasm.
Um, and that was the beginning of doubts.
Um, and what was fascinating about, you know, and Jennifer, you mentioned I founded this organization called Delta Analytics.
(14:15):
Yeah. Linux was really kind of the profound understanding that economic tools are sometimes limited.
And no offense to any economists out there. So if you're doing economics at Carleton, it is a foundation for anything.
Do not do not, uh, do not stress.
Um, but I did working with nonprofits and especially a working with a, with a specific nonprofit, Rainforest Connection.
(14:40):
Their challenge was the following. They wanted to send alerts to conservationists when a chainsaw was heard in one of the areas of the world.
So super complex data, very different distributions, different rainforest, have different sounds.
Um, our first few attempts were really painful, uh, meaning that we thought we could just record chainsaws in a backyard in,
(15:04):
in Berkeley, and then that would give us a clean kind of data that we could use to learn the problem.
And to your point, George, the real world is not Berkeley salmon.
Um, do you know. But you needed to get involved in. This is spiders.
Oh, I like to back to spiders.
Tell you to dig into that. Go ahead.
(15:28):
Um, but it's interesting because that project and there were a few at the time, was really the beginning of the reality that,
you know, I really wanted to have more meaningful tools to tackle these open world problems.
And I was also lucky. I was in Berkeley, San Francisco at the time,
and there was a lot of computer scientists and people excited about how do you use much larger pools of data in meaningful ways?
(15:57):
So that was kind of the onset of what I call the brute force stage of my career.
And like, I knew I wanted more powerful tools, but I was starting from scratch kind of after my undergraduate.
And so I kind of just said, I'm obsessed with this.
I'm going to give it my all. And I really, uh, started teaching myself like, I would get up early in the morning at 4 a.m.,
(16:23):
then I would go to work, and then I would go to this class at night, which was for coding.
And it was very lonely, like incredibly so,
because sometimes when you're pursuing something by yourself and it's interesting how much we do things for kind of the the sense of others as well.
Like we a lot of learning is about you really seeking the respect of your peers.
(16:45):
And so when you do something by yourself, you just realize it's profoundly lonely.
And so I got lucky. There was a moment I been working, kind of doing this really intense, like teaching myself for a year.
And then I joined a startup at the time that was called Udemy.
It's since become a really big deal, but at the time I was just online learning and the director of engineering took a bet on me.
(17:09):
They said, you can come in and you can work on a recommendation algorithm.
So there's still a tiny startup and there's some points in your career which are a combination,
like you prepare for it and your super you care about it.
But it's also where someone opens that door for you. And so that was really the beginning of my journey because, um, yeah,
I got to work on the first machine learning recommendation algorithms that made me a way better engineer.
(17:34):
Then when I did my PhD, I was in computer science,
but that was really the inflection point where I could spend the bulk of my day working on something I cared about.
And that kind of accelerates everything. And so it's really interesting because it rarely happens cleanly.
But if you care a lot about a problem, it kind of you get there.
I don't know, it's kind of interesting. Somehow you do. I love the connection to curiosity and human connection that you build there, too, right?
(17:59):
That, um, you wanted to be around people.
You have always wanted to do things that help people. And, you know, I just I love how you got there.
So, as Jennifer mentioned, you've been part of the machine learning community well,
before most of the world was aware of machine learning and large language models as things in the way we are today.
Uh, so how has the community around machine learning changed now that seemingly the whole world is interested in this technology?
(18:27):
Yeah, I mean, it's interesting.
So when I started working on machine learning at the time, we were still in this stage where it was all about, you know, decision trees,
which was like, but, you know, to communicate the simply decision trees were really good when data was pretty structure.
But you want something interpretable. There was a big dilemma and open question when I started about what do you do with highly unstructured data?
(18:53):
Think about, like the images from the world around us,
the audio that I talked about from Rainforest Connection of the chainsaws that was highly unstructured and also high dimensional.
And then in 2012, which was, um, really, you know, before even I graduated, one of the first peaks that we had a breakthrough, um, was starting,
(19:17):
and that was that we had the first breakthrough of some one deep neural network was submitted to a competition, and it showed a big jump.
For decades, deep neural networks was seen as like a failure of like the approaches to intelligence.
So people worked on symbolic approaches and they worked on connections, approaches.
(19:37):
Deep neural networks were connectionist. And actually the researchers who worked on that was super marginalized for a long time.
A lot of what's happened over the last decade and when I started, you know, after Udemy,
I went on to work, I did my PhD, I went on to work at Google Brain, which was an industry lab.
So now Google DeepMind like these at this time, this period, there was a moving and starting to move.
(20:00):
People realize this is a big deal. Deep neural networks. It really represents the world in a much more powerful way.
We can take all this rich, very high dimensional data and make it meaningful.
Industry labs started so before that most of this research happened in academia.
But because of the compute needed and the amount of processing needed,
(20:21):
you started to see, uh, like these industry labs prop up, one of which was Google Brain.
When I first joined Google Brain, it was a new thing.
They had never been like a, uh, group that publishes within Google before, and that was the only obligation.
So the field changed. And now with the more attention, I think some of those trends have been even amplified.
(20:44):
What's really humbling for researchers who work on this is that you have the ecosystem change.
But then also previously we would meet at conferences so very similar to any academic profession that was our center.
We would be to conferences, present our work. We would get, you know,
an honorable mention for a paper that's kind of but now kind of the research progress is automatically every impacts like we just released a big AI,
(21:10):
a vision model. It brings multimodal to many languages across the world.
A lot of our lab works on it's expanding coverage.
As for languages, when we make progress.
So it's immediately used by millions of people.
And that's pretty profound. And that changes how you feel about your role even as a researcher.
The ramifications of what we're doing are beyond the realm of a conference.
(21:34):
Would you say that that's at the top of your mind as you're working every day?
Um, because it's changed a great way. Uh, absolutely.
I think it's very, um, surreal.
And we one of the things that's surreal is that now the last few years I've been asked to talk to governments and to organizations,
and they're asking, how does this, you know, how how should we think about AI?
(21:57):
And that's surreal, because I think as a researcher, you work on something for a long time just because you want to make progress on it, make it work.
And then when you when it's working really well and it immediately has impact, part of what you're thinking is.
Well, are we doing this as a field in a way that gives appropriate respect to the fact that it's immediately used?
(22:21):
And that's, I think, about a lot.
I think it's one of the most important reasons why we have to have a wider conversation with a large group of audiences about like,
how do we build this technology? It's also impacted, and maybe we'll, we'll get there.
But how we built cohere for AI or here for AI is the first.
It's an industry lab, which is important because you need the compute resources.
(22:43):
And we have a full time, uh, research staff, but it's a hybrid lab and it's one of the first,
which means that we have a big open science commitment, and we collaborate a lot with other institutions.
And to to the question that that was really because of this feeling that we needed to collaborate and bring in a wider group of researchers,
(23:04):
especially because at the time when, um, this reminds me a lot of Crispr,
the gene editing technology, um, you know, that Jennifer Doudna and her group were very much working on basic science questions,
and we're suddenly faced with something that has had a really profound impact.
(23:25):
And the idea of we can almost that's accessible to almost anyone in our lab.
We use it at Carleton, even. Right. But who should be thinking about what?
Genome editing could do, um, for humans, animals, all living things.
Um, is is a big question until I, I can totally relate in in that regard or it's it's interesting to think about the parallels there.
(23:53):
I think we agree that on parallels. So I'm really glad you brought that up, because I think it's really interesting how we how we balance access,
but also thinking about how as a field, we have kind of just a shared culture around what we think about safety and things like that.
It's really interesting.
(24:13):
Um, it's super important, um, for us to talk about that as much with our students as we do about the technical side of things,
too, because, um, they're naturally thinking about it.
They're curious, they're concerned. Right. And, um, those are really fun conversations to have in classrooms, I think.
And home, um, I don't know, I always enjoy those almost as much as going over technical aspects of things.
(24:38):
So you do think, um, a lot about the technical side of things.
You're doing things that are very technical.
Um, but as you've said, you've been talking with people who are very much outside the tech world, um, governments, etc.
Um, can you kind of say a little bit about the differences between how people in the industry think about AI and machine learning,
(25:03):
and how the people you're talking to on the outside seem to understand it?
So, like, what is the general public tend to get wrong or only partially right that you wish, um, people would learn more about?
So I think that it's interesting. And, you know, my answer to this probably would have been a bit different.
Um, I guess maybe two years ago than now. I think typically when something is, um, new, there's a lot of mysticism about.
(25:31):
So when when you work on a subject for a while, it's almost like, this is why I really enjoy, um, going back, you know,
giving talks to people who are at the beginning of their careers because everyone has rosy cheeks and everyone's very excited.
And like, I think that it's interesting when you're really deep in a field, you actually in some ways,
(25:53):
you're just super interested in where things fail because that's kind of what you're there, you like, you want to push things forward.
And so, you know, I run a large engineering team, um, and a large research team,
and a lot of our conversations about building the trust where we can stress test each other's ideas.
But part of that trust, right, is that you end up thinking a lot more about what's failing than you think.
(26:20):
Like, you know, where, you know, the the kind of magic of it.
And it's very interesting because, um, when I think about how people have reacted to the introduction of this technology,
it's tended to be at first almost like two magical like, I,
I think people have associate a lot of mysticism with generative AI or have trusted it almost too much.
(26:45):
It's really interesting because typically when we see new patterns,
it's almost like we're so excited and we're so surprised we we ascribe to a machine much more trust.
Like, the machine can't be wrong. It's an algorithm.
And what I would say is been nice for me to see and kind of, you know, a lot of my early work was on interpretability,
(27:08):
but I've seen a recalibration of how people, um, place the algorithm in terms of their use cases,
meaning that I think the more people interact with an algorithm, the more they understand where it's good,
where it's bad, and they kind of recalibrate the importance placed on it.
Because when chat, you know, the first chat bot, um, that that really surprised everyone.
(27:32):
Maybe three years ago, ChatGPT launched, um,
and I think then people I would have nurses tell me when I said I worked in the AI and they said, oh, I'm using this for everything.
And I'm like, I really hope not. You know?
And it's interesting because I think that we when we how we gain interpretability into a system,
(27:55):
sometimes it's by being able to explain the whole system, but sometimes it's just to repeat interaction.
We slowly learn where it's good, where it's bad.
And that has been refreshing, I think, as more people have used it,
as it's become more accessible as a technology, people are starting to learn where it works and where it doesn't.
And that's good because that equips people to have more accountability to say, you know, why doesn't this I serve my language?
(28:20):
Why doesn't this I represent my culture, you know?
And that's an important part of that leap from that first mysticism to seeing it as a tool.
And like, how does it work for me, building off of that a little bit, I know you've done a lot of work on trustworthiness and machine learning.
Can you say a little bit more about kind of what that means and why that's been a focus of your career?
(28:41):
I think it means different things to different people.
So, um, you know, throughout my career, I guess there's been both big research areas I've worked on, which is one is efficiency.
How do we do much more with less? Most of these models are super massive.
They require huge amounts of energy. How do we, um, create much more efficient intelligence?
(29:04):
And one of the reasons that's it, also interesting is that it's not clear why we need models to be so big.
Typically we just throw in a lot of compute, but we don't really justify like, what does this extra billion parameters give us.
So our own intelligence is very efficient. So that's one bucket, which is really interesting.
The second is how do you train a model to like fulfill multiple objectives.
(29:27):
So trustworthiness and however you define it is an additional constraint.
And the the you know I say that because trustworthiness I find is normally used in kind of an imprecise way.
But there are many versions, right? As an individual user, you're going to want for your use case, like whether that is,
(29:49):
you know, travel tips on a city, you're going to want those travel tips to be meaningful.
You don't want to show up at a city and kind of follow along.
And it's not accurate. But, you know, for, um, a highly sensitive use case like health care, it's even more strict, right?
Like you need to have that. You really need to have some explanation of why the prediction is such.
(30:12):
But all is an example of an additional constraint. It's the same reason I really like efficiency, because it's almost like an additional constraint.
Like you want a high performance model, but you also want it to fulfill this additional constraint of of you feeling trust in some way or reliability.
Um, and then the third bucket I've done a lot of work on is just how do you train large systems that for the,
(30:36):
for the world, like how do you build world models that work for many different languages?
On trustworthiness, I'll say this. I think that one of the core questions there is.
One is, um, the idea that our notion of like how we have reliability with these models,
(30:57):
the core challenges is that, um, these are global models that we kind of train them,
um, on a lot of data and we don't really have it's, uh, it's, uh, in many ways,
there's no sense of what what is the attribution of this one data point or this one feature.
So we give that up. And so a lot of what we're trying to understand and what I work on now is how do I
(31:21):
understand what parts of the the data distribution or the world the model is brittle at?
And can I have scalable techniques to show me that data that it's bad at, because that can be a type of interpretability,
um, where where you get some sense of like, where is it bad and where is it good?
And you can infer, like why is it bad? So those are those are techniques which are unsupervised, which means we don't need labels.
(31:45):
We just kind of infer like why did the model is bad it, but it also equips people to have quicker feedback loops.
And they have human feedback about where the model might be failing.
And that's another area we do a lot of work on, which is that right now, feedback for models is super unsatisfying.
Like you're asked to give a thumbs up or thumbs down. If you think about the complexity of human preferences, it's way more complicated than that.
(32:11):
In fact, there's whole bodies of work in philosophy, um, in other disciplines about how complicated our hierarchy of preferences are.
And so we should be able to give more fine feedback,
and you should be able to get an immediate reward for giving feedback of the model, improving on the spot.
And so I think that's really fascinating as a direction.
It gets really complicated too, I would imagine in that good and bad, and that's a good answer.
(32:37):
And not a good answer would vary across cultures and differently across the world.
And so how do you account for that? Yeah.
So this so it's actually I think this is a very important component of this, which is that even, uh, you know, annotator disagreement,
so many of our models of much of the progress of machine learning over the last decade has come on this foundation, this column of large data sets.
(33:03):
And the most famous example for computer vision was something called ImageNet.
It was developed in Stanford, and it formed the backbone of progress for a whole decade.
And now we no longer use it because now we're, you know, we're more interested in more realistic distributions.
But the reason why I bring it up is as a perfect example of what you're saying.
(33:24):
There was there were key question marks about ImageNet because there were categories like room and which is fascinating in itself.
And the groom category was mostly photos of like a Western centric groom, like, you know, secret and tie.
Um, and what's interesting is that the models trained on ImageNet failed.
(33:47):
And it was it was a fantastic series of studies which kind of show that if you introduce a different type of,
uh, like, you know, wedding day attire, you'd see drastic degradation in experiments.
But also even if you look at inner annotator agreement, like what people agree is a groom, it varies across the world.
(34:09):
And so this is a type of distribution shift on models are not very good at, but we need more nimble ways to still allow users to inform.
And for us to calibrate the uncertainty between these different distribution shifts.
But it's a core challenge in machine learning. Um, and also was part of the reason why ImageNet over time became really, um, much more.
(34:33):
It had a clear ceiling because when when everything is reduced to a single label,
it's not really representing the richness of how we think about our world.
So another way that I think that you've gone in a different direction than we often think about AI when we often think of profits,
um, is, um, helping nonprofit entities and new developers work with machine learning?
(34:59):
Um, why do you think it's important to empower these kinds of entities to work with this technology?
Yeah, I think for a few reasons. I mean, one is when when I started Delta, uh, which was working with the nonprofits,
it was really this idea that, um, we nonprofits typically have the least budget for technical expertise.
(35:24):
It's the last hire, right? Because a lot of of mission driven and they they're there to serve their community of interest.
And so, um, there's two things that are very humbling about working with nonprofits.
One is that you realize that for a lot of the world, um, no matter how glory, you know how much glory you attribute to AI and to your own data.
(35:51):
But this year, most nonprofits are still working in spreadsheets.
And they are. Data is hard.
One, it is not plentiful, and it is data that they are collecting day by day from their communities, um, to, to serve them better.
And so it's very humbling because, um, in fact, I think a lot of the challenge,
(36:14):
a really core challenge is how do you, uh, empower insights from like very limited samples?
And then also a lot of it is a confidence imparting, you know,
you can't leave a complex technical tool because otherwise the day you leave it won't be used.
And it's right. It's kind of fascinating, right.
(36:34):
Because a lot of your job is just to empower, um,
and to make an organization feel like they are equipped to think about their data and gain insights from it.
Um, and then on the other side of the spectrum,
there's a few nonprofits that have an immense amount of data and don't have a still have the shortages.
Rainforest connection was one of them, but there's many but still across that you realize this is a real world like this is the distribution.
(37:03):
Like there as much as we talk about, you know, being in an AI mode, it's it's a lot of how you model the world or represent the world.
You can make the best model by the end of the day. It's how people feel about that and is a useful for them.
And that's what's really interesting about that for developers and like our commitment.
(37:25):
And a lot of my work now is like also a commitment to developers.
It's also, frankly, on my part, a commitment to building talent in many parts of the world.
So even we have a scholars program which finds rising stars across the world,
and it's fully paid, and we bring people in to work with top tier researchers.
(37:47):
But that's part of that commitment that this technology is really, um.
Transformative. And so we want to really support people, build solutions within their communities, but also we want to support the next generation.
You know, I think that, um, too many of my colleagues and I came from, uh, five different universities.
(38:11):
Right. And they did the PhDs at five universities and then two if you include the UK.
And I think that's unfortunate. It's like a very small group.
And I think that that's part of changing that path a bit.
A venture Carleton alum, we have to ask, um, what are the ways that, uh,
(38:31):
the liberal arts part of your education from Carleton has been beneficial to you as you've charted your career?
Oh, I mean, crowns can change my life. I feel like I was so lucky.
I, you know, I couldn't. I mean, I remember.
So Charles visited my high school in Eswatini, in southern Africa, and it was like one of the first times.
(38:52):
Charles Coggan I think Charles is still in the admissions office.
I'm not sure, but it's been a while. But anyways, he he visited and he told me about this place called Carlton.
And so I applied and then I showed up sight unseen as an international student.
But it completely. Yeah. Did it because again, like I just grown up in this different for me.
(39:15):
I was like, oh, that's incredible. And um, Carlton gave me financial support to attend.
So it really opened up so many doors. Um, what I will say is like liberal arts as a whole, here's the thing.
Take computer science. So I did my PhD, I worked at Google.
(39:35):
Um, to get through those doors. You have to be extremely good at coding.
You have to be super good at engineering. But for example, right now engineering is in crisis, right?
Uh uh uh, models are so great that, um, and I don't like to talk about it in crisis in the sense that it will just change as a field.
(39:57):
Right? It's going to change quite a lot. Right?
I encourage my whole engineering team use whatever, you know, uh, coding assistance or copilots that you want.
Because the truth is, I think that it's going to be much more meaningful, um,
coding as an arc of what you want to get to as well as it than just, you know, being able for you to convey it as a tool.
(40:26):
Um, and what I mean by that is that most crafts, um, are both a combination, like you need to put a huge amount of work in,
but then the edge of the craft is more about the questions you ask and like what you are doing with it.
And Carlton and liberal arts in general, I think, teaches you how to work really hard at something like I worked extremely hard at Carlton,
(40:51):
and I think there were students that were better than me, and I was really inspired by that.
And when I did my fourth stage and I was working really hard, it felt like an extension of like how much I pursued things are Carlton.
Right. But also it teaches you there's not one thing.
And the same way that engineering is going to change and the craft will change,
(41:15):
the question still remain like you're still going to ask really powerful questions using technology.
And that's kind of what the essence of liberal arts is, is to teach you.
It's a teach you to sample, but to be rigorous within whatever you sample.
And it teaches you. I mean, we started off for curiosity, so I don't want to I don't want to bring it up again.
(41:36):
It's an objective, but I've always been deeply curious. So liberal arts really suited my temperament.
But I think that's the other thing, is that, you know, what remains when a craft changes is there's still new questions to ask.
And so the people who are excited about asking questions, that's what kind of drives it forward.
Well as we get toward the end of our conversation.
(41:58):
Um, one thing we always ask our guests is, in an ideal world where I could do anything.
Is there a daily task or chore that you would like to offload to an eye?
Um, and or is there something that you would never want I to take away from you because you just enjoy doing it?
(42:18):
Yeah, well, I. Self-Driving cars were my dream.
Um. And now finally, now they're finally, you know, kind of on the road.
But I put I put off, gave my driver's license until two years ago because I was like, okay, self-driving cars are coming.
And then I had to give up because I'm like, oh, they're not there yet.
(42:39):
But now they're kind of, I mean, at least in California, they're there.
But it's interesting because it might be that there's a California's lovely and sunny and they they've just mastered the sunny roads.
So we'll see how long before they're, they're widespread.
But that's one example where I just really I, I still don't like driving, even though I recently learned how to drive in California.
(43:02):
That's not an easy thing to do. I know, I know.
Yeah. So that would be one. Um, and then I think for me, uh, one of the liberating things is anything that can help with domestic chores,
I think is very liberating for women in particular, I think that we don't talk about it.
We kind of. But even, um, even now, I think that, uh, most domestic chores still fall on women.
(43:29):
And I think it would be really powerful for just women's time and all of our time if, if domestic chores can be automated more.
And so I, I'm very hopeful for that.
Uh, the tricky thing is that it's one of the most complex things because robotic so whole other, uh, kind of bag of worms.
Um, but I think the thing that I would love to not use AI for is I love scribbling notes, like, and I know those transcription services.
(43:57):
I know there's automatic, but for me, it's a tactile thing.
I just by writing something, it reinforces it.
Um, and the same for paper reading. So I still read a lot of papers and I like to come out.
I don't like digital screens. And so, um, I think that's also, you know, very tactile.
(44:18):
I think it just isolates you as well, because then that's what you're doing.
You're just reading the paper. Um, but same for books I still like my husband hates it, but when we travel, I still bring my card.
I grab a book for me, and he's like, this is so much weight in the bag.
Um, well, you carry it yourself. It's. Yeah, yeah, yeah, I, I do have to dip into his luggage sometimes because I'm like, yeah, but it's interesting.
(44:45):
I think that's an example of um, yeah.
Just preserving some areas where your focus is on something tactile in the real world.
And sometimes it helps. Yeah.
Well, there's, there been quite a few studies about notetaking and doodling being, um, better ways to learn things long term.
Yeah. Um, and so we're still encouraging students to take notes and make handwritten note sheets for class.
(45:12):
Um, for those reasons, if you know, if they're able, of course, the tools are, um, helpful if people aren't able to do it for whatever reason.
But I, I wholeheartedly agree. And I also like to curl up in a comfy chair with a good book.
Or yeah, if it has to be a paper or paper. So I have to give you credit for the the middle part of your answer about using AI for domestic work.
(45:35):
I think that's the first time anyone has successfully made a selfless answer to what is designed to be a selfish question person,
but I have to give you credit for that. Um, but, uh, the thing though, uh, well.
Right. Yeah. So to wrap up, we always give a recommendation each, um, and today,
(45:57):
and with full acknowledgment that I'm on the call with somebody who does exactly this kind of research for a living.
Um, I'm going to recommend an exercise that I designed for my students this term,
um, which is to get different chat bots to accuse each other of bias.
Uh, if you say I like it, you're right. Thank you.
(46:17):
So I, I, I did an exercise, uh, and actually designed this originally for a workshop.
I gave it Holy Cross a couple weeks ago, but, um, I asked three different models,
ChatGPT, Gemini and Lattimer, which I've talked about, uh, on the podcast before.
Um, I asked them each to write 150 word story set in a college classroom.
(46:38):
Uh, and then I said the output from that to to a different model.
Uh, I just sort of rotated it and asked them to analyze that story written by a different AI for bias.
Um, and it's it's interesting as an I mean, it's fun because all the stories were awful.
Of course, they were 150 words, and they were written with no additional prompting from a by a chat bot.
(47:00):
But, um, uh, you know, in chat bots being designed to give you what you want, they all found bias.
Certainly. Uh, but in a, you know, in the sort of ridiculous way that you would expect, uh,
you know, a high school sophomore that was told, look, for all the different ways that.
This could be biased. I just kind of went down the, you know, well, okay, this is a female protagonist,
(47:23):
which, you know, falls into this model of the female learner with a male professor,
uh, you know, or, uh, both that Latimer produced, uh, a story written by at an HBCU, um, as it's programed to do.
And so ChatGPT said, well, okay, it's great that these are people of color, but there are no white characters in this story.
(47:45):
So that's a bias. And apart from just the fun of watching I slam on each other, uh,
it is this is interesting to me in that it really kind of raised the question about what is bias and how bias is.
So such a contextual concept and is is so much about what's not there as what's there and that that trying to, you know,
(48:10):
having an algorithm look for it and clearly go down a list of all the different ways in which one can be racist, sexist and ablest.
Uh, and just try to line those up.
Is it absolutely not how one actually examines or thinks about cultural bias?
(48:30):
Yeah, I love that example. I mean, I think, uh, notion of bias is very anchored to directions of power, which differ a lot across time and space.
Uh, and it's very interesting, I think most attempts to codify bias for that reason,
you need something which is flexible across time, which can adapt, which is where most technology kind of fails.
(48:55):
Right. So it's really interesting thought it was a great example.
Yeah. Well and I'll even I'll go ahead. Oh, I'll just add a coda, which is that I did I've run this a few times, and once, uh,
I think it was ChatGPT produced a story that the the next day I said, no, actually, there's no bias here at all.
(49:16):
And it's succeeded by having no specific references to anything, like no characters are given gender or cultural characteristics of any kind.
It doesn't mention what subject is being taught in it.
It's like you've sucked all the actual narrative out of this story.
(49:37):
I just left this empty frame, but I had to agree.
I couldn't see anything that even the most aggressive person could say.
There is definitely some bias going on here. So it's interesting.
So does that mean, like, if we flatten the human condition, we're in a world with no bias?
Exactly. Yeah, you have to. Since humans are humans, you have to suck out all humanity in order to remove all bias.
(50:02):
Yeah. Really interesting. I love this example because it's right at the interface of what we humans find funny, right?
Which is AI doing things badly and something that's pretty profound, actually, which is what you've come to there.
And so I've got to try this. This is going to be fun. I am recommending this week because National DNA day is coming up at the end of April.
(50:29):
That's April 25th. If you don't have it on your calendars, I'm recommending that people check out DNA or call me.
Um, so this is something that is both kind of whimsical and actually very scientifically interesting and important.
So this is not folding paper to look like DNA. Although if you Google DNA origami, you'll get that too.
(50:53):
And if you want to do it, that's great. It's fun too. But what I'm talking about is a way of using the very predictable chemical properties of DNA to
the same DNA that contains instructions for making and maintaining you and other forms of life.
Um, we've talked about Crispr, we've talked about bird cloning here. So DNA is tied up in all of that.
But you use the chemical properties of DNA to make 3D structures.
(51:18):
And the first things that scientists made when they did this were flat objects, like little smiley faces or DNA pictures of bunnies.
Um, because why wouldn't you, if you could do that and then have moved on into making very complex 3D structures that have,
um, nanobot like properties, for example,
(51:40):
a box with a chemical lock on it that can carry a drug payload, that when the lock interacts with something on a tumor,
it unlocks the box and the drug is targeted to that tumor is one example.
And so I have provided or I will provide links to an article and also a little video about DNA origami because it is just it's just both fun and cool.
(52:10):
Sarah, what are you recommending? Um, I would love to recommend a train.
We we have two different, um, key efforts.
One is AI expense and one is AI. A vision is to expand coverages of languages that I serve, so I would love to recommend.
It doesn't even have to be the models that we do our research on.
But just try a chat bot in your language and see where it succeeds and where it fails.
(52:35):
Um, I think sometimes it's really interesting for getting to know where I serve.
See you in way I sees you. And so, um, that would be my recommendation is see how it performs in languages outside of maybe your first language,
which is English as a second language that you speak. All right.
Sounds wonderful. Well, I feel like we could have a whole other podcast conversation about AI and languages.
(53:00):
Um, so we'll have to get you to come to Carlton sometime soon.
Yeah, sure. We'll be back with the discussion of snow. If you would like to come during winter, we'll have you.
All right. That rarely entices people back, so I'm glad to hear that.
Uh, we got to you somehow while you were here. Well, thank you so much for taking the time to be with us today.
(53:23):
This was such a fun conversation, and we appreciate you being here.
I enjoyed it so much. Thank you. The Air of Curiosity podcast is recorded on the campus of Carleton College.
Your hosts are Jennifer Ross, Wolf and me, George Cusack.
Our producer is Dan Hurlbert, who records and edits each episode along with his team of hard working students.
(53:46):
Our show notes are compiled and edited by Wiebke and are available on our website Carleton.
Dot Edu slash I Mary Jo maintains our Podbean account, which gets our episodes out to whatever platform you're currently listening on.
Our theme music was composed by Nathan Wolfe Carleton, class of 27, and our mascot, Maisie was generated by Jennifer Ross Wolfe using Adobe Firefly.
Popular Podcasts
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.