On this Episode of PDXPLORES, Ameeta Agrawal, an Assistant Professor of Computer Science at the Maseeh College of Engineering & Computer Science, and Antonie Jetter, the Associate Dean for Research at the Maseeh College and a Professor of Engineering and Technology Management, discuss applying interdisciplinary research techniques to artificial intelligence (AI) language tools that fairly and accurately serve diverse communities. Through the Compassionate Computing (or, CoCo) labs, they seek to mitigate text-based biases and inequality attributes within AI to evolve this accelerated technology into natural, equitable and representational digital language.
Follow PSU research on Twitter: @psu_research and Instagram: @portlandstateresearch
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
Welcome to PDXPLORES, a Portland Stateresearch podcast featuring scholarship,
(00:20):
innovations and discoveries pushingthe boundaries of knowledge, practice,
and what is possible for the benefitof our communities and the world.
My name is Antonie Jetter.
I'm a professor in engineeringand technology management and
also the Associate Dean forResearch in the Maseeh College of
(00:40):
Engineering and Computer Science.
My name is Ameeta Agrawal, andI'm an assistant professor in the
Department of Computer Sciencein the Maseeh College of Computer
Science and Engineering here at PSU.
So, I'm interested in developing efficientand accessible computational tools,
mostly artificial intelligence tools.
Within artificial intelligence, myresearch focuses on natural language
(01:02):
processing, which is the field of makingcomputers understand human language, not
just in English, but many other languages.
In terms of grants, uh, I've beenfortunate to receive a few recently.
An NSF grant, along with Antonie,that was the first one that
got the whole thing started.
It is focusing on strengtheningAmerican infrastructure.
(01:24):
And it's a planning grant, and we'reworking with transportation experts.
The other one is NSF CRII, whichis for increasing diversity and
fairness in summarization models.
So that was another one.
And very recently, again, along withAntonie and our other partner from Coco
Lab, Compassionate Computing Lab, wewere lucky to get another grant which is
(01:47):
going to study trust in cryptocurrency.
I'm, um, what's called a participatorysystem modeler, which is a very huge
word, but fundamentally what I'm tryingto do is represent very complex systems.
Usually systems where technologyand people interact in a way that
we can make sense of them andmake good decisions about them.
(02:08):
And the participatory part meansthat I'm talking to people who know
something about these systems becausethey're either a part of the system
or they're experts in this aspect.
And I've always been super interestedin using technology to do this.
And so when I met Ameeta, Iimmediately said, "Well that's perfect.
Here's somebody who does natural languageprocessing and we need to team up."
(02:30):
And hence the grants we've worked on.
Compassionate computing was an ideaborn in 2021, and then it took a couple
of more months for it to actuallyconceptualize and become concrete.
And it was basically just understandingall of our computational tools, existing
ones, how well do they work for diversecommunities that actually use them?
(02:53):
Do these computational toolsactually benefit the people
that need them the most?
And it was largely inspired from alot of discussion around fairness
in AI, biases in a lot of ourartificial intelligence models.
So that was one aspect of it.
Mitigating the biases andthe unfairness attributes.
But the other aspect is how do youmake these tools available more
(03:16):
broadly and make them more accessible.
And also educate communitiesabout these evolving tools, which
there are so many these days.
Along Minu, Mrinalini Tanka fromthe Department of Anthropology,
also an assistant professor.
We decided to found CompassionateComputing Lab, or CoCo, and
(03:37):
there's been no looking back.
Yeah, and we had a lot of discussionsaround what to call this lab.
And we finally settled on CompassionateComputing because we were looking
for compassion in technologies.
They should be compassionate to theusers who have to put up with them.
Technical systems should also becompassionate towards different
aspects of society, differentgoals, different constraints.
(04:01):
And so we figured that bestreflects our vision with this lab.
We are an interdisciplinary lab.
As such, we are kind of virtual becausewe have students sitting in the Department
of Engineering Technology Management orin Anthropology or in Computer Science.
But we have met and we have hadstudents work together actually
in the same space and we've donehackathons and things like that.
(04:23):
But we're mostly virtual.
So natural language processing,or NLP for short, is a subfield
of artificial intelligence, or AI.
And it boils down to making computersunderstand as well as generate text
in more natural form as humans speak.
And of course, this is very difficult,given the very complex nuances
(04:50):
of our languages and not justEnglish, so many other languages.
By some estimates, there are almost7, 000 languages in the world.
And at the moment, these modelsroughly cater to about 100 or so.
So we have a long way to go.
And so that's natural language processing.
Yeah, and computational system modelingis simply representing complex system
(05:11):
in a computer so that you can doquantitative analysis, forecast how the
system will likely behave in the future.
And one example that my researchhas absolutely nothing to do
with, but that everybody'sfamiliar with, is climate models.
As you must have heard of largelanguage models, or LLMs for
(05:32):
short, are a generative AI.
These models are humongous,gigantic, massive.
They have billions to trillions ofparameters, which of course cost
a lot to develop these models.
What are millions of dollars?
But that's only one part of it, giventheir immense sizes, even deploying
(05:54):
these models for regular userswith limited technical expertise or
resources is also a huge challenge.
So that is definitely one of thebarriers for making these computational
tools a little more accessible.
And given the scaling issues thesedays, uh, we don't foresee these
models getting smaller anytime.
However, there is a great interest inmaking these models more efficient.
(06:18):
And there's a huge body of work thatis looking at more efficient AI or
efficient NLP models and how to eithermake these models smaller or make
them more reasonable for inference.
So, there's hope.
Yeah, my field is ultimatelymanagement and I understand business
and I understand the motivations ofcompanies building these large language
(06:41):
models and very often the motivationis large markets, things that are
usable in very, very broad context.
And very often good enough is good enough.
So it doesn't have to be super precise.
It also doesn't have to representeverybody and everybody's opinion.
Good enough pays the bills.
And that sometimes is a barrier tobeing truly compassionate towards
(07:04):
different groups and different users.
I think market forces certainly factorinto this because when we look at
particularly those large languagemodels, they are, while the fundamental
technology came out of research,they are really industry defined.
And with all the billions of dollarsthat need to be spent on this, they
don't happen in universities anymore.
(07:27):
And so Markets shape what companies investin and what they care about and where
they might say, "Well, we're not quitethat hung up on quality or precision."
So I think one way to close thegap is to simply think deeply
about where gaps might be and to doresearch understanding these gaps.
(07:47):
And I think another piece is byallowing people to use these tools
in the context of their needs.
To also empower them to fully understandthe technology, to see the limitations, to
advocate for improvements, and to leveragewhat's out there for their own goals and
(08:07):
their own projects and have access to it.
So, Compassionate Computing.
Basically, we try to have two priorities.
One is to educate the communities byinvolving them in our discussions and
making them aware of all the latesttechnologies and how they can use it
in their context, as Antonie mentioned.
And then the other one is hearing theirvoices and their concerns, and hopefully
(08:30):
developing tools that work for them.
And so learning from communities andunderstanding what their priorities
are, so we can focus on makingtools that are actually usable.
So, it goes both ways.
I feel it's changing quite rapidlyright now, because I think what we
are seeing is that also these largelanguage models will be off the shelf
(08:51):
capabilities where people can just kindof call them for their own projects.
And so I think with a fairly limitedamount of understanding of computer
science basics, people will be able tobuild models and use these technologies.
The problem, of course, is if youdon't know that much about it,
(09:12):
it is very difficult to judge,"Am I doing the right thing?
Might I even be harming my community?"And of course, even this very
basic computer science knowledgeis certainly not equally
distributed across all populations.
And so there will always bepeople who have more access
to the experts than others.
And that's one of the goals withthe education piece in CoCo Lab.
(09:34):
To work towards fixing that.
As I mentioned earlier, we havethousands of languages in this world.
And these models are trained on datathat has been scraped from the Internet.
And what we consider some languageshave a lot of data available.
So if you think of languagessuch as English, French, Spanish,
(09:54):
Mandarin, they already have alot of data available on the web.
Hence, these models are predominantlytrained on those language data.
For the other languages, the data is less.
And for a lot of languages,the data is non existent.
So this imbalanced distribution oflanguages in the modeled training
itself makes the model be a littlebit more favorable to processing
(10:18):
and understanding English text.
So that's just one problem.
But the problem is, given themassive scale and training and the
investment that goes into this,how do we even get to a point where
we will have The other languagesrepresented in our training data?
Who has the motivationto make that happen?
And motivations aside, thereare also technical challenges
(10:40):
in the foreseeable future.
We don't think we will have enough datafor those languages, but we would ideally
like our models to serve all communities.
So that's a technical challenge as wellas a financial challenge, as well as
who is in the power to make this happen.
So all of that really, reallycomplicates the scenario.
But you have a lot of othercommunities who are taking off the
(11:02):
shelf models and trying to comeup with innovative solutions that
work for them in their context.
The other part of that English centricview is even when you do try to have
these other languages represented,that predominant view still persists.
And that's a question that remainsopen and somebody hopefully
(11:24):
will have a solution to it.
The Internet consists of a lotof technical texts, a lot of
proper English newspaper articles.
Certainly male voices are morerepresented than female voices.
If you want to train catvideos, you're in great shape.
If you want to train otherthings, maybe not so much.
And so I think even because of how theinternet has evolved, we don't have
(11:47):
equal amounts of data and representationfor the English language either.
And so I think there's a lot of space forexploring how this might affect different
groups differently and how to fix this.
If you see that the models are notreflecting or not answering the way
you were expecting them to answer,sometimes even when you ask them to
(12:08):
answer in a language X, it might respondin language Y, if it's especially
extremely low resource language.
There are inaccuraciesright there as well.
The other risk is, it mightactually reflect some stereotypical
biases in its responses.
And that's because when the datain the beginning is less, it also
is not of a very high quality.
(12:30):
Sometimes it's plain gibberish orhas a lot of strong stereotypes.
So those get reflected in its responses.
So, not just for otherlanguages, even for English.
The response can be inaccurate or biased.
But this effect is more extremein those low resource languages
and less represented languages.
(12:52):
I think that's probably not even aspecific feature of big language models.
When whenever people haveconversations, they of course
influence each other's thinking.
But now we build technology where I'mhaving the conversation with an AI.
And that impacts my thinking.
And the AI was trained on data that,that might have a certain worldview.
(13:13):
And of course, yeah, that does causepeople to maybe adopt this worldview a
little bit more and reinforce stereotypes.
I think that's exactly thebig change that is happening.
Suddenly, these models talkto me and I talk to them as
if I talk to a real person.
The landscape of PortlandCity is very diverse.
(13:34):
We have lots of multiculturalcommunities here and that shows
in our student base, for instance.
But some non-native students, as smart asthey are, are now starting to lean on chat
GPT type of models to write their emails.
So even though they are perfectlycapable of writing an email and
expressing things that they're wantingto talk to you about, they will still
(13:57):
once pass it through a ChatGPT.
And of course it is taking away theirown style and way of expressing.
And everything is starting tolook more and more similar.
So that's just a change I've noticedas an instructor in just the last few
months, a lot of these ChatGPT emails.
And not to mention thatthey are unnecessarily long.
(14:20):
What could have been said injust a couple of sentences before
is now three paragraphs long.
So, there's that concern.
The other concern is it is also, all ofthis artificial intelligence or machine
generated text, because it looks sopolished on the surface --almost fluent
and free of any grammatical errors--it also sort of limits our ability
to distinguish between what's realand what might not be genuine.
(14:45):
But just because it's packaged ina nice way, we tend to believe it
a little bit more than we would'vebelieved something else in the past.
And I think that the other interestingquestion in this is not only do
things start to look very muchthe same, but as more and more content
on the web is produced by AI, andthe next generation AI model is then
(15:06):
trained on this content, what happensto the quality of these models?
What happens to their abilityto kind of express nuance?
If a large portion of the trainingdata is AI generated already?
So I think we're looking atjust some pretty interesting
development within the next years.
I don't think there has been a technologythat hasn't impacted our language.
(15:29):
I mean, we've used terms like,"I got my wires crossed."
Shakespeare certainlydidn't use that term, right?
So, of course, yeah, I think AIwill change language over time.
All communities areaffected by technology.
Honoring community voice would simplybe understanding what fears and
concerns people have about technology--how they want to use it, where they
(15:51):
think it might be beneficial to them--and make this part of the technology
planning to make good decisions.
And also look at things like,"Well, do we need to regulate?
Do we need to incentivize certain things?
What do we want as a societyand what is it we don't want?"
In an ideal world, we would want anartificial intelligence model, or AI
(16:11):
models, that is unbiased and fair andnot have these theoretical biases.
But it's a challenge to get there.
The biases in our AI models?
For instance, again, in the contextof NLP, assume you have a task such
as sentiment analysis, where given apiece of text, you would like it to
(16:33):
categorize it as positive or negative.
And it happens because, --again,of the training data and sometimes
during the training process, thebiases are actually amplified--
the model might say text written bycertain communities or in different
dialects or just based on certain ethnicnames present in that text is enough
to classify that text as negative.
(16:55):
And so these are some superficialassociations that the model
has picked up over time, andthey're extremely hard to revert.
And often, given the scale of our data, wemay not even notice them for a long time.
So that's one thing.
So that's in the contextof positive and negative.
But how does that reallyaffect the communities?
(17:15):
So let's say you're on a social mediaplatform, and they're using content
moderation tools, where if text isdeemed to be toxic or harmful, it may
be removed, or flagged, or deleted.
So again, the same biased sentimentanalysis models that were labeling text
as negative may also label it as harmfulor toxic, and ultimately there are voices
(17:37):
that are being removed or suppressedfrom being expressed on the social media
platforms, even if they were not toxic.
Even if they were not harmful.
So that is a very real example ofhow these bias models may end up
changing what we hear --the narrativesthat are being expressed --unfairly.
(17:58):
So for the detection part ofthese biases is where we actually
need community involvement.
Just a very small set of researchersdon't have the knowledge or the resources
to really identify these biases.
And that is precisely where communityinvolvement in using these models and
developing these models is critical.
They will be able to identify whenthese models are being biased or unfair.
(18:23):
And that's one of the goalsof CoCo, to get communities
more involved in the process.
Identifying the biases is step one,but mitigating it is even harder.
Because we're mostly using theseoff the shelf models, we don't
always have control over theparameters of the model, in that way.
(18:44):
So you're not easilyable to change the model.
And so there's a lot of activework going on in the fairness
communities in how to mitigate someof these biases.
I think there's even a weird trade-offor potential trade-off between being
biased and being a powerful, useful model.
We know it's a value to not stereotype,yet if I, um, have a big suitcase --I
(19:08):
want to get it on the MAX-- I will not askthe little old lady to help me with this.
I will probably look to some youngman who looks strong and athletic.
Because the stereotype isquite useful in this situation.
And so, as you start kind ofde-biasing models, you might run
into situations where the model justdoesn't perform that well anymore.
(19:31):
And that's why I think it's so importantto research bias in the wild in the
context of real applications so that youcan really identify, "This is bias that
hurts people and needs to be removed.
And this is bias that doesn't do anythingbad and it helps produce powerful models."
Given our other prior work and many otherstudies in this area, we were interested
(19:53):
in looking how our summarization model...
so, summarization model is somethingthat takes a really long document
and generates a shorter, more conciseversion of it, or takes a set of
documents, multiple documents, andtries to generate a coherent summary.
So, summarization model is avery interesting question or
problem to think about because ourinput can be very, very diverse.
(20:17):
It can be from differentcommunities of different dialects.
And so the question that we're exploringin this project is, "When you provide
very diverse input to your models, isthat diversity of input actually reflected
in the summary that was generated?"Our initial preliminary analysis into
this showed us that the answer was "no."And so as scientists,
(20:40):
we had to quantify it.
So we first created a new data set.
So that's what you call DivSumm.
And we collected text from threedifferent dialects --African
American Vernacular English,Hispanic English, and White English--
and we created a summarization data set.
So you have all of thesedialects represented in the text.
(21:04):
And the question is, "First of all, howwould humans summarize such diverse data?"
So that brought us back to communityinvolvement where we had to, through
the help of many self expressed diversestudents who are fluent in these different
dialects, help us summarize the text.
And it was very, very satisfyingto see that humans generate
(21:27):
very balanced summaries.
So in the summaries that wereactually generated by humans, we
saw a proportional distributionof these dialects, which is great.
Gives us hope that we can go ahead andactually try to do this computationally.
And like I mentioned before, the offthe shelf models are not really good
at generating balanced summaries.
So this project then explores this problemand tries to propose new summarization
(21:51):
models that can actually generate morebalanced, more representative, summaries.
And one of the ideas that went intothis project actually came from The
world of participatory system modelingwhere we learn that there is this
notion that if you ask a lot of folksin the community and they all contribute
to the model, the more knowledgeyou embed, the better the model.
(22:11):
And in our own work, we foundthat that is not always true.
But sometimes the collective intelligenceof way too many people causes our system
models to kind of focus on the evidencestuff that everybody can agree on.
And to use all of the diversityand knowledge about system aspects
that not everybody has observed.
(22:32):
So I kind of posed the questionto Ameeta and said, "Well,
doesn't that happen in NLP too?
Is more not always better?"Because currently, NLP models are trained
on the most amount of data available.
So if you have this assumption, it alwaysgets better as you add data points.
And at least for the summarization,we found that some of the tricks that
work for the system modeling approachto fix this problem of crowding-out
(22:55):
important information also appliesto summarization and NLP models.
So as educators, we've noticedthat students get really, really,
really enthusiastic and excitedto try out any new technology.
So when ChatGPT was introduced, itwas very natural for CoCo to start
thinking of how can we come up with anactivity, an event, that is centered
(23:16):
around all of this new AI tools.
And from there, we decided tohave a one day event, so we called
it the Ideathon, based on whatAntonia has once done in the past.
And also at that point, I realized thatthe word "hackathon," which I'm more
familiar with as a computer scientist,doesn't have the same connotation in
other communities and disciplines.
(23:37):
So, we settled on Ideathon, and it wasAI tools for creative storytelling.
And of course, you need to bondyour problem a little bit, so the
scope was climate change in Oregon.
Something that we're all very, verypassionate about, worried about.
So, we brainstormed for a couple ofweeks, and thanks to a lot of help from
(23:57):
our students, from computer science,as well as ETM, we all came up with
this very neatly planned one day event.
And, it was open to students fromany discipline, frankly, at PSU.
And at the end we had about10 to 12 different disciplines
represented that day.
It was amazing, to say the least,to see what the students came
(24:21):
up with in a span of mere hours.
So of course we had breakfast and ateam building activity, and then lunch.
Of these students go in verydifferent, diverse teams.
So one of the constraints thatwe put on the ideathon was there
should be at least three differentdisciplines represented in each team.
And so that was good.
And these themes went along, brainstormeda couple of ideas, came back for
(24:44):
lunch, let these ideas simmer, andthen actually went back again and
executed those ideas, which was followedby a presentation by the students.
And it was mind blowing tosee what they had created.
So there was a team that did a mockup of a phone application, or an
app, for showing what the world --or,specifically Oregon-- would look
(25:05):
like if we don't address the effectsof climate change versus if we did.
So you would still have MountHood to go to, or, you know, what
would the Oregon coast look like?
And so they were showing you all ofthese doomsday scenarios versus, uh, what
if we took better care of our planet?
And of course, I should clarifythis point, that ChatGPT is just
(25:27):
one of the AI tools that we have.
It does text and increasinglystarting to also do visuals, but
there are also many other toolsthat the students use that day.
One of them is called Midjourney,which generates images, and then
there's several other AI tools thatcan help you generate something.
So a combination oftools were used that day.
(25:48):
Another team did a chatbot, aconversational model, except that
it speaks like an Oregon farmer.
Because, you know, when farmershave questions and they would
like to maybe possibly use one ofthese technologies, how can they
get more contextualized responses?
So you know that, what does it mean toask for how to take care of strawberries
(26:09):
in June in Oregon versus in Florida?
Another team created a storybook forchildren, because naturally they're the
future and how can we convey climatechange to young minds in a more creative,
visual way, in a more interesting way?
And that was fabulous.
That was really really very cool to seethe visuals that they came up with and the
(26:31):
characters were also generated using AI.
So you had a beaver, a mountainhood and a Doug fur, as your main
characters in that storybook.
And then my personal favorite was a play.
Again, all the screenplay, thewriting, the backdrops and the
narratives, everything generatedusing multiple AI tools.
(26:52):
And the motivation for that was,"Well, climate change affects
farmers or rural community severely.
But how do we conveythis to the community?"
And of course, they're not readingresearch papers or technical articles.
They go to community fairs and learnand discuss from friends and family.
So this was a play to convey theeffects and possible mitigation
(27:16):
strategies, adaptation strategies.
And that was my personal favorite.
But, the teams, they blew my mind.
It's something that I'm still thinkingabout a couple of months later.
And it was amazing.
And I can't wait to do another one.
We can't wait.
Of course, all this was done not justby CoCo, but we also had some fabulous
(27:37):
mentors --faculty mentors-- who were verygenerous with their time and their ideas.
They came to the event, and I'd liketo mention a special thanks to Dr.
Brianne Suldovsky from the ScienceCommunication Department and Dr.
Kathi Inman Berens from the EnglishLiterature Department, who really helped
the students think about and emphasizenot just what they're developing, but also
(27:59):
first, who they're developing this for,which actually shapes what you develop.
My name is Antonio Jenner, andin my research I try to make
sense of super complex systemswith the help of what
people know about them.
And increasingly, I'mtrying to do this with NLP.
And my name is Ameeta Agrawal, andI'm interested in developing AI models
(28:24):
that actually work for people, thatreflect their voices, and also in
educating people about these AI tools.
We have this CoCo Lab, and underthis umbrella of interdisciplinary
research, we are able to do thingsthat we probably and most likely were
not going to be able to do otherwise.
(28:55):
Thank you for listening to PDXPLORES.
If you liked what you heard on thisepisode, please rate and follow the
show anywhere you get your podcasts.
Welcome to PDXPLORES, a Portland Stateresearch podcast featuring scholarship,
(00:20):
innovations and discoveries pushingthe boundaries of knowledge, practice,
and what is possible for the benefitof our communities and the world.
My name is Antonie Jetter.
I'm a professor in engineeringand technology management and
also the Associate Dean forResearch in the Maseeh College of
(00:40):
Engineering and Computer Science.
My name is Ameeta Agrawal, andI'm an assistant professor in the
Department of Computer Sciencein the Maseeh College of Computer
Science and Engineering here at PSU.
So, I'm interested in developing efficientand accessible computational tools,
mostly artificial intelligence tools.
Within artificial intelligence, myresearch focuses on natural language
(01:02):
processing, which is the field of makingcomputers understand human language, not
just in English, but many other languages.
In terms of grants, uh, I've beenfortunate to receive a few recently.
An NSF grant, along with Antonie,that was the first one that
got the whole thing started.
It is focusing on strengtheningAmerican infrastructure.
(01:24):
And it's a planning grant, and we'reworking with transportation experts.
The other one is NSF CRII, whichis for increasing diversity and
fairness in summarization models.
So that was another one.
And very recently, again, along withAntonie and our other partner from Coco
Lab, Compassionate Computing Lab, wewere lucky to get another grant which is
(01:47):
going to study trust in cryptocurrency.
I'm, um, what's called a participatorysystem modeler, which is a very huge
word, but fundamentally what I'm tryingto do is represent very complex systems.
Usually systems where technologyand people interact in a way that
we can make sense of them andmake good decisions about them.
(02:08):
And the participatory part meansthat I'm talking to people who know
something about these systems becausethey're either a part of the system
or they're experts in this aspect.
And I've always been super interestedin using technology to do this.
And so when I met Ameeta, Iimmediately said, "Well that's perfect.
Here's somebody who does natural languageprocessing and we need to team up."
(02:30):
And hence the grants we've worked on.
Compassionate computing was an ideaborn in 2021, and then it took a couple
of more months for it to actuallyconceptualize and become concrete.
And it was basically just understandingall of our computational tools, existing
ones, how well do they work for diversecommunities that actually use them?
(02:53):
Do these computational toolsactually benefit the people
that need them the most?
And it was largely inspired from alot of discussion around fairness
in AI, biases in a lot of ourartificial intelligence models.
So that was one aspect of it.
Mitigating the biases andthe unfairness attributes.
But the other aspect is how do youmake these tools available more
(03:16):
broadly and make them more accessible.
And also educate communitiesabout these evolving tools, which
there are so many these days.
Along Minu, Mrinalini Tanka fromthe Department of Anthropology,
also an assistant professor.
We decided to found CompassionateComputing Lab, or CoCo, and
(03:37):
there's been no looking back.
Yeah, and we had a lot of discussionsaround what to call this lab.
And we finally settled on CompassionateComputing because we were looking
for compassion in technologies.
They should be compassionate to theusers who have to put up with them.
Technical systems should also becompassionate towards different
aspects of society, differentgoals, different constraints.
(04:01):
And so we figured that bestreflects our vision with this lab.
We are an interdisciplinary lab.
As such, we are kind of virtual becausewe have students sitting in the Department
of Engineering Technology Management orin Anthropology or in Computer Science.
But we have met and we have hadstudents work together actually
in the same space and we've donehackathons and things like that.
(04:23):
But we're mostly virtual.
So natural language processing,or NLP for short, is a subfield
of artificial intelligence, or AI.
And it boils down to making computersunderstand as well as generate text
in more natural form as humans speak.
And of course, this is very difficult,given the very complex nuances
(04:50):
of our languages and not justEnglish, so many other languages.
By some estimates, there are almost7, 000 languages in the world.
And at the moment, these modelsroughly cater to about 100 or so.
So we have a long way to go.
And so that's natural language processing.
Yeah, and computational system modelingis simply representing complex system
(05:11):
in a computer so that you can doquantitative analysis, forecast how the
system will likely behave in the future.
And one example that my researchhas absolutely nothing to do
with, but that everybody'sfamiliar with, is climate models.
As you must have heard of largelanguage models, or LLMs for
(05:32):
short, are a generative AI.
These models are humongous,gigantic, massive.
They have billions to trillions ofparameters, which of course cost
a lot to develop these models.
What are millions of dollars?
But that's only one part of it, giventheir immense sizes, even deploying
(05:54):
these models for regular userswith limited technical expertise or
resources is also a huge challenge.
So that is definitely one of thebarriers for making these computational
tools a little more accessible.
And given the scaling issues thesedays, uh, we don't foresee these
models getting smaller anytime.
However, there is a great interest inmaking these models more efficient.
(06:18):
And there's a huge body of work thatis looking at more efficient AI or
efficient NLP models and how to eithermake these models smaller or make
them more reasonable for inference.
So, there's hope.
Yeah, my field is ultimatelymanagement and I understand business
and I understand the motivations ofcompanies building these large language
(06:41):
models and very often the motivationis large markets, things that are
usable in very, very broad context.
And very often good enough is good enough.
So it doesn't have to be super precise.
It also doesn't have to representeverybody and everybody's opinion.
Good enough pays the bills.
And that sometimes is a barrier tobeing truly compassionate towards
(07:04):
different groups and different users.
I think market forces certainly factorinto this because when we look at
particularly those large languagemodels, they are, while the fundamental
technology came out of research,they are really industry defined.
And with all the billions of dollarsthat need to be spent on this, they
don't happen in universities anymore.
(07:27):
And so Markets shape what companies investin and what they care about and where
they might say, "Well, we're not quitethat hung up on quality or precision."
So I think one way to close thegap is to simply think deeply
about where gaps might be and to doresearch understanding these gaps.
(07:47):
And I think another piece is byallowing people to use these tools
in the context of their needs.
To also empower them to fully understandthe technology, to see the limitations, to
advocate for improvements, and to leveragewhat's out there for their own goals and
(08:07):
their own projects and have access to it.
So, Compassionate Computing.
Basically, we try to have two priorities.
One is to educate the communities byinvolving them in our discussions and
making them aware of all the latesttechnologies and how they can use it
in their context, as Antonie mentioned.
And then the other one is hearing theirvoices and their concerns, and hopefully
(08:30):
developing tools that work for them.
And so learning from communities andunderstanding what their priorities
are, so we can focus on makingtools that are actually usable.
So, it goes both ways.
I feel it's changing quite rapidlyright now, because I think what we
are seeing is that also these largelanguage models will be off the shelf
(08:51):
capabilities where people can just kindof call them for their own projects.
And so I think with a fairly limitedamount of understanding of computer
science basics, people will be able tobuild models and use these technologies.
The problem, of course, is if youdon't know that much about it,
(09:12):
it is very difficult to judge,"Am I doing the right thing?
Might I even be harming my community?"And of course, even this very
basic computer science knowledgeis certainly not equally
distributed across all populations.
And so there will always bepeople who have more access
to the experts than others.
And that's one of the goals withthe education piece in CoCo Lab.
(09:34):
To work towards fixing that.
As I mentioned earlier, we havethousands of languages in this world.
And these models are trained on datathat has been scraped from the Internet.
And what we consider some languageshave a lot of data available.
So if you think of languagessuch as English, French, Spanish,
(09:54):
Mandarin, they already have alot of data available on the web.
Hence, these models are predominantlytrained on those language data.
For the other languages, the data is less.
And for a lot of languages,the data is non existent.
So this imbalanced distribution oflanguages in the modeled training
itself makes the model be a littlebit more favorable to processing
(10:18):
and understanding English text.
So that's just one problem.
But the problem is, given themassive scale and training and the
investment that goes into this,how do we even get to a point where
we will have The other languagesrepresented in our training data?
Who has the motivationto make that happen?
And motivations aside, thereare also technical challenges
(10:40):
in the foreseeable future.
We don't think we will have enough datafor those languages, but we would ideally
like our models to serve all communities.
So that's a technical challenge as wellas a financial challenge, as well as
who is in the power to make this happen.
So all of that really, reallycomplicates the scenario.
But you have a lot of othercommunities who are taking off the
(11:02):
shelf models and trying to comeup with innovative solutions that
work for them in their context.
The other part of that English centricview is even when you do try to have
these other languages represented,that predominant view still persists.
And that's a question that remainsopen and somebody hopefully
(11:24):
will have a solution to it.
The Internet consists of a lotof technical texts, a lot of
proper English newspaper articles.
Certainly male voices are morerepresented than female voices.
If you want to train catvideos, you're in great shape.
If you want to train otherthings, maybe not so much.
And so I think even because of how theinternet has evolved, we don't have
(11:47):
equal amounts of data and representationfor the English language either.
And so I think there's a lot of space forexploring how this might affect different
groups differently and how to fix this.
If you see that the models are notreflecting or not answering the way
you were expecting them to answer,sometimes even when you ask them to
(12:08):
answer in a language X, it might respondin language Y, if it's especially
extremely low resource language.
There are inaccuraciesright there as well.
The other risk is, it mightactually reflect some stereotypical
biases in its responses.
And that's because when the datain the beginning is less, it also
is not of a very high quality.
(12:30):
Sometimes it's plain gibberish orhas a lot of strong stereotypes.
So those get reflected in its responses.
So, not just for otherlanguages, even for English.
The response can be inaccurate or biased.
But this effect is more extremein those low resource languages
and less represented languages.
(12:52):
I think that's probably not even aspecific feature of big language models.
When whenever people haveconversations, they of course
influence each other's thinking.
But now we build technology where I'mhaving the conversation with an AI.
And that impacts my thinking.
And the AI was trained on data that,that might have a certain worldview.
(13:13):
And of course, yeah, that does causepeople to maybe adopt this worldview a
little bit more and reinforce stereotypes.
I think that's exactly thebig change that is happening.
Suddenly, these models talkto me and I talk to them as
if I talk to a real person.
The landscape of PortlandCity is very diverse.
(13:34):
We have lots of multiculturalcommunities here and that shows
in our student base, for instance.
But some non-native students, as smart asthey are, are now starting to lean on chat
GPT type of models to write their emails.
So even though they are perfectlycapable of writing an email and
expressing things that they're wantingto talk to you about, they will still
(13:57):
once pass it through a ChatGPT.
And of course it is taking away theirown style and way of expressing.
And everything is starting tolook more and more similar.
So that's just a change I've noticedas an instructor in just the last few
months, a lot of these ChatGPT emails.
And not to mention thatthey are unnecessarily long.
(14:20):
What could have been said injust a couple of sentences before
is now three paragraphs long.
So, there's that concern.
The other concern is it is also, all ofthis artificial intelligence or machine
generated text, because it looks sopolished on the surface --almost fluent
and free of any grammatical errors--it also sort of limits our ability
to distinguish between what's realand what might not be genuine.
(14:45):
But just because it's packaged ina nice way, we tend to believe it
a little bit more than we would'vebelieved something else in the past.
And I think that the other interestingquestion in this is not only do
things start to look very muchthe same, but as more and more content
on the web is produced by AI, andthe next generation AI model is then
(15:06):
trained on this content, what happensto the quality of these models?
What happens to their abilityto kind of express nuance?
If a large portion of the trainingdata is AI generated already?
So I think we're looking atjust some pretty interesting
development within the next years.
I don't think there has been a technologythat hasn't impacted our language.
(15:29):
I mean, we've used terms like,"I got my wires crossed."
Shakespeare certainlydidn't use that term, right?
So, of course, yeah, I think AIwill change language over time.
All communities areaffected by technology.
Honoring community voice would simplybe understanding what fears and
concerns people have about technology--how they want to use it, where they
(15:51):
think it might be beneficial to them--and make this part of the technology
planning to make good decisions.
And also look at things like,"Well, do we need to regulate?
Do we need to incentivize certain things?
What do we want as a societyand what is it we don't want?"
In an ideal world, we would want anartificial intelligence model, or AI
(16:11):
models, that is unbiased and fair andnot have these theoretical biases.
But it's a challenge to get there.
The biases in our AI models?
For instance, again, in the contextof NLP, assume you have a task such
as sentiment analysis, where given apiece of text, you would like it to
(16:33):
categorize it as positive or negative.
And it happens because, --again,of the training data and sometimes
during the training process, thebiases are actually amplified--
the model might say text written bycertain communities or in different
dialects or just based on certain ethnicnames present in that text is enough
to classify that text as negative.
(16:55):
And so these are some superficialassociations that the model
has picked up over time, andthey're extremely hard to revert.
And often, given the scale of our data, wemay not even notice them for a long time.
So that's one thing.
So that's in the contextof positive and negative.
But how does that reallyaffect the communities?
(17:15):
So let's say you're on a social mediaplatform, and they're using content
moderation tools, where if text isdeemed to be toxic or harmful, it may
be removed, or flagged, or deleted.
So again, the same biased sentimentanalysis models that were labeling text
as negative may also label it as harmfulor toxic, and ultimately there are voices
(17:37):
that are being removed or suppressedfrom being expressed on the social media
platforms, even if they were not toxic.
Even if they were not harmful.
So that is a very real example ofhow these bias models may end up
changing what we hear --the narrativesthat are being expressed --unfairly.
(17:58):
So for the detection part ofthese biases is where we actually
need community involvement.
Just a very small set of researchersdon't have the knowledge or the resources
to really identify these biases.
And that is precisely where communityinvolvement in using these models and
developing these models is critical.
They will be able to identify whenthese models are being biased or unfair.
(18:23):
And that's one of the goalsof CoCo, to get communities
more involved in the process.
Identifying the biases is step one,but mitigating it is even harder.
Because we're mostly using theseoff the shelf models, we don't
always have control over theparameters of the model, in that way.
(18:44):
So you're not easilyable to change the model.
And so there's a lot of activework going on in the fairness
communities in how to mitigate someof these biases.
I think there's even a weird trade-offor potential trade-off between being
biased and being a powerful, useful model.
We know it's a value to not stereotype,yet if I, um, have a big suitcase --I
(19:08):
want to get it on the MAX-- I will not askthe little old lady to help me with this.
I will probably look to some youngman who looks strong and athletic.
Because the stereotype isquite useful in this situation.
And so, as you start kind ofde-biasing models, you might run
into situations where the model justdoesn't perform that well anymore.
(19:31):
And that's why I think it's so importantto research bias in the wild in the
context of real applications so that youcan really identify, "This is bias that
hurts people and needs to be removed.
And this is bias that doesn't do anythingbad and it helps produce powerful models."
Given our other prior work and many otherstudies in this area, we were interested
(19:53):
in looking how our summarization model...
so, summarization model is somethingthat takes a really long document
and generates a shorter, more conciseversion of it, or takes a set of
documents, multiple documents, andtries to generate a coherent summary.
So, summarization model is avery interesting question or
problem to think about because ourinput can be very, very diverse.
(20:17):
It can be from differentcommunities of different dialects.
And so the question that we're exploringin this project is, "When you provide
very diverse input to your models, isthat diversity of input actually reflected
in the summary that was generated?"Our initial preliminary analysis into
this showed us that the answer was "no."And so as scientists,
(20:40):
we had to quantify it.
So we first created a new data set.
So that's what you call DivSumm.
And we collected text from threedifferent dialects --African
American Vernacular English,Hispanic English, and White English--
and we created a summarization data set.
So you have all of thesedialects represented in the text.
(21:04):
And the question is, "First of all, howwould humans summarize such diverse data?"
So that brought us back to communityinvolvement where we had to, through
the help of many self expressed diversestudents who are fluent in these different
dialects, help us summarize the text.
And it was very, very satisfyingto see that humans generate
(21:27):
very balanced summaries.
So in the summaries that wereactually generated by humans, we
saw a proportional distributionof these dialects, which is great.
Gives us hope that we can go ahead andactually try to do this computationally.
And like I mentioned before, the offthe shelf models are not really good
at generating balanced summaries.
So this project then explores this problemand tries to propose new summarization
(21:51):
models that can actually generate morebalanced, more representative, summaries.
And one of the ideas that went intothis project actually came from The
world of participatory system modelingwhere we learn that there is this
notion that if you ask a lot of folksin the community and they all contribute
to the model, the more knowledgeyou embed, the better the model.
(22:11):
And in our own work, we foundthat that is not always true.
But sometimes the collective intelligenceof way too many people causes our system
models to kind of focus on the evidencestuff that everybody can agree on.
And to use all of the diversityand knowledge about system aspects
that not everybody has observed.
(22:32):
So I kind of posed the questionto Ameeta and said, "Well,
doesn't that happen in NLP too?
Is more not always better?"Because currently, NLP models are trained
on the most amount of data available.
So if you have this assumption, it alwaysgets better as you add data points.
And at least for the summarization,we found that some of the tricks that
work for the system modeling approachto fix this problem of crowding-out
(22:55):
important information also appliesto summarization and NLP models.
So as educators, we've noticedthat students get really, really,
really enthusiastic and excitedto try out any new technology.
So when ChatGPT was introduced, itwas very natural for CoCo to start
thinking of how can we come up with anactivity, an event, that is centered
(23:16):
around all of this new AI tools.
And from there, we decided tohave a one day event, so we called
it the Ideathon, based on whatAntonia has once done in the past.
And also at that point, I realized thatthe word "hackathon," which I'm more
familiar with as a computer scientist,doesn't have the same connotation in
other communities and disciplines.
(23:37):
So, we settled on Ideathon, and it wasAI tools for creative storytelling.
And of course, you need to bondyour problem a little bit, so the
scope was climate change in Oregon.
Something that we're all very, verypassionate about, worried about.
So, we brainstormed for a couple ofweeks, and thanks to a lot of help from
(23:57):
our students, from computer science,as well as ETM, we all came up with
this very neatly planned one day event.
And, it was open to students fromany discipline, frankly, at PSU.
And at the end we had about10 to 12 different disciplines
represented that day.
It was amazing, to say the least,to see what the students came
(24:21):
up with in a span of mere hours.
So of course we had breakfast and ateam building activity, and then lunch.
Of these students go in verydifferent, diverse teams.
So one of the constraints thatwe put on the ideathon was there
should be at least three differentdisciplines represented in each team.
And so that was good.
And these themes went along, brainstormeda couple of ideas, came back for
(24:44):
lunch, let these ideas simmer, andthen actually went back again and
executed those ideas, which was followedby a presentation by the students.
And it was mind blowing tosee what they had created.
So there was a team that did a mockup of a phone application, or an
app, for showing what the world --or,specifically Oregon-- would look
(25:05):
like if we don't address the effectsof climate change versus if we did.
So you would still have MountHood to go to, or, you know, what
would the Oregon coast look like?
And so they were showing you all ofthese doomsday scenarios versus, uh, what
if we took better care of our planet?
And of course, I should clarifythis point, that ChatGPT is just
(25:27):
one of the AI tools that we have.
It does text and increasinglystarting to also do visuals, but
there are also many other toolsthat the students use that day.
One of them is called Midjourney,which generates images, and then
there's several other AI tools thatcan help you generate something.
So a combination oftools were used that day.
(25:48):
Another team did a chatbot, aconversational model, except that
it speaks like an Oregon farmer.
Because, you know, when farmershave questions and they would
like to maybe possibly use one ofthese technologies, how can they
get more contextualized responses?
So you know that, what does it mean toask for how to take care of strawberries
(26:09):
in June in Oregon versus in Florida?
Another team created a storybook forchildren, because naturally they're the
future and how can we convey climatechange to young minds in a more creative,
visual way, in a more interesting way?
And that was fabulous.
That was really really very cool to seethe visuals that they came up with and the
(26:31):
characters were also generated using AI.
So you had a beaver, a mountainhood and a Doug fur, as your main
characters in that storybook.
And then my personal favorite was a play.
Again, all the screenplay, thewriting, the backdrops and the
narratives, everything generatedusing multiple AI tools.
(26:52):
And the motivation for that was,"Well, climate change affects
farmers or rural community severely.
But how do we conveythis to the community?"
And of course, they're not readingresearch papers or technical articles.
They go to community fairs and learnand discuss from friends and family.
So this was a play to convey theeffects and possible mitigation
(27:16):
strategies, adaptation strategies.
And that was my personal favorite.
But, the teams, they blew my mind.
It's something that I'm still thinkingabout a couple of months later.
And it was amazing.
And I can't wait to do another one.
We can't wait.
Of course, all this was done not justby CoCo, but we also had some fabulous
(27:37):
mentors --faculty mentors-- who were verygenerous with their time and their ideas.
They came to the event, and I'd liketo mention a special thanks to Dr.
Brianne Suldovsky from the ScienceCommunication Department and Dr.
Kathi Inman Berens from the EnglishLiterature Department, who really helped
the students think about and emphasizenot just what they're developing, but also
(27:59):
first, who they're developing this for,which actually shapes what you develop.
My name is Antonio Jenner, andin my research I try to make
sense of super complex systemswith the help of what
people know about them.
And increasingly, I'mtrying to do this with NLP.
And my name is Ameeta Agrawal, andI'm interested in developing AI models
(28:24):
that actually work for people, thatreflect their voices, and also in
educating people about these AI tools.
We have this CoCo Lab, and underthis umbrella of interdisciplinary
research, we are able to do thingsthat we probably and most likely were
not going to be able to do otherwise.
(28:55):
Thank you for listening to PDXPLORES.
If you liked what you heard on thisepisode, please rate and follow the
show anywhere you get your podcasts.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
iHeartRadio 24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com