March 26, 2025 • 40 mins
Throughout human history, we have relied on technology to make our work easier. In this episode, Michelle Miller joins us to discuss how to foster students’ critical thinking skills in the age of AI.
Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology, Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World and A Teacher’s Guide to Learning Students’ Names: Why You Should, Why It’s Hard, How You Can. Michelle is also a frequent contributor of articles on teaching and learning in higher education to a variety of publications including The Chronicle of Higher Ed.
A transcript of this episode and show notes may be found at http://teaforteaching.com.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
Throughout human history, we have relied on technology to
make our work easier. In this episode, we discuss how to foster students’ critical
thinking skills in the age of AI. Thanks for joining us for Tea for Teaching,
an informal discussion of innovative and effective practices in teaching and learning.
(00:23):
This podcast series is hosted by John Kane, an economist...
...and Rebecca Mushtare, a graphic designer......and features guests doing important research
and advocacy work to make higher education more inclusive and supportive of all learners.
(00:46):
Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and
President’s Distinguished Teaching Fellow at Northern Arizona University. She is the
author of Minds Online (00:55):
Teaching Effectively with Technology, Remembering and Forgetting in the Age
of Technology (01:00):
Teaching, Learning, and the Science of Memory in a Wired World and A Teacher’s Guide
to Learning Students’ Names (01:06):
Why You Should, Why It’s Hard, How You Can. That list keeps growing
since we first interviewed you. And Michelle is also a frequent contributor of articles on
teaching and learning in higher education to a variety of publications including The
Chronicle of Higher Ed. Welcome back, Michelle.Oh, thank you. And yes, it's really satisfying to
(01:28):
see these books making their way out into the world. And if it weren't for engaged faculty
audiences, that wouldn't happen. So I guess I'll put that gratitude out to the audience as well.
Today's teas are:... Michelle, are you drinking tea?
I sure am, I've got a nice green tea from Hong Kong.
Nice. How about you, John?I am drinking a pure peppermint tea today.
(01:50):
So far, we've got a good variety, and I'll follow up with some black
tea. I have a Lady Grey today.Very good. So we've invited you
here to discuss the development of critical thinking skills. And let's begin by talking
about what critical thinking is and any misconceptions or missed opportunities that
you've noted in your work around this issue.So this is something that I've really started
(02:13):
engaging with myself back when I was writing the book Minds Online. And those who are familiar with
its structure know that I try to use a really, really simple little heuristic for divvying up
the cognitive demands and some of the relevant cognitive research and processes that are involved
in the type of learning that we address in higher education, and one of those is thinking, and even
(02:35):
within that is this topic of critical thinking skills. And so I had just kind of caught myself,
year after year, monitoring what was going on in the development of research around this,
of course, thinking about it in my own teaching, but also finding myself being really reticent when
I was working with faculty audiences to say, well, this is a really kind of a, if not totally unique,
(02:57):
a pretty specialized subset within thinking skills more generally, and then the bigger,
broader umbrella of cognition in general. So it's almost like the first missed opportunity is my
opportunity to engage with faculty around this. So I love it, it's this really deep topic. But
that's one reason why it's hard to just say, oh, and in five minutes, sandwiched in between five
(03:18):
other things, we're gonna address this. And so that's what I've kind of been reexamining myself.
So when we talk about what is critical thinking, that is actually the first really big challenge
and, or sometimes misconception around the topic. And no surprise, there’s differences of opinion
in defining things, and of course, really getting into that challenge of how do we define a thing
(03:40):
and pin it down, and let's all kind of debate the definition. But critical thinking here really is a
special case in that there have been quite a few attempts to make these very general sort of top
level, kind of elaborations of what it is. Think about words like evaluate or analyze, reflect and
so on. But I don't think we have converged on one of these as really the best across disciplines,
(04:06):
and I kind of fall in with some other thinkers in the sphere who've said maybe a discipline
independent, context agnostic definition doesn't exist, because maybe the phenomenon itself doesn't
exist. So I've, myself, really been drawn towards the more, I guess you'd call them,
situated or contextualized definitions of what it is. If you drill down a little bit further,
(04:29):
you get, for example, Diane Halpern, psychologist, this towering figure in our field. So naturally,
I would gravitate towards what she says, but she says things like evaluating and analyzing
evidence in the service of making decisions, so thought processes that lead to a good outcome. So
that's a little bit more practical, but it still doesn't really make it so a given faculty member
(04:52):
teaching in a given course can say this is what it looks like when students came in without a
critical thinking skill and they leave with it. And as with anything, if we don't have that,
then no surprise that we're maybe not going to get there, or not get there as completely as we
had hoped to. And so that's one kind of set of, if not misconceptions, challenges or things that
(05:14):
people often don't realize when they wade into this issue of teaching critical thinking. And
it follows from that, unfortunately, that we can have the misconception, that we can describe it
to students and they'll get it. I know, for me, I would be embarrassed to say how many years I
taught, for example, Introduction to Psychology, which we pride ourselves on being like this is the
critical thinking class. And even if you never take another psychology course, you're going
(05:36):
to come away with these great reasoning skills. And gosh darn it, I had a whole half of one class
period that we discussed it, maybe, and it was on the study guide somewhere. So we think, “Oh,
okay, I'll just tell students this is important.” Say, “think critically.” And of course, naturally,
they don't know what it is either. And I can really see, in my mind, at least,
(05:57):
how students could come away with the idea that critical thinking means just sort of criticizing
everything indiscriminately. It means nobody knows anything, a total relativism, that no, that's not
what we want, or in its worst forms, it can read too as just like, agree with me. Like I can go and
say, Look, “I'm a psychologist, and I'm pretty critical of the idea of horoscopes.” And if you
(06:17):
think that your horoscope is going to predict your future, you're not engaging in critical thinking,
you're wrong. So that's what we want to get away from. And where I think that it can be this really
great fertile ground for faculty if they step back and say, “Okay, what does this really mean at that
more actionable level,” and then start to align instruction to that, because it's gonna take a lot
more than half a day, I'll tell you that much. Seems like you're suggesting that it would be
(06:41):
helpful for students to be able to identify specific behaviors or specific practices that
we would identify as critical thinking. But if they know what the beginning, middle and end of
an action is, they have a better idea of what that looks like and what that feels like.
Yeah, absolutely. And that's also a pretty psychological way of coming at it as well. Like,
(07:01):
okay, of course, this is something that happens in your mind, out of view, between your ears,
but I want to see what happens on the page or the decisions that you do make in the world that align
with what I would consider to be evaluating and analyzing evidence in a powerful way. And yeah,
when we get into a little bit more of the challenges, I mean what I've laid out for faculty,
(07:22):
and again, really drawing on a lot of research that's been out there for a while, even beyond
that defining issue, which, yeah, that's going to challenge faculty. It's going to get in the way of
designing the activities. It's going to challenge students. They don't know what they're trying to
aim for to meet this thing that's on the syllabus. It is also what I would almost call a cueing
problem, or someone's just kind of off the cuff, called a “when to” versus a “how to” problem with
(07:45):
a lot of the skills that we are helping students develop. It's “Okay, here is how you break this
down. Here is how you conduct the statistical test. Here is how you read an APA style
research article.” But this is something that's like, “Okay, here's how to find, for example,
issues with correlation versus causation in the interpretation of a study.” Great,
(08:06):
we can do that. But are you going to know to do that when you're writing your paper? And even
a more far transfer of that as you're scrolling through whatever social media feed of choice is,
and you get the like, well, I drank a cup of lemon juice a day, and now I'm healthy as a horse. It's
sort of the same problem. Am I going to have that sort of inclination or disposition to say,
(08:26):
“Oh, right, this is when I need to do this.” So there's that. A lot of folks have talked about
effort as well, part of this running theme of cognitive miserliness, that our brains and our
minds really seem to run in high-efficiency mode, where we save time and save energy over
getting to a more idealized solution. And also, as a lot of folks have talked about,
a motivation to maintain beliefs, whether it's a very profound thing, like, “I don't
(08:49):
want to critique this belief because it's really foundational for intellectually or emotionally
or spiritually,” or even just dumb things like, “Oh wow, I really want to buy this thing I just
saw an ad for. And maybe I just won't take a look at all those negative reviews,” or “I'll assume
that the reviews that all have these wonderful five stars are perfectly reputable.” So all of
(09:10):
those things form quite a steep set of challenges for students and for faculty alike, right?
Very much so. And what makes it more complex, and I think you alluded to this, is that each
discipline has its own definition of what critical thinking skills are. I remember I spent about
10 years on faculty assembly one semester where they were coming up with a new Gen Ed curriculum,
(09:32):
and one of the main issues was, how are we going to build critical thinking into our requirements?
And ultimately, it was left up to the individual disciplines because there could not be a consensus
formed. Each individual department had a very good definition of what they believed critical
thinking was, but it differed quite a bit from what other departments thought were important
(09:52):
skills. So it makes it somewhat challenging for students when they're faced with this wide
variety of definition of critical thinking in terms of being able to develop those skills.
What you're describing, what a perfect example. I mean, it sounds everybody went in good faith,
had this really strong set of ideas going in. But I don't see this as just like, “Oh, there go the
(10:13):
academics again… consensus… they can't reach it.” I think that they were probably running into a
real issue. And again, there's a couple things at work here. I think that there's also this sort of
flawed metaphor that I think it's flawed, and I think it runs through a lot of cognitive aspects
of learning, of the muscle metaphor, of like, “Oh, there's a critical thinking muscle in your brain,
and if you just build it up, then it's gonna be strong for a really long time.” And most things
(10:36):
in the mind actually don't totally work like that. They again, are more situated or contextualized.
But I always say to faculty, I'm like, if we go in, we can sort of say, “That's okay, that's all
right.” And hopefully more institutions will do what it sounds like may have happened more or less
by default in this particular example of like, well, okay, we're gonna go back to our colleges
(10:57):
social sciences and so on, or we're gonna go back to our individual disciplines and try
to operationalize it there. So I look at it, and I see that this is a very rich spectrum.
It's like in business, I've been trying to compile all these different examples, and if
I'm a business major identifying pattern that suggests some kind of financial mismanagement,
don't take the company's word for it. What does it look like? One hears that's critical
(11:19):
thinking there. If I'm in healthcare, being able to look at relevant research findings,
and try to translate that into selecting the right kind of interventions. Engineering has its applied
systems thinking and so on. And so if we come at it the right way, that can be a strength, not a
weakness, I think, and I may be a little unusual in that, but that's what I come away with.
One of the things that you mentioned earlier about knowing like when to do a thing really makes me
(11:44):
think a lot about a move in some disciplines, and I know this was true in design and art a bit is to
focus a bit more on process rather than always on outcomes, so that you can see some of the thinking
and really build in when to do things and when to implement particular techniques to work towards a
particular goal, and slowing things down a bit so that we could emphasize some of those things.
(12:07):
I think that that is also a really important and strong approach. And here, too, I've heard across
lots of different disciplines as well, sitting with faculty and kind of grappling with lots of,
to me, interesting teaching issues. They'll say like, “Okay, yeah, I taught that how of how to
work their problem,” or, like, “if I'm teaching students how to do statistical analyzes in their
(12:29):
research projects,” like, “Okay, I can go through and conduct the steps for a t-test or an ANOVA and
so on.” And I'm not dismissing that, that's important stuff that is important to be able
to do, but I've heard so many variations on “But they can't set up the problem. Once it's set up,
we're good to go.” And that's where, instead of kind of saying, “Ah, our students can't,” well,
what are their opportunities to do that? So let's build in more of those really messy this is more
(12:54):
about watching you do it and having you explore different approaches then you have to check all
the boxes exactly right. There's a time for checking all the boxes exactly right. Again,
not knocking that, but yeah, of putting our students more systematically in those situations,
we might see some real growth there.One of the challenges, I think,
that we're facing is that we're seeing so much misinformation being shared on social media and
(13:19):
AI tools provide increasingly realistic, deep-fake multimedia files with audio and video. And there's
a lot of politicians out there who seem to have little regard for centuries of science,
for example, where they are questioning things like whether or not vaccines are useful,
despite over a century of evidence on this. How can we help students develop their critical
(13:43):
thinking skills they need to navigate this environment where there's so many opportunities
to be misled by all this information. Oh my gosh, yeah. You've really brought
this together, and it's such a powerful way. I mean that really says it, right? And as you're
describing this, absolutely. Talk about a perfect storm. Talk about an absolute perfect storm.
(14:03):
So here we have something that runs off of low effort, and I don't buy into it, like, “Oh, people
these days are cognitively lazy.” I think that's just an overriding feature of human cognition that
I feel like, if there have been changes, that's been a drop in the bucket, but I do have access
to something that is incredibly easy to use, click it open, start scrolling. And so it really biases
(14:30):
us towards low effort. It biases us towards speed too. I mean, Instagram has never kind of been my
thing, but those who really like it, it is amazing how that that's what is sort of designed into that
system, the ability to just whiz through and visual processing is so fast, and that's great
and all, very entertaining. But here I am in that fast, quick, low-effort mode, as you said,
(14:53):
it's now multimedia. That is also what makes it so compelling, is the visuals and the audio and “Oh,
guess what just came along?” Really great ways to fake that. And is anybody else doing it? Is
there that? Can I see anybody else who's cueing me and saying, You know what? Maybe we should
authenticate this video before we post it 48 million times? No one is. And last of all, you've
(15:14):
thrown motivation to maintain our beliefs into the mix. Wow. And so if I am inclined, if I've
always had a bad emotional feeling about vaccines. And I'll just say that, as a psychologist, I kind
of think that that's where some of it comes from. Vaccines are weird. They're not pleasant . They're
okay to think about, and I haven't liked it. I don't want to get one. Last year's flu vaccine
(15:35):
made me sick and made my arm hurt. I mean, it does not take much to knock me straight off
into. To okay, I'm posting it, I'm believing it, and that's where we are. So that's what I see when
I kind of spin that scenario out. So there's lots of things that we can look at as far, and we'll
probably delve into a little bit further, as far as kind of the instructional strategies for
(15:58):
critical thinking. But in this one instance, let's take that perspective of instead of starting from
50,000 feet with critical thinking of, let's have a general definition and then try to figure out a
way to have it filtered down. Let's do bottom up. Let's say, “You know what? If critical thinking
looks like stopping the feed and before I hit post, if it means at least some proportion of the
(16:19):
time I ask about authentication, if I'm applying what I should know about the potential for AI deep
fakes. If that's what it looks like, then that's the practice that students will need.” And I
would throw it out to the ever creative and always innovative faculty across disciplines to say, “All
right, if this is something that is relevant to thinking in your discipline, or something that you
(16:40):
just want to reinforce in your students, how can we have students exercising that and build that
from the bottom up?” And then maybe that 50,000 foot view of critical thinking and that bottom up,
“Okay, I learned to at least process my feed in a more critical way. Now I'm looking at how I read
things. Now I'm really looking at media literacy and so on, and so maybe those things will meet
in the middle.” This is an audio only podcast. I'm making all these gestures here, my little
(17:04):
hands getting closer and closer together, but that's what I see when I look at that issue.
I hear a lot of scaffolding and backwards design in what you're suggesting. Can you
talk a little bit more about some of these practical strategies that we can use in the
classroom to address critical thinking?Yeah, and it's another really tough one to
communicate with faculty maybe why I've been so like, squeamish in some of my work with them,
(17:28):
just because you kind of have to start with a negative message. And that's why I share of like,
“Oh yeah, me and my whole one half of a whole day on critical thinking,” because that's the
thing. you first have to know, this is a big, long journey, and I'll throw out a metaphor.
Some of you know that I spent a lot of time here at the Grand Canyon in Arizona, and I'm always on
forums discussing all the big hikes and so on and what we always see is like, “Oh my gosh, I'm gonna
(17:51):
go to the river and back. I'm gonna hike from the north rim all the way to the south rim. What am I
in for?” It's like you need to know that this is gonna be no walk in the park. Really wrap your
brain around what's involved with this. Worth it? Totally, absolutely, an incredible accomplishment
if you get there. So hopefully I can infuse a little bit of that spirit in it. So first of all,
go in forewarned, don't walk to the bottom in 110 degrees with one bottle of water and figure
(18:15):
it's going to be okay. So with that, yeah. And then reflecting, we always want to do this with
thinking skills. But again, backing up and saying, “all right, what does this look like?” Free
yourself as much as you can from “Oh no, it has to look like somebody else's definition.” Or is this
general or all encompassing enough? Just really get concrete about that, and then focusing on the
why. And that's also been a running theme here. When you talked about process, that to the extent
(18:40):
that things do work, and there are no silver bullets in this arena. We don't have anything,
unfortunately, like that. But to the extent that there are things that we can say, yeah,
that was really effective, it's where the focus is on the why and not the answer. Whether it's just
a multiple-choice question of, “Hey, if you got a question wrong, come and explain it or discuss
it and maybe recover some of your points,” or something very, very big, like we're having a big
(19:04):
debate in class about two approaches to an issue, and one of the big things I'm going to want to
look at is your reasoning. And sometimes this is talked about in cognitive psychology as focusing
on the deep structure of a problem. So all those kind of things that make similar problems link
them together, like all those correlation versus causation problems that I can see as an expert,
but the novice is just like, these are all totally unrelated instances, but when you start to say,
(19:27):
“Okay, but here's the sort of working parts underneath that, the mechanism,” that can move
the needle a little bit. And direct practice. Students almost always need more practice over
time. And the research that's out there that I've seen is showing like more like a couple of months
of practice instead of a couple of hours or a week or two. So if that's what we mean. So practice,
(19:48):
practice. And lastly, that motivation to maintain beliefs. That's a really tricky one. Most of the
folks in the field seem to say you got to take careful and indirect approach. If I come in again
and be like, “How could you think that horoscopes could possibly…?”, you know, if somebody asked me
my sign on a first date, that would be the last date. You come in with that kind of attitude,
well, naturally, people shut down. So going with respect and sometimes asking the questions of,
(20:14):
“why?” Okay, so explain to me how this works. That can be a better thing than just mowing it
down. So be direct in the practice, and be indirect and use a light touch when there's
a possibility of those kind of things that are maintained by motivations.
One of the things that's been happening is that with the increasing use of ChatGPT and other AI
(20:36):
tools, there's a lot of concern that students are using it to offload some of the cognitive
processing that they might normally do while they're reading texts or analyzing materials.
And there was a study by Michael Gerlich at this Swiss Business School earlier in
January that found that students who used AI tools more, tended to do a bit more cognitive
(20:58):
offloading of these tasks, and that there seems to be at least some correlation with the decline
in students’ critical thinking skills when they rely, to a greater extent on AI. What
types of learning activities can help students develop critical thinking skills so that they
can more critically evaluate the types of output they're receiving from generative AI tools?,
(21:19):
Yeah and here too, that could further decompose into different multiple issues. There's critiquing
output you get. And I've heard some really clever assignment ideas where that's part of it is like,
“Okay, go ahead and generate this essay,” like I heard a professor, there was a very cliched topic
that her students were always coming back to, and instead of just forbidding it, she's like,
“Okay, have ChatGPT write that essay, and then you critique it.” And then there's the more process
oriented (21:44):
“Here's the process that I want you to engage in,” for example, reading and evaluating,
say, research studies in psychology, the way we do a number of my classes. And the alternative
is stick it and say, “ask ChatGPT to give you the talking points from the article. So part of it is
going to depend on what exactly do we mean, but I did want to really address too, or come back to,
(22:08):
the study with the correlation between use of AI tools and cognitive offloading, in particular,
and then critical thinking skills. This is a research area that absolutely I hope that we kind
of build out on. But there's some things that I have curiosity about and want to kind of at least
have people pump the brakes a little bit, because this is one of these dynamics that can happen,
(22:30):
maybe a little bit of motivated reasoning here. If I like, “Ah, AI, is this issue in my classes? I'm
really wrestling with it. We're wrestling with it on campus. Oh, AI, kills critical thinking.” And
this is the dynamic that I feel like we've seen a little bit before, for example, with the flap
over laptops and whether those degrade learning. And boy, people ran away with that in ways that
(22:52):
were not supported by the original study, and then even with what the author said about the
study. So a couple of things that we should keep in mind with that research, which, again, I'm
really glad they conducted it, and I hope there's more research like that. This is not a laboratory
study where we're assigning people to groups, of course, and saying, “Here's what happened to
critical thinking abilities,” and this was a multi age study. Was also we want to be cognizant this
(23:14):
wasn't like in a class or educational setting, for what that's worth. There's a lot of self report
here. I always have an issue with that in lots of research on technology. So if you ask people,
how often do you use AI to, I think, it was answer questions or solve problems, which also a couple
different things are covered there in the original study. If you just ask people that, I don't know
that people can track that, that's different than actually measuring that. It's a start. But
(23:37):
know what you're kind of getting there. And I mean cognitive offloading too. This is the thing I've
really noticed this tilt towards interpreting cognitive offloading as a bad thing, and that
I don't think is in the spirit of how it was originally intended. It can sometimes be a good
thing. It's a neutral thing. And everything from using Post-It notes to GPS on your phone to using
(23:58):
an abacus or counting on your fingers are all instances of cognitive offloading. So this is not
something that just started. So there's a couple of things there. So does it predict a decrease in
critical thinking skills? And again, here, this is self report using some good measures adapted
from decent established instruments. But here, again, we weren't watching to see okay, here's
(24:20):
students who are actually grappling with academic problems or making decisions in real life. So I
would absolutely encourage folks get out there and read the study itself and make your own decision,
also it’s very much in the spirit of what we're talking about here. So there is that. But I do
think that a lot of experts who are looking at the development of AI in higher education, they're all
(24:41):
saying some variations of the same things, like, you can't send students home with something and
say, here's a really difficult effortful thing that I know is important for you to engage in.
Maybe you know that, maybe you don't. Here's a trap door that you can evade the entire thing with
AI. You cannot assume that students are going to get what they need to get out of that assignment
(25:02):
anymore. So that is a real question that we had. So let's not run away with the idea that, “Oh,
AI is there, and it's going to destroy critical thinking by its very existence.” But let's really
be thinking through what we're assigning students to do. And I'm hoping we'll hear
more from faculty, sort of in the field, say, “You know what? I was really transparent with students,
and here's how I got them on board with ‘Yeah, we're actually going to read these articles,
(25:26):
even though ChatGPT can summarize them for you.’“ Or here's how I just closed that trap door up and
actually made it work that way. So that's a complicated take on a complicated issue.
The onset of AI in teaching and many other things have led to a lot of faculty overwhelm
around a wide variety of topics, and AI is certainly just one of them. Have you seen
(25:48):
this? And are there things that might help faculty as they navigate new technology,
new policies, many new things in their work?Yeah, and the issue of overwhelm, I mean,
this is one that just kept coming up into my mind again and again, as I've done at least my first
little modest set of things that I've written about and shared with folks. A couple of things,
(26:09):
and some of these are going to, no doubt, rub some people the wrong way, but I'm going to put
them out there anyway, because I think they're a really practical way to go. The first thing is,
I think that a really important starting point is always going to be our own professional practice,
and again, discipline independence, just like critical thinking, it's really hard to say,
“Okay, make sure you evaluate evidence and make that somehow stick across a curriculum.” I've
(26:32):
really pushed against the like, “Well, there's an AI future and AI workplace,” and I don't know what
that is, but I can prepare students for careers that are related to psychology and future lives
that are very well informed by all the wonderful things that psychological science has revealed
for us. That I can do, and I do that every day, and so really kind of narrowing, in a good way,
(26:56):
the focus and the scope here. I hope our leaders are getting this message as well. We'll put out
fewer of these like, here's “AI for everybody,” and more like, “Okay, I want to know how you're
going to be preparing students in social science for uses of AI that are appropriate to them in the
future.” And part of that is getting in there and using the tools yourself. And I will tell you one
(27:17):
really important thing, even more so, believe it or not, than all the other ed tech I've seen come
and go over the years, the level of hype and over claiming that seems to just get by is astonishing,
and it falls apart very rapidly when you sit down with the tool yourself. And I think that's where
we faculty will actually really, really shine. I won't name too many names right now, but like,
(27:40):
“Oh, wow, AI can do that for me.” And that's terrible. I'm not gonna recommend this to my
students. And other ones where I am like,”Oh, okay,” and only a disciplinary expert knows
this. And to give like, an example, I was working with a faculty member in a workshop. She was in
pharmacology. Wow, talk about something that's way out of my field, and so I'm a little odd. And she
talks about “And I made this assignment, and part of it is to critique some AI output.” I'm like,
(28:02):
“Oh, I know where this is going.” She's gonna tell me that “Oh, AI is gonna tell me bizarre
things about medications.” And now she's like, “Oh no, no, no. I use it to generate some scenarios.”
It was something involving, like rare toxins, a really cool assignment. She's like,”Oh no,
no. For this particular tool, it taps into this particular database. I happen to know that that's
really, really strong, and I haven't seen a problem with it.” So that's where we can really
(28:24):
get in there and start to craft things. So you can start it at that level. And another really,
probably unpopular piece of advice, but one that I think is important at this juncture, is, okay,
we have these big questions of the ethics of AI use: How did this stuff get built? Who's going
to be regulating it? I think that's incredibly important. And I would imagine that pretty much
(28:45):
everybody listening to this podcast would agree with that, however, going and saying like, “I have
to figure that out myself before I start writing an AI policy for my syllabus” or trying some
of the tools, so that I can talk intelligently to students about the pros and cons. That's an
issue. So I would say those kind of have to go on parallel tracks if we're going to make some
(29:06):
immediate progress. And I feel like, and I felt of myself initially, of like, “oh my gosh, all these
big issues.” Well, that's not my job to solve. It's great if I can contribute to that and support
solutions and approaches that I think are ethical, but I'm not the one who's going to figure that out
all by myself. So here's some positive things I can do with my students in the meantime.
(29:28):
One of the things that Michael Gerlich suggested in that article is that in case there is some
issue with a decline in critical thinking skills, we probably should assume that students are going
to be experimenting with AI tools, and he suggests that maybe we should focus a bit more on training
students to apply critical thinking to the output of AI, because if they're going to use it anyway,
(29:50):
we might as well use it as a tool to help develop these skills that we'd like to improve. Does that
seem like a reasonable strategy at this point, with a lot of experimentation going
on by a lot of faculty right now in trying to integrate this into their practices?
Yeah, and I think higher education faculty are in a great position to do that,
(30:11):
because from industry, we get this kind of kind of cutesy pooh, “Oh, it's hallucinating.” It's like,
no, it's wrong. It's inaccurate. If this was a book, you would roast it in an Amazon review if
it was just wrong. So we can get away from that. We can also not fall into the “Well,
use it, but don't use it, trust it, but maybe not so much.” And that starts to remind me of like all
(30:34):
the terms of service. Like, well, read the terms of service when you click into something, but we
know you do didn’t really. I have no patience for that. So really being able to nail down the
limitations, and particularly, again, situated in these disciplines where we are incredibly skilled
experts at knowing where it is likely to trip up and what sorts of mistakes are a problem and which
(30:58):
are less of a problem and guiding students into that. So if we can design those ideas,
and yeah, sometimes we can take it in another direction and use the power of the tools for
our own purposes. And that's also where I get a little nervous about the like, blanket, “Oh,
AI will kill your critical thinking.” Well, I wouldn't want a faculty member to say, “Well, then
I'm not going to touch it, because if there's any way that it's sort of contaminated my assignment,
(31:20):
that's bad.” I don't think we'd consciously think that, but it can start to imply that.
So I'm looking back when I used to teach research methods, one of my assignments I was very proud
of was their final exam was I wrote beginning to end a fake study with fake data that was full of
all kinds of problems that spanned everything from study ethics to degrees of freedom in the
(31:41):
statistics module. So I had maybe two practices, and then, for their final example, they went in
and spotted the problems. They didn't have to spot all of them, but they had to justify and
say what's wrong with it. And students would go wild over this, even though they know that I made
this thing up. They would go crazy with all these issues. And it was perfect, if not perfect, I
loved it, because that's what we want students to come away with, but the labor that went into that,
(32:04):
and I find myself like reusing the same scenarios across semesters. Okay, we're gonna have this one
be practice, and it was just so hard. And today, every student in the class could have a new one
every day, and so it could do that work. I come in and here we go. Here's these rich scenarios
with students grappling with the messy problems, focusing on the why, but in the middle of it has
(32:27):
to be a faculty expert to make all that work. There's just some great points and great fun use
of AI for sure. I know that I've used it for scenarios and other things in my teaching and
in other contexts, as well as we work with leadership on our campuses, thinking about
policies and other ways to approach AI and making sure that we're not losing sight of critical
(32:49):
thinking and any kind of deep thinking what kinds of recommendations should we be making?
Ah, for leadership, and I love it, the receptivity to new ideas that's definitely going to be key?
Well, I'm gonna probably rehash a few of those points of be very careful about convening, like,
say, a big, expensive task force to find the one consensus, to converge on the one solution. There
(33:14):
are probably situations where that is absolutely what needs to happen. But one of the things that
I think we've seen, too, you don't really like to talk about it that much, I don't think, but
higher education is spectacularly unprepared to deal with just the pace of change here. And yeah,
partly, industry is sort of driving the bus and the pace of change, and we don't always have to
accept their timelines. But the fact of the matter is, is that you go to sleep, you wake up, and
(33:38):
there's a new version of something, and so I was really surprised. I shouldn't have been, but I was
pretty surprised by, like, how it unfolded in my local context, and just among what I could see of,
like, “Okay, here's a little PowerPoint that I can take around,” and then two weeks later, I'm like,
“I can throw that away?” I'm not used to that. So, yeah, higher education, our deliberative nature
(33:58):
is not a bad thing, necessarily. That's good. It's kind of like that's what you pay us for,
is to take the long view that's grounded in a lot of evidence. But in this o ne instance,
it makes it really, really hard for us not to feel like we're flat footed. So beware of those sorts
of things, and it's demoralizing for faculty who well meaning, they're interested in this,
(34:20):
they're excited about it, they want to contribute, and they spend four months doing something they
can't see how it played out. So my gut tells me that going for that college level is going to be
important. So maybe I don't want to chase down every single department or every single course
to say, “How is AI going to be involved here, especially around critical thinking issues,”
(34:42):
but saying, “Okay, the School of Engineering, or the School of Earth Sciences, the School
of Social Sciences, Literature and Humanities, however, it's sort of organized.” I think that
that's a pretty powerful sort of organizational approach there. And that's something where too,
it's enough to where we can get some multiple voices in there and get some buy in, but it's
(35:04):
specific enough to where I'm not trying to align what I'm doing in psychology with something that
people are doing in art history, with what forestry is doing, which, again, I think is
really gonna neutralize us if we try to take that approach. And also dividing it up. The last thing
that makes this such a hard thing for us is that it's just so inherently multifaceted, much more so
than, for example, even when learning management systems came on the scene and we're like, “Okay,
(35:27):
do we require everybody use these? Which one do we want? How do we pick? How do we train
people?” At the very bare minimum, we got to kind of separate out academic dishonesty is over here,
and it's got its own little set of policies, and let's set a team working on that, and positive
uses and preparing students for that workplace of the future, and so on. These are separate
(35:49):
issues over there. So yeah, be prepared to go in and just say, “Okay, two separate things,
we're gonna deal with them two separate ways.” I can't imagine trying to sit at home on a Saturday
and craft some long email that is somehow going to bring clarity to all the different aspects
of AI. And I'm wondering if there's any sort of resonance or spark of recognition there,
(36:10):
as leaders, that you might see.Do you have any other suggestions
on how we can help students develop their critical thinking skills.
One little theme that pops up from time to time in all the great stuff that's been written about
critical thinking, and one that I think might be worth its own special mention, is an element
of what you could call intellectual humility, or even, I guess, the flip side of that too,
(36:34):
is curiosity. Think of me with kind of cutting down some student over horoscopes in my class,
even though I find the basis for that in psychological sciences. If I do not care
at all about why a student believes that, or what it might add to their lives to believe in,
I mean, maybe it's just something fun that is like a little Rorschach test that I wake up with every
(36:55):
morning. If I have no interest in that, and then I'm also not demonstrating critical reflection,
nor am I setting a good stage for a discussion that foregrounds critical thinking. So yeah,
and that is our big challenge to faculty with our incredibly developed intellectual abilities
to balance that. Yeah, evidence is important, and some facts can be right or wrong. There's
(37:18):
different ways of interpreting them, but here's some areas where they have to support
the analysis, balancing that with all right and I could be wrong. I could have a cognitive bias
of my own that I'm not as aware of right now. I kind of have to be on the lookout for always
checking myself as I'm scrolling through that feed or going through those reviews for the thing
that I really do want to buy, no matter how many one star reviews that the company suppressed. I
(37:42):
need myself to get in the habit of doing that, and then I can model that for my students, and again,
set the stage for more productive discussion. So I'm hoping that more research also gets into that
too. What is that wonderful, magical balance of really valuing evidence and reasoning and
then realizing and I could be wrong? Well, we always appreciate, Michelle,
conversations with you. I know I always leave thinking a little more deeply
(38:05):
about whatever we've talked about, but we always wrap up by asking, what's next?
Oh, thank you, and you all as well. Even just kind of looking forward to this conversation has been
such a catalyst to revisit topics and say, “Yeah, this is something that I really want to be doing
more in.” And so that's going to be a bit of a what's next. I have a sabbatical year right now,
(38:25):
and I'm kind of trying to do what one ideally does, which is say, “Okay, what do I refocus on?”
And so I'm really at that very early stage of continuing to design ways of studying,
interventions that I could then share with faculty, and just getting an ever needed sense
of the lay of the land in this area of critical thinking. So you can definitely look for more on
that. And in fact, for folks who might be considering attending an in-person event,
(38:50):
I am going to be speaking at a couple of upcoming events where I know that I'm going to be talking
about this particular thing, again, Rocky Mountain Psychological Association for the
psychologists out there, my people. If you want to come out, I think it's in Denver this year, I have
a featured session. So that's coming up. I'll also be speaking at the Teaching Professor Conference,
(39:10):
which is in, I believe, Washington, DC, over the summer. Some of your listeners may be regulars,
or maybe thinking about that. So that's a big “what's next” item? I continue to work on this
big writing project on attention. It's still at this kind of embryonic, or if not embryonic,
partially formed stage, where I'm still working on it. But folks can watch this space for more on
(39:31):
that, and that is actually related. I will tease a little bit, that attention, what do you know,
gets involved in critical thinking, because it is involved in pretty much all other interesting
cognitive processes that take place in the mind. So that's my big what's next?
Well, thank you. It's always great talking to you, and again, as Rebecca said,
it always spurs some new ideas and some new thoughts on these topics. So, thank you.
(39:53):
Thank you. You as well.I look forward to your upcoming work.
If you've enjoyed this podcast, please subscribe and leave a review on iTunes
or your favorite podcast service. To continue the conversation, join us on
our Tea for Teaching Facebook page. You can find show notes, transcripts and
(40:16):
other materials on teaforteaching.com. Music by Michael Gary Brewer.
Throughout human history, we have relied on technology to
make our work easier. In this episode, we discuss how to foster students’ critical
thinking skills in the age of AI. Thanks for joining us for Tea for Teaching,
an informal discussion of innovative and effective practices in teaching and learning.
(00:23):
This podcast series is hosted by John Kane, an economist...
...and Rebecca Mushtare, a graphic designer......and features guests doing important research
and advocacy work to make higher education more inclusive and supportive of all learners.
(00:46):
Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and
President’s Distinguished Teaching Fellow at Northern Arizona University. She is the
author of Minds Online (00:55):
Teaching Effectively with Technology, Remembering and Forgetting in the Age
of Technology (01:00):
Teaching, Learning, and the Science of Memory in a Wired World and A Teacher’s Guide
to Learning Students’ Names (01:06):
Why You Should, Why It’s Hard, How You Can. That list keeps growing
since we first interviewed you. And Michelle is also a frequent contributor of articles on
teaching and learning in higher education to a variety of publications including The
Chronicle of Higher Ed. Welcome back, Michelle.Oh, thank you. And yes, it's really satisfying to
(01:28):
see these books making their way out into the world. And if it weren't for engaged faculty
audiences, that wouldn't happen. So I guess I'll put that gratitude out to the audience as well.
Today's teas are:... Michelle, are you drinking tea?
I sure am, I've got a nice green tea from Hong Kong.
Nice. How about you, John?I am drinking a pure peppermint tea today.
(01:50):
So far, we've got a good variety, and I'll follow up with some black
tea. I have a Lady Grey today.Very good. So we've invited you
here to discuss the development of critical thinking skills. And let's begin by talking
about what critical thinking is and any misconceptions or missed opportunities that
you've noted in your work around this issue.So this is something that I've really started
(02:13):
engaging with myself back when I was writing the book Minds Online. And those who are familiar with
its structure know that I try to use a really, really simple little heuristic for divvying up
the cognitive demands and some of the relevant cognitive research and processes that are involved
in the type of learning that we address in higher education, and one of those is thinking, and even
(02:35):
within that is this topic of critical thinking skills. And so I had just kind of caught myself,
year after year, monitoring what was going on in the development of research around this,
of course, thinking about it in my own teaching, but also finding myself being really reticent when
I was working with faculty audiences to say, well, this is a really kind of a, if not totally unique,
(02:57):
a pretty specialized subset within thinking skills more generally, and then the bigger,
broader umbrella of cognition in general. So it's almost like the first missed opportunity is my
opportunity to engage with faculty around this. So I love it, it's this really deep topic. But
that's one reason why it's hard to just say, oh, and in five minutes, sandwiched in between five
(03:18):
other things, we're gonna address this. And so that's what I've kind of been reexamining myself.
So when we talk about what is critical thinking, that is actually the first really big challenge
and, or sometimes misconception around the topic. And no surprise, there’s differences of opinion
in defining things, and of course, really getting into that challenge of how do we define a thing
(03:40):
and pin it down, and let's all kind of debate the definition. But critical thinking here really is a
special case in that there have been quite a few attempts to make these very general sort of top
level, kind of elaborations of what it is. Think about words like evaluate or analyze, reflect and
so on. But I don't think we have converged on one of these as really the best across disciplines,
(04:06):
and I kind of fall in with some other thinkers in the sphere who've said maybe a discipline
independent, context agnostic definition doesn't exist, because maybe the phenomenon itself doesn't
exist. So I've, myself, really been drawn towards the more, I guess you'd call them,
situated or contextualized definitions of what it is. If you drill down a little bit further,
(04:29):
you get, for example, Diane Halpern, psychologist, this towering figure in our field. So naturally,
I would gravitate towards what she says, but she says things like evaluating and analyzing
evidence in the service of making decisions, so thought processes that lead to a good outcome. So
that's a little bit more practical, but it still doesn't really make it so a given faculty member
(04:52):
teaching in a given course can say this is what it looks like when students came in without a
critical thinking skill and they leave with it. And as with anything, if we don't have that,
then no surprise that we're maybe not going to get there, or not get there as completely as we
had hoped to. And so that's one kind of set of, if not misconceptions, challenges or things that
(05:14):
people often don't realize when they wade into this issue of teaching critical thinking. And
it follows from that, unfortunately, that we can have the misconception, that we can describe it
to students and they'll get it. I know, for me, I would be embarrassed to say how many years I
taught, for example, Introduction to Psychology, which we pride ourselves on being like this is the
critical thinking class. And even if you never take another psychology course, you're going
(05:36):
to come away with these great reasoning skills. And gosh darn it, I had a whole half of one class
period that we discussed it, maybe, and it was on the study guide somewhere. So we think, “Oh,
okay, I'll just tell students this is important.” Say, “think critically.” And of course, naturally,
they don't know what it is either. And I can really see, in my mind, at least,
(05:57):
how students could come away with the idea that critical thinking means just sort of criticizing
everything indiscriminately. It means nobody knows anything, a total relativism, that no, that's not
what we want, or in its worst forms, it can read too as just like, agree with me. Like I can go and
say, Look, “I'm a psychologist, and I'm pretty critical of the idea of horoscopes.” And if you
(06:17):
think that your horoscope is going to predict your future, you're not engaging in critical thinking,
you're wrong. So that's what we want to get away from. And where I think that it can be this really
great fertile ground for faculty if they step back and say, “Okay, what does this really mean at that
more actionable level,” and then start to align instruction to that, because it's gonna take a lot
more than half a day, I'll tell you that much. Seems like you're suggesting that it would be
(06:41):
helpful for students to be able to identify specific behaviors or specific practices that
we would identify as critical thinking. But if they know what the beginning, middle and end of
an action is, they have a better idea of what that looks like and what that feels like.
Yeah, absolutely. And that's also a pretty psychological way of coming at it as well. Like,
(07:01):
okay, of course, this is something that happens in your mind, out of view, between your ears,
but I want to see what happens on the page or the decisions that you do make in the world that align
with what I would consider to be evaluating and analyzing evidence in a powerful way. And yeah,
when we get into a little bit more of the challenges, I mean what I've laid out for faculty,
(07:22):
and again, really drawing on a lot of research that's been out there for a while, even beyond
that defining issue, which, yeah, that's going to challenge faculty. It's going to get in the way of
designing the activities. It's going to challenge students. They don't know what they're trying to
aim for to meet this thing that's on the syllabus. It is also what I would almost call a cueing
problem, or someone's just kind of off the cuff, called a “when to” versus a “how to” problem with
(07:45):
a lot of the skills that we are helping students develop. It's “Okay, here is how you break this
down. Here is how you conduct the statistical test. Here is how you read an APA style
research article.” But this is something that's like, “Okay, here's how to find, for example,
issues with correlation versus causation in the interpretation of a study.” Great,
(08:06):
we can do that. But are you going to know to do that when you're writing your paper? And even
a more far transfer of that as you're scrolling through whatever social media feed of choice is,
and you get the like, well, I drank a cup of lemon juice a day, and now I'm healthy as a horse. It's
sort of the same problem. Am I going to have that sort of inclination or disposition to say,
(08:26):
“Oh, right, this is when I need to do this.” So there's that. A lot of folks have talked about
effort as well, part of this running theme of cognitive miserliness, that our brains and our
minds really seem to run in high-efficiency mode, where we save time and save energy over
getting to a more idealized solution. And also, as a lot of folks have talked about,
a motivation to maintain beliefs, whether it's a very profound thing, like, “I don't
(08:49):
want to critique this belief because it's really foundational for intellectually or emotionally
or spiritually,” or even just dumb things like, “Oh wow, I really want to buy this thing I just
saw an ad for. And maybe I just won't take a look at all those negative reviews,” or “I'll assume
that the reviews that all have these wonderful five stars are perfectly reputable.” So all of
(09:10):
those things form quite a steep set of challenges for students and for faculty alike, right?
Very much so. And what makes it more complex, and I think you alluded to this, is that each
discipline has its own definition of what critical thinking skills are. I remember I spent about
10 years on faculty assembly one semester where they were coming up with a new Gen Ed curriculum,
(09:32):
and one of the main issues was, how are we going to build critical thinking into our requirements?
And ultimately, it was left up to the individual disciplines because there could not be a consensus
formed. Each individual department had a very good definition of what they believed critical
thinking was, but it differed quite a bit from what other departments thought were important
(09:52):
skills. So it makes it somewhat challenging for students when they're faced with this wide
variety of definition of critical thinking in terms of being able to develop those skills.
What you're describing, what a perfect example. I mean, it sounds everybody went in good faith,
had this really strong set of ideas going in. But I don't see this as just like, “Oh, there go the
(10:13):
academics again… consensus… they can't reach it.” I think that they were probably running into a
real issue. And again, there's a couple things at work here. I think that there's also this sort of
flawed metaphor that I think it's flawed, and I think it runs through a lot of cognitive aspects
of learning, of the muscle metaphor, of like, “Oh, there's a critical thinking muscle in your brain,
and if you just build it up, then it's gonna be strong for a really long time.” And most things
(10:36):
in the mind actually don't totally work like that. They again, are more situated or contextualized.
But I always say to faculty, I'm like, if we go in, we can sort of say, “That's okay, that's all
right.” And hopefully more institutions will do what it sounds like may have happened more or less
by default in this particular example of like, well, okay, we're gonna go back to our colleges
(10:57):
social sciences and so on, or we're gonna go back to our individual disciplines and try
to operationalize it there. So I look at it, and I see that this is a very rich spectrum.
It's like in business, I've been trying to compile all these different examples, and if
I'm a business major identifying pattern that suggests some kind of financial mismanagement,
don't take the company's word for it. What does it look like? One hears that's critical
(11:19):
thinking there. If I'm in healthcare, being able to look at relevant research findings,
and try to translate that into selecting the right kind of interventions. Engineering has its applied
systems thinking and so on. And so if we come at it the right way, that can be a strength, not a
weakness, I think, and I may be a little unusual in that, but that's what I come away with.
One of the things that you mentioned earlier about knowing like when to do a thing really makes me
(11:44):
think a lot about a move in some disciplines, and I know this was true in design and art a bit is to
focus a bit more on process rather than always on outcomes, so that you can see some of the thinking
and really build in when to do things and when to implement particular techniques to work towards a
particular goal, and slowing things down a bit so that we could emphasize some of those things.
(12:07):
I think that that is also a really important and strong approach. And here, too, I've heard across
lots of different disciplines as well, sitting with faculty and kind of grappling with lots of,
to me, interesting teaching issues. They'll say like, “Okay, yeah, I taught that how of how to
work their problem,” or, like, “if I'm teaching students how to do statistical analyzes in their
(12:29):
research projects,” like, “Okay, I can go through and conduct the steps for a t-test or an ANOVA and
so on.” And I'm not dismissing that, that's important stuff that is important to be able
to do, but I've heard so many variations on “But they can't set up the problem. Once it's set up,
we're good to go.” And that's where, instead of kind of saying, “Ah, our students can't,” well,
what are their opportunities to do that? So let's build in more of those really messy this is more
(12:54):
about watching you do it and having you explore different approaches then you have to check all
the boxes exactly right. There's a time for checking all the boxes exactly right. Again,
not knocking that, but yeah, of putting our students more systematically in those situations,
we might see some real growth there.One of the challenges, I think,
that we're facing is that we're seeing so much misinformation being shared on social media and
(13:19):
AI tools provide increasingly realistic, deep-fake multimedia files with audio and video. And there's
a lot of politicians out there who seem to have little regard for centuries of science,
for example, where they are questioning things like whether or not vaccines are useful,
despite over a century of evidence on this. How can we help students develop their critical
(13:43):
thinking skills they need to navigate this environment where there's so many opportunities
to be misled by all this information. Oh my gosh, yeah. You've really brought
this together, and it's such a powerful way. I mean that really says it, right? And as you're
describing this, absolutely. Talk about a perfect storm. Talk about an absolute perfect storm.
(14:03):
So here we have something that runs off of low effort, and I don't buy into it, like, “Oh, people
these days are cognitively lazy.” I think that's just an overriding feature of human cognition that
I feel like, if there have been changes, that's been a drop in the bucket, but I do have access
to something that is incredibly easy to use, click it open, start scrolling. And so it really biases
(14:30):
us towards low effort. It biases us towards speed too. I mean, Instagram has never kind of been my
thing, but those who really like it, it is amazing how that that's what is sort of designed into that
system, the ability to just whiz through and visual processing is so fast, and that's great
and all, very entertaining. But here I am in that fast, quick, low-effort mode, as you said,
(14:53):
it's now multimedia. That is also what makes it so compelling, is the visuals and the audio and “Oh,
guess what just came along?” Really great ways to fake that. And is anybody else doing it? Is
there that? Can I see anybody else who's cueing me and saying, You know what? Maybe we should
authenticate this video before we post it 48 million times? No one is. And last of all, you've
(15:14):
thrown motivation to maintain our beliefs into the mix. Wow. And so if I am inclined, if I've
always had a bad emotional feeling about vaccines. And I'll just say that, as a psychologist, I kind
of think that that's where some of it comes from. Vaccines are weird. They're not pleasant . They're
okay to think about, and I haven't liked it. I don't want to get one. Last year's flu vaccine
(15:35):
made me sick and made my arm hurt. I mean, it does not take much to knock me straight off
into. To okay, I'm posting it, I'm believing it, and that's where we are. So that's what I see when
I kind of spin that scenario out. So there's lots of things that we can look at as far, and we'll
probably delve into a little bit further, as far as kind of the instructional strategies for
(15:58):
critical thinking. But in this one instance, let's take that perspective of instead of starting from
50,000 feet with critical thinking of, let's have a general definition and then try to figure out a
way to have it filtered down. Let's do bottom up. Let's say, “You know what? If critical thinking
looks like stopping the feed and before I hit post, if it means at least some proportion of the
(16:19):
time I ask about authentication, if I'm applying what I should know about the potential for AI deep
fakes. If that's what it looks like, then that's the practice that students will need.” And I
would throw it out to the ever creative and always innovative faculty across disciplines to say, “All
right, if this is something that is relevant to thinking in your discipline, or something that you
(16:40):
just want to reinforce in your students, how can we have students exercising that and build that
from the bottom up?” And then maybe that 50,000 foot view of critical thinking and that bottom up,
“Okay, I learned to at least process my feed in a more critical way. Now I'm looking at how I read
things. Now I'm really looking at media literacy and so on, and so maybe those things will meet
in the middle.” This is an audio only podcast. I'm making all these gestures here, my little
(17:04):
hands getting closer and closer together, but that's what I see when I look at that issue.
I hear a lot of scaffolding and backwards design in what you're suggesting. Can you
talk a little bit more about some of these practical strategies that we can use in the
classroom to address critical thinking?Yeah, and it's another really tough one to
communicate with faculty maybe why I've been so like, squeamish in some of my work with them,
(17:28):
just because you kind of have to start with a negative message. And that's why I share of like,
“Oh yeah, me and my whole one half of a whole day on critical thinking,” because that's the
thing. you first have to know, this is a big, long journey, and I'll throw out a metaphor.
Some of you know that I spent a lot of time here at the Grand Canyon in Arizona, and I'm always on
forums discussing all the big hikes and so on and what we always see is like, “Oh my gosh, I'm gonna
(17:51):
go to the river and back. I'm gonna hike from the north rim all the way to the south rim. What am I
in for?” It's like you need to know that this is gonna be no walk in the park. Really wrap your
brain around what's involved with this. Worth it? Totally, absolutely, an incredible accomplishment
if you get there. So hopefully I can infuse a little bit of that spirit in it. So first of all,
go in forewarned, don't walk to the bottom in 110 degrees with one bottle of water and figure
(18:15):
it's going to be okay. So with that, yeah. And then reflecting, we always want to do this with
thinking skills. But again, backing up and saying, “all right, what does this look like?” Free
yourself as much as you can from “Oh no, it has to look like somebody else's definition.” Or is this
general or all encompassing enough? Just really get concrete about that, and then focusing on the
why. And that's also been a running theme here. When you talked about process, that to the extent
(18:40):
that things do work, and there are no silver bullets in this arena. We don't have anything,
unfortunately, like that. But to the extent that there are things that we can say, yeah,
that was really effective, it's where the focus is on the why and not the answer. Whether it's just
a multiple-choice question of, “Hey, if you got a question wrong, come and explain it or discuss
it and maybe recover some of your points,” or something very, very big, like we're having a big
(19:04):
debate in class about two approaches to an issue, and one of the big things I'm going to want to
look at is your reasoning. And sometimes this is talked about in cognitive psychology as focusing
on the deep structure of a problem. So all those kind of things that make similar problems link
them together, like all those correlation versus causation problems that I can see as an expert,
but the novice is just like, these are all totally unrelated instances, but when you start to say,
(19:27):
“Okay, but here's the sort of working parts underneath that, the mechanism,” that can move
the needle a little bit. And direct practice. Students almost always need more practice over
time. And the research that's out there that I've seen is showing like more like a couple of months
of practice instead of a couple of hours or a week or two. So if that's what we mean. So practice,
(19:48):
practice. And lastly, that motivation to maintain beliefs. That's a really tricky one. Most of the
folks in the field seem to say you got to take careful and indirect approach. If I come in again
and be like, “How could you think that horoscopes could possibly…?”, you know, if somebody asked me
my sign on a first date, that would be the last date. You come in with that kind of attitude,
well, naturally, people shut down. So going with respect and sometimes asking the questions of,
(20:14):
“why?” Okay, so explain to me how this works. That can be a better thing than just mowing it
down. So be direct in the practice, and be indirect and use a light touch when there's
a possibility of those kind of things that are maintained by motivations.
One of the things that's been happening is that with the increasing use of ChatGPT and other AI
(20:36):
tools, there's a lot of concern that students are using it to offload some of the cognitive
processing that they might normally do while they're reading texts or analyzing materials.
And there was a study by Michael Gerlich at this Swiss Business School earlier in
January that found that students who used AI tools more, tended to do a bit more cognitive
(20:58):
offloading of these tasks, and that there seems to be at least some correlation with the decline
in students’ critical thinking skills when they rely, to a greater extent on AI. What
types of learning activities can help students develop critical thinking skills so that they
can more critically evaluate the types of output they're receiving from generative AI tools?,
(21:19):
Yeah and here too, that could further decompose into different multiple issues. There's critiquing
output you get. And I've heard some really clever assignment ideas where that's part of it is like,
“Okay, go ahead and generate this essay,” like I heard a professor, there was a very cliched topic
that her students were always coming back to, and instead of just forbidding it, she's like,
“Okay, have ChatGPT write that essay, and then you critique it.” And then there's the more process
oriented (21:44):
“Here's the process that I want you to engage in,” for example, reading and evaluating,
say, research studies in psychology, the way we do a number of my classes. And the alternative
is stick it and say, “ask ChatGPT to give you the talking points from the article. So part of it is
going to depend on what exactly do we mean, but I did want to really address too, or come back to,
(22:08):
the study with the correlation between use of AI tools and cognitive offloading, in particular,
and then critical thinking skills. This is a research area that absolutely I hope that we kind
of build out on. But there's some things that I have curiosity about and want to kind of at least
have people pump the brakes a little bit, because this is one of these dynamics that can happen,
(22:30):
maybe a little bit of motivated reasoning here. If I like, “Ah, AI, is this issue in my classes? I'm
really wrestling with it. We're wrestling with it on campus. Oh, AI, kills critical thinking.” And
this is the dynamic that I feel like we've seen a little bit before, for example, with the flap
over laptops and whether those degrade learning. And boy, people ran away with that in ways that
(22:52):
were not supported by the original study, and then even with what the author said about the
study. So a couple of things that we should keep in mind with that research, which, again, I'm
really glad they conducted it, and I hope there's more research like that. This is not a laboratory
study where we're assigning people to groups, of course, and saying, “Here's what happened to
critical thinking abilities,” and this was a multi age study. Was also we want to be cognizant this
(23:14):
wasn't like in a class or educational setting, for what that's worth. There's a lot of self report
here. I always have an issue with that in lots of research on technology. So if you ask people,
how often do you use AI to, I think, it was answer questions or solve problems, which also a couple
different things are covered there in the original study. If you just ask people that, I don't know
that people can track that, that's different than actually measuring that. It's a start. But
(23:37):
know what you're kind of getting there. And I mean cognitive offloading too. This is the thing I've
really noticed this tilt towards interpreting cognitive offloading as a bad thing, and that
I don't think is in the spirit of how it was originally intended. It can sometimes be a good
thing. It's a neutral thing. And everything from using Post-It notes to GPS on your phone to using
(23:58):
an abacus or counting on your fingers are all instances of cognitive offloading. So this is not
something that just started. So there's a couple of things there. So does it predict a decrease in
critical thinking skills? And again, here, this is self report using some good measures adapted
from decent established instruments. But here, again, we weren't watching to see okay, here's
(24:20):
students who are actually grappling with academic problems or making decisions in real life. So I
would absolutely encourage folks get out there and read the study itself and make your own decision,
also it’s very much in the spirit of what we're talking about here. So there is that. But I do
think that a lot of experts who are looking at the development of AI in higher education, they're all
(24:41):
saying some variations of the same things, like, you can't send students home with something and
say, here's a really difficult effortful thing that I know is important for you to engage in.
Maybe you know that, maybe you don't. Here's a trap door that you can evade the entire thing with
AI. You cannot assume that students are going to get what they need to get out of that assignment
(25:02):
anymore. So that is a real question that we had. So let's not run away with the idea that, “Oh,
AI is there, and it's going to destroy critical thinking by its very existence.” But let's really
be thinking through what we're assigning students to do. And I'm hoping we'll hear
more from faculty, sort of in the field, say, “You know what? I was really transparent with students,
and here's how I got them on board with ‘Yeah, we're actually going to read these articles,
(25:26):
even though ChatGPT can summarize them for you.’“ Or here's how I just closed that trap door up and
actually made it work that way. So that's a complicated take on a complicated issue.
The onset of AI in teaching and many other things have led to a lot of faculty overwhelm
around a wide variety of topics, and AI is certainly just one of them. Have you seen
(25:48):
this? And are there things that might help faculty as they navigate new technology,
new policies, many new things in their work?Yeah, and the issue of overwhelm, I mean,
this is one that just kept coming up into my mind again and again, as I've done at least my first
little modest set of things that I've written about and shared with folks. A couple of things,
(26:09):
and some of these are going to, no doubt, rub some people the wrong way, but I'm going to put
them out there anyway, because I think they're a really practical way to go. The first thing is,
I think that a really important starting point is always going to be our own professional practice,
and again, discipline independence, just like critical thinking, it's really hard to say,
“Okay, make sure you evaluate evidence and make that somehow stick across a curriculum.” I've
(26:32):
really pushed against the like, “Well, there's an AI future and AI workplace,” and I don't know what
that is, but I can prepare students for careers that are related to psychology and future lives
that are very well informed by all the wonderful things that psychological science has revealed
for us. That I can do, and I do that every day, and so really kind of narrowing, in a good way,
(26:56):
the focus and the scope here. I hope our leaders are getting this message as well. We'll put out
fewer of these like, here's “AI for everybody,” and more like, “Okay, I want to know how you're
going to be preparing students in social science for uses of AI that are appropriate to them in the
future.” And part of that is getting in there and using the tools yourself. And I will tell you one
(27:17):
really important thing, even more so, believe it or not, than all the other ed tech I've seen come
and go over the years, the level of hype and over claiming that seems to just get by is astonishing,
and it falls apart very rapidly when you sit down with the tool yourself. And I think that's where
we faculty will actually really, really shine. I won't name too many names right now, but like,
(27:40):
“Oh, wow, AI can do that for me.” And that's terrible. I'm not gonna recommend this to my
students. And other ones where I am like,”Oh, okay,” and only a disciplinary expert knows
this. And to give like, an example, I was working with a faculty member in a workshop. She was in
pharmacology. Wow, talk about something that's way out of my field, and so I'm a little odd. And she
talks about “And I made this assignment, and part of it is to critique some AI output.” I'm like,
(28:02):
“Oh, I know where this is going.” She's gonna tell me that “Oh, AI is gonna tell me bizarre
things about medications.” And now she's like, “Oh no, no, no. I use it to generate some scenarios.”
It was something involving, like rare toxins, a really cool assignment. She's like,”Oh no,
no. For this particular tool, it taps into this particular database. I happen to know that that's
really, really strong, and I haven't seen a problem with it.” So that's where we can really
(28:24):
get in there and start to craft things. So you can start it at that level. And another really,
probably unpopular piece of advice, but one that I think is important at this juncture, is, okay,
we have these big questions of the ethics of AI use: How did this stuff get built? Who's going
to be regulating it? I think that's incredibly important. And I would imagine that pretty much
(28:45):
everybody listening to this podcast would agree with that, however, going and saying like, “I have
to figure that out myself before I start writing an AI policy for my syllabus” or trying some
of the tools, so that I can talk intelligently to students about the pros and cons. That's an
issue. So I would say those kind of have to go on parallel tracks if we're going to make some
(29:06):
immediate progress. And I feel like, and I felt of myself initially, of like, “oh my gosh, all these
big issues.” Well, that's not my job to solve. It's great if I can contribute to that and support
solutions and approaches that I think are ethical, but I'm not the one who's going to figure that out
all by myself. So here's some positive things I can do with my students in the meantime.
(29:28):
One of the things that Michael Gerlich suggested in that article is that in case there is some
issue with a decline in critical thinking skills, we probably should assume that students are going
to be experimenting with AI tools, and he suggests that maybe we should focus a bit more on training
students to apply critical thinking to the output of AI, because if they're going to use it anyway,
(29:50):
we might as well use it as a tool to help develop these skills that we'd like to improve. Does that
seem like a reasonable strategy at this point, with a lot of experimentation going
on by a lot of faculty right now in trying to integrate this into their practices?
Yeah, and I think higher education faculty are in a great position to do that,
(30:11):
because from industry, we get this kind of kind of cutesy pooh, “Oh, it's hallucinating.” It's like,
no, it's wrong. It's inaccurate. If this was a book, you would roast it in an Amazon review if
it was just wrong. So we can get away from that. We can also not fall into the “Well,
use it, but don't use it, trust it, but maybe not so much.” And that starts to remind me of like all
(30:34):
the terms of service. Like, well, read the terms of service when you click into something, but we
know you do didn’t really. I have no patience for that. So really being able to nail down the
limitations, and particularly, again, situated in these disciplines where we are incredibly skilled
experts at knowing where it is likely to trip up and what sorts of mistakes are a problem and which
(30:58):
are less of a problem and guiding students into that. So if we can design those ideas,
and yeah, sometimes we can take it in another direction and use the power of the tools for
our own purposes. And that's also where I get a little nervous about the like, blanket, “Oh,
AI will kill your critical thinking.” Well, I wouldn't want a faculty member to say, “Well, then
I'm not going to touch it, because if there's any way that it's sort of contaminated my assignment,
(31:20):
that's bad.” I don't think we'd consciously think that, but it can start to imply that.
So I'm looking back when I used to teach research methods, one of my assignments I was very proud
of was their final exam was I wrote beginning to end a fake study with fake data that was full of
all kinds of problems that spanned everything from study ethics to degrees of freedom in the
(31:41):
statistics module. So I had maybe two practices, and then, for their final example, they went in
and spotted the problems. They didn't have to spot all of them, but they had to justify and
say what's wrong with it. And students would go wild over this, even though they know that I made
this thing up. They would go crazy with all these issues. And it was perfect, if not perfect, I
loved it, because that's what we want students to come away with, but the labor that went into that,
(32:04):
and I find myself like reusing the same scenarios across semesters. Okay, we're gonna have this one
be practice, and it was just so hard. And today, every student in the class could have a new one
every day, and so it could do that work. I come in and here we go. Here's these rich scenarios
with students grappling with the messy problems, focusing on the why, but in the middle of it has
(32:27):
to be a faculty expert to make all that work. There's just some great points and great fun use
of AI for sure. I know that I've used it for scenarios and other things in my teaching and
in other contexts, as well as we work with leadership on our campuses, thinking about
policies and other ways to approach AI and making sure that we're not losing sight of critical
(32:49):
thinking and any kind of deep thinking what kinds of recommendations should we be making?
Ah, for leadership, and I love it, the receptivity to new ideas that's definitely going to be key?
Well, I'm gonna probably rehash a few of those points of be very careful about convening, like,
say, a big, expensive task force to find the one consensus, to converge on the one solution. There
(33:14):
are probably situations where that is absolutely what needs to happen. But one of the things that
I think we've seen, too, you don't really like to talk about it that much, I don't think, but
higher education is spectacularly unprepared to deal with just the pace of change here. And yeah,
partly, industry is sort of driving the bus and the pace of change, and we don't always have to
accept their timelines. But the fact of the matter is, is that you go to sleep, you wake up, and
(33:38):
there's a new version of something, and so I was really surprised. I shouldn't have been, but I was
pretty surprised by, like, how it unfolded in my local context, and just among what I could see of,
like, “Okay, here's a little PowerPoint that I can take around,” and then two weeks later, I'm like,
“I can throw that away?” I'm not used to that. So, yeah, higher education, our deliberative nature
(33:58):
is not a bad thing, necessarily. That's good. It's kind of like that's what you pay us for,
is to take the long view that's grounded in a lot of evidence. But in this o ne instance,
it makes it really, really hard for us not to feel like we're flat footed. So beware of those sorts
of things, and it's demoralizing for faculty who well meaning, they're interested in this,
(34:20):
they're excited about it, they want to contribute, and they spend four months doing something they
can't see how it played out. So my gut tells me that going for that college level is going to be
important. So maybe I don't want to chase down every single department or every single course
to say, “How is AI going to be involved here, especially around critical thinking issues,”
(34:42):
but saying, “Okay, the School of Engineering, or the School of Earth Sciences, the School
of Social Sciences, Literature and Humanities, however, it's sort of organized.” I think that
that's a pretty powerful sort of organizational approach there. And that's something where too,
it's enough to where we can get some multiple voices in there and get some buy in, but it's
(35:04):
specific enough to where I'm not trying to align what I'm doing in psychology with something that
people are doing in art history, with what forestry is doing, which, again, I think is
really gonna neutralize us if we try to take that approach. And also dividing it up. The last thing
that makes this such a hard thing for us is that it's just so inherently multifaceted, much more so
than, for example, even when learning management systems came on the scene and we're like, “Okay,
(35:27):
do we require everybody use these? Which one do we want? How do we pick? How do we train
people?” At the very bare minimum, we got to kind of separate out academic dishonesty is over here,
and it's got its own little set of policies, and let's set a team working on that, and positive
uses and preparing students for that workplace of the future, and so on. These are separate
(35:49):
issues over there. So yeah, be prepared to go in and just say, “Okay, two separate things,
we're gonna deal with them two separate ways.” I can't imagine trying to sit at home on a Saturday
and craft some long email that is somehow going to bring clarity to all the different aspects
of AI. And I'm wondering if there's any sort of resonance or spark of recognition there,
(36:10):
as leaders, that you might see.Do you have any other suggestions
on how we can help students develop their critical thinking skills.
One little theme that pops up from time to time in all the great stuff that's been written about
critical thinking, and one that I think might be worth its own special mention, is an element
of what you could call intellectual humility, or even, I guess, the flip side of that too,
(36:34):
is curiosity. Think of me with kind of cutting down some student over horoscopes in my class,
even though I find the basis for that in psychological sciences. If I do not care
at all about why a student believes that, or what it might add to their lives to believe in,
I mean, maybe it's just something fun that is like a little Rorschach test that I wake up with every
(36:55):
morning. If I have no interest in that, and then I'm also not demonstrating critical reflection,
nor am I setting a good stage for a discussion that foregrounds critical thinking. So yeah,
and that is our big challenge to faculty with our incredibly developed intellectual abilities
to balance that. Yeah, evidence is important, and some facts can be right or wrong. There's
(37:18):
different ways of interpreting them, but here's some areas where they have to support
the analysis, balancing that with all right and I could be wrong. I could have a cognitive bias
of my own that I'm not as aware of right now. I kind of have to be on the lookout for always
checking myself as I'm scrolling through that feed or going through those reviews for the thing
that I really do want to buy, no matter how many one star reviews that the company suppressed. I
(37:42):
need myself to get in the habit of doing that, and then I can model that for my students, and again,
set the stage for more productive discussion. So I'm hoping that more research also gets into that
too. What is that wonderful, magical balance of really valuing evidence and reasoning and
then realizing and I could be wrong? Well, we always appreciate, Michelle,
conversations with you. I know I always leave thinking a little more deeply
(38:05):
about whatever we've talked about, but we always wrap up by asking, what's next?
Oh, thank you, and you all as well. Even just kind of looking forward to this conversation has been
such a catalyst to revisit topics and say, “Yeah, this is something that I really want to be doing
more in.” And so that's going to be a bit of a what's next. I have a sabbatical year right now,
(38:25):
and I'm kind of trying to do what one ideally does, which is say, “Okay, what do I refocus on?”
And so I'm really at that very early stage of continuing to design ways of studying,
interventions that I could then share with faculty, and just getting an ever needed sense
of the lay of the land in this area of critical thinking. So you can definitely look for more on
that. And in fact, for folks who might be considering attending an in-person event,
(38:50):
I am going to be speaking at a couple of upcoming events where I know that I'm going to be talking
about this particular thing, again, Rocky Mountain Psychological Association for the
psychologists out there, my people. If you want to come out, I think it's in Denver this year, I have
a featured session. So that's coming up. I'll also be speaking at the Teaching Professor Conference,
(39:10):
which is in, I believe, Washington, DC, over the summer. Some of your listeners may be regulars,
or maybe thinking about that. So that's a big “what's next” item? I continue to work on this
big writing project on attention. It's still at this kind of embryonic, or if not embryonic,
partially formed stage, where I'm still working on it. But folks can watch this space for more on
(39:31):
that, and that is actually related. I will tease a little bit, that attention, what do you know,
gets involved in critical thinking, because it is involved in pretty much all other interesting
cognitive processes that take place in the mind. So that's my big what's next?
Well, thank you. It's always great talking to you, and again, as Rebecca said,
it always spurs some new ideas and some new thoughts on these topics. So, thank you.
(39:53):
Thank you. You as well.I look forward to your upcoming work.
If you've enjoyed this podcast, please subscribe and leave a review on iTunes
or your favorite podcast service. To continue the conversation, join us on
our Tea for Teaching Facebook page. You can find show notes, transcripts and
(40:16):
other materials on teaforteaching.com. Music by Michael Gary Brewer.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!