August 25, 2023 • 38 mins
Ever wondered how to navigate the complex world of reading research as an educator? Get ready for a great conversation with education researcher Neena Saha, who reveals the role of science in education and the importance of pseudoscience in developing theories. We delve into the significance of meta-analyses and discuss the need for caution when using this method. Neena then shares some current reading research with us!
Resources
- SamePage Reading Facebook Group
- MetaMetrics’ Reading Research Recap
- Join the Reading Research mailing list
- The Reading League Journal
- The Reading League recording on what teachers need to know about “how science works” (skip to 1 hour, 40 min)
- Meta-Analysis Review
- Robert Slavin’s Blogs
- Criticisms of Meta-Analysis (but also great rebuttals)
- Morphology Recent Research Studies
- Executive Functions and Morphological Awareness Explain the Shared Variance between Word Reading and Listening Comprehension
- Effects of Target Age and Genre on Morphological Complexity in Children’s Reading Material
- Contrasting Direct Instruction in Morphological Decoding and Morphological Inquiry-Analysis Interventions in Grade 3 Children With Poor Morphological Awareness
- Fluency Recent Research Study
- Comprehension Recent Research Study
We answer your questions about teaching reading in The Literacy 50-A Q&A Handbook for Teachers: Real-World Answers to Questions About Reading That Keep You Up at Night.
Grab free resources and episode alerts! Sign up for our email list at literacypodcast.com.
Join our community on Facebook, and follow us on Instagram, Facebook, & Twitter.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Lori (00:00):
You're listening to.
Melissa and Lori Love Literacy.
Today we'll be talking toresearcher Neena Saha.
She will share tips forteachers about how to read
research, includingmeta-analysis, and tell us what
reading research is hot rightnow.
Welcome, teacher friend.
I'm Lori and I'm Melissa.
(00:22):
We are two literacy educatorsin Baltimore.
Melissa (00:25):
We want the best for all kids and we know you do too.
Our district recently adopted anew literacy curriculum, which
meant a lot of change foreveryone, Lori and I can't wait
to keep learning about literacywith you today.
Lori (00:42):
Hi everyone, welcome to Melissa and Lori Love Literacy.
We are ready for today's guestto share tips for educators
reading or reviewing researchand what's hot right now.
Melissa (00:54):
Yeah, I can't wait.
We're here with Neena Saha, whowe found through a reading
research recap, which was anewsletter, turned into a
YouTube channel and gave us alot of information for some of
our podcasts.
Lori (01:06):
Yeah, Neena, welcome.
Thank you for being here.
Neena Saha (01:10):
Thank you for having me.
I'm excited to be here today.
Lori (01:12):
Yeah well, we talk a lot about reading science, research
on this podcast.
So we're thinking, before wedive into research with you, we
should take a step back and kindof ground ourselves in science.
So we're hoping that you, youmight be able to share what is
science for.
I know that's like the biggestquestion, but we thought it was
(01:34):
important to start broadly.
Neena Saha (01:37):
Yeah, well, that's it.
It's almost like a trickquestion, right, because science
is very hard to define, right,and sort of all these
definitions that we have aboutit.
There's always an exception,and I think a really good
example and I have a link ofthis is that there was, you know
, once a theory is, you know,proven false by observational or
(01:59):
experimental data, you shouldget rid of it, right.
But that's not what they didwith, like, the theory of
gravity, and I think it was likeNeptune or Saturn, one of those
planets.
They found an anomaly in it andbut they still held on and a
few years later they discoveredanother planet that was like
affecting it.
And so you know, there's it'svery hard to sort of come to a
(02:21):
consensus on what science is,but I think your question was
what is science for?
Lori (02:26):
so I've already got a little bit off track.
Neena Saha (02:28):
But in education it's to figure out
evidence-based practices.
We want to know what works forchildren, and there are some
things that science you knowcannot answer, and I think it's
important to state that at thestart sort of moral judgments,
ethical things value.
Science can't answer those andnever will be able to.
(02:50):
And then there's sort of thesequestions that science you know
maybe can answer, but we don'treally look to science to answer
them, like there's probablyevidence about, you know,
brushing your teeth.
There's something likerandomized controlled trials on
it.
But like most of us, it'sobvious, right, we don't need
that.
And then I like to you know,for back to education though, in
(03:13):
terms of what practices aremost effective for most students
.
We really do want to know thatand it's often counterintuitive
to our observations.
And if you listen to a Solderstory and the whole Fouches and
Pinot, what happened there?
You know our observations ofstudents aren't always correct.
(03:34):
So it's a great starting point.
I have links in that, you knowI'll share with the audience
here too, that go into that.
It's like pseudoscience oftenleads to really good science.
So we don't want to get rid ofthose.
You know, this sort of sciencefiction, fantasy, old wives
tales.
Those observations are criticalfor coming up with theories or
(03:56):
hypotheses that can then laterbe tested.
So science is for figuring outthose.
You know what works for themost children and even I we
could talk about this laterindividuals.
I think that there's gonna bemore.
There's this whole fascinatingfield of research called single
case research design.
It doesn't mean a single person, though it could still mean you
(04:18):
know groups of kids, but it's.
It tells you what works formore individual students and
it's really fascinating.
Sorry, long-winded answer toyour question that's okay.
Lori (04:28):
I actually expected a really long answer to that
because I feel like it was verybroad, so thank you.
Thank you for yeah.
I think, that's also reallyhelpful to contextualize outside
of education too.
So I appreciate those examplesthat really ground us in like
everyday experiences of you knowwhat is it for and and when is
it helpful.
(04:49):
So I would love to hear fromyou I mean, you took the
research from that, you thatyou've been reading a whole lot
about and you started readingresearch recap and we think
that's so helpful to giveteachers access to research.
Neena Saha (05:07):
So I'm wondering if you might be able to share some
tips you can give busy teacherswho are looking for reading
research, now that we know whatscience is for in education yeah
, and so again, this is like atough question because there's
so many resources out there andactually there's even research
on those resources which I foundand kind of blew my mind when I
(05:29):
found this research articlethat tested all these different
sites and how easy it was to getto the evidence-based practice
and whether you could trust it.
And I think my key tips thatwould be find blogs that you
feel are comfortable like.
If you're comfortable with theauthors, you know their experts
in the field and you like theirstyle of writing, that
(05:54):
translates so the reading leaguejournals a good place to start
for translational research.
Other reading experts havetheir own blogs too that they
share.
But if you really want thedirect, you know primary sources
, the research papers I wouldsay you know.
Obviously, look at the readingresearch recap.
That's the monthly blog put outby MetaMetrics.
(06:14):
It's great.
It collates or curates.
You know the research accordingto like the headings.
You can also go to open sourcesites.
You can use Google Scholar.
Sometimes it's a little bitoverwhelming and because it's
for all science fields you don'talways like if you type in
morphology it might come up withlike biology stuff, so you have
to be a little bit specific,but if there's an open access
(06:37):
article it'll let you know.
Ed archive, psi archive theseare run by the open science
framework, so a lot of these arepreprints.
So if the readers aren't awarewhat that is, it's just a pre.
It's sometimes before it's peerreviewed, before it goes peer
review, you can put up amanuscript and even after it's
peer reviewed you could say thisis, you know, the unformatted,
(06:58):
peer reviewed or in-pressarticle.
But also one thing I heard a lotwith the reading research recap
is how can I go back?
You know what's the recentresearch on?
I don't know morphology, forexample, and even I would have
to kind of go back.
Or I remembered an article andI'd have to search for it by
(07:20):
scrolling through my old blogs.
On Substack, which was where Ioriginally started it,
metametrics acquired it.
They put it on their blog, theMetametrics site, but it was
still very hard to search.
You'd have to click on eachthing.
And so that's what.
Samepage is kind of new sideproject that I'm figuring out.
You know where to take it next,but you can go to SamePage type
(07:42):
in morphology, open access, andyou'll see.
Actually, I think at leastthere should be four or five
papers that are open access.
Lori (07:49):
That's so neat.
Yeah, nina.
So we'll share SamePage a wholelot as this podcast launches,
so also put it in our show notes.
So SamePage reading is it?
I have it up, samepageorg.
Yeah, I was likeorg.
I'm wondering, though, if youmight be able to dive into,
before we kind of get ahead alittle bit more with research,
(08:10):
can you dive into peer review,just like pause, because I think
that's so important, and youand I had a conversation about
this, and I was just.
I learned so much, so I feellike our teacher listeners will
really learn too.
It was, you were, so youexplained it so easily to me.
So what is, why is it importantfor like things to be peer
(08:32):
reviewed and like why does thatmatter?
Neena Saha (08:35):
Yeah, it's a great question and it's not clear to
people kind of outside ofacademia.
And even in academia, I have tosay it's nebulous.
Nebulous in the sense thatdifferent journals will use
different methods, so there'salways kind of an exception.
I'll tell you the general sortof overview.
So peer review is when youwrite up a paper, you've done
(08:57):
your experiment, you write upyour results and you submit it
to a journal.
The first thing that usuallyhappens is the editor of that
journal kind of checks it foroverall.
Is it suitable for our journal,does it fit here?
If it is, they'll send it outto content area experts or
methods experts who will reviewit.
So if it's about three to fivepeople and they will, then the
(09:20):
editor will kind of come back,synthesize those, give you, you
know, the main points.
They'll either reject itoutright, except with major
revisions, or except with minor,some sort of flavor of those
categories.
And it's important and this Ithink this is interesting.
In prepping kind of for thispodcast, I learned a little bit
more too about it.
It's important becausescientists have said it's
(09:42):
important.
I know that that's kind of aweird answer, but hear me out.
So they've decided you knowthat this is the way to get
trusted science out there andtheir research, though
surprisingly, on peer review,doesn't show that it's, you know
, more effective, like there'snot research saying that it's
like the gold standard, the bestway to do it, and there's
(10:03):
actually some detractors in itfor it.
But my, what I think and othershave said this to, I think
Shanahan in one of his blogssays it'll catch major mistakes
or errors and it will check itjust to make sure you have the
right information there.
Science is it has to bereplicated and it's communal
(10:24):
right.
So if you're going through asystem outside a peer review,
scientists, experts in yourfield, aren't going to see it,
because that's not the acceptedway that science works.
As you know, society hasdecided, or the society of
scientists.
It's a social endeavor bypeople seeing the data you know,
testing it out in their lab,trying to replicate it.
(10:46):
And so peer review, basically,is important because scientists
have said it's important.
This is their process theyadopted.
Melissa (10:54):
What does make me feel better, like when I think about
it?
I think you know, if we'reputting so much stock in what's
science right, like this isresearch says the science says
this then we don't want justanyone to be able to come up and
say like, hey, I did thisexperiment or I wrote this
research article and you shouldbelieve me, so I don't know.
It gives me some trust that,like at least some other people
(11:16):
have checked this out Exactly.
Neena Saha (11:19):
Exactly Right.
When you have like threecontent area expertise, you know
experts checking over whatyou've done, you have some level
of confidence that, okay, thisis you know, sort of decent.
Melissa (11:30):
Right, right.
Yeah, that's interesting,though, about the research that
the research about.
Neena Saha (11:36):
I know I found that interesting too.
Melissa (11:39):
Yeah, I don't know what to do with that in my brain.
Think more about it.
Well, we wanted to talk alittle bit about you know you
made this.
We mentioned it already, butwe'll say it again same page
readingorg, which is just.
You could lose yourself forhours on that page I at least I
(12:00):
can and you really made itbecause they were.
You saw, you specifically sawfour ways to improve how we get
research to teachers.
Do you want to dive into thosefour things and tell us about
them?
Neena Saha (12:13):
Yeah, of course, and so I didn't see a good
framework when I was kind ofdoing my research for this.
So I came up with my own and Imight tweak this, you know, over
the next few months.
But there's problems of access,right.
So, getting open access papers,and don't get me started on
that A lot of this research isfunded by federal grants, which
is funded by tax dollars, sothis shouldn't be a problem.
(12:36):
But, you know, because of thepublishers and I've linked to
some, you know stories aboutthat and paywalls, access does
remain a problem.
That's a whole nother.
You know issue so, but accessis the first thing, then
aggregation, let's say eveneverything was, you know, open
access.
How do you go about?
You know there's hundreds ofjournals which are the good ones
(12:58):
.
How do you, you know, bringthose together?
And for me, I've been doing itmanually but at the same page,
we're working on trying toautomate this, so I get you know
so many emails, not hundreds.
Lori (13:10):
I was going to say hundreds, but it's not hundreds
a day.
Neena Saha (13:12):
I get a lot of emails a day of like alerts and
kind of cross-checking, you know, different Google alerts with
like research alerts for onlinefirst articles and triaging
those.
So it's a lot of work right nowbut we're coming up with a
method to make that easier, sothat's.
The second thing is aggregationis a problem and then there's
(13:33):
kind of well, two, maybe threemore.
So understanding, synthesis andtranslation.
Understanding, you know, let'ssay you have the article in
front of you, how do you makesense of it?
Right, there's a lot ofscientific jargon.
How do you understand the statsand the methods?
And I don't have a few tips forteachers that we can get to on
that, but there's that's.
(13:54):
You know this might be anunpopular opinion, but I do
think teachers need you know acourse in scientific design or
logic and causal inference aspart of their teacher prep
programs.
This idea that like no, we needgatekeepers who will always kind
of translate the research andthen give it because teachers
(14:15):
won't understand it.
You look back through history,sharing of information and
knowledge has always been in thefavor of you know, like people
like eventually getting it andlearning it right To people like
the Gutenberg and the press andeverything.
It's like no, keep theinformation away, because there
will be like social uprising and, you know, riots and stuff with
all this information.
Never, this stuff doesn'thappen, and teachers need to
(14:37):
have access to this, even ifthey don't understand it all.
But there are a lot of tools tohelp with understanding.
And then synthesis, like onesingle paper doesn't really mean
anything, and so that's kind ofcrazy, right, you have to take
that paper and see, well, werethere five other papers on the
topic that said something theopposite?
Right?
You have to view that paperwithin the body of the
(15:00):
scientific research on it, andso that's another kind of
problem.
We have methods to deal withthat, such as meta-analysis.
And then I kind of added thislast one, I think, since we
talked, which is translation andthat's, you know, synthesis is
great, but then how do you?
You could have, you know, basicscience papers and synthesizers
that have nothing to do withthe classroom, and how to
(15:21):
translate that.
So that's kind of another issueI'm trying to think through.
And, to be perfectly clear,same page is just now addressing
access and aggregation.
I have ideas to help withunderstanding and synthesis and
translation, but right now theyare very bare bones, such as,
like pulling out, you know, keystatements.
If there's something thattranslates directly to the
classroom, right?
(15:42):
So working on those other ones.
Melissa (15:45):
I was just going to say I love the idea of translation.
I'm really glad you broughtthat up because even you know if
it's is a classroom, it doesn'tmean that classroom looked like
your classroom in the studyright.
So you have to also think doesthis work for me in my classroom
with my students, just becauseit worked for that classroom in
the study?
Neena Saha (16:04):
That is so important , melissa, because that's one of
the tips that you canimmediately hand off to teachers
Check the population, how arethey describing it?
Because if it's not, yourstudents, if you don't see your
student demographics reflectedthere and like the racial
breakdown, the SES status, thebilingual that, you're right,
that study doesn't have, youknow, like the geeky term is
external validity.
(16:25):
It doesn't, you know,generalize to your students, so
that's not one that will behelpful.
Lori (16:30):
That's a good point.
I was thinking a lot about thesynthesis point you made.
Like you know, I think it'seasy to see what we want to see
as we read research and justhave that like confirmation bias
, like oh yeah, okay, so thisthing I am doing is the right
thing to do, instead of seeingthe things that are harder to
see, which are the things thatmaybe we don't want to see, that
(16:53):
it's not, I don't know, it'shard, yeah it's my pet peeve
when I see people evenresearchers will do this where
they'll like post one study onFacebook and it's like, okay,
great, but what else is outthere?
Neena Saha (17:06):
Right, like it may answer your question, but it's
like we know there's other.
So it's always good to give,like you know, some sort of
couching of the background, oflike here's what we know,
because you know you have to.
One study could disagree withanother and, like I think I
linked to a really fascinatingNew York Times article, maybe
two years ago, about these tworandomized controlled trials and
(17:27):
it was for Alzheimer'smedication that you know
directly contrasted each other,like they had results that were
in opposition.
So it's like what do you dothen?
And of course, like you said,laura, you'll, if you'll only
see the one that you kind ofwant to believe with maybe not
on purpose, right, but likeconfirmation bias.
Melissa (17:45):
Yeah, yeah, there's almost always like.
You always feel like if youhave something you want to prove
, you can almost always findsome research study, Exactly
right, you could find something.
There's something to prove thatthis is right.
I'm wondering if, since we'reon this, if we could talk a
little bit about meta-analysisyou brought those up and I know
that I've seen them can talkedabout a bit more, as we're
(18:06):
talking about the science ofreading and sometimes I worry
because it feels like hey, ifthis is called a meta-analysis,
like stamp done, this is the endof the story, like don't ask
any more questions, and I getworried.
But like I don't even quiteknow what a meta-analysis
entails and how they've comealong.
So I'm wondering if you couldjust dive into that and tell us
(18:28):
all you know about those.
Neena Saha (18:30):
Yes, and I'll put the caveat out here, I am not an
expert by any means, but I willshare what I know, because I
did do a meta-analysis, anetwork meta-analysis, which is
like a different type, as partof my graduate work at
Vanderbilt.
So, but I will tell you what Iknow about it.
First, though, I want to backup because I want to make two
points about this.
Meta-analysis are kind of putthere as this, almost above,
(18:52):
like the gold standard of RCTs,right, you know, you have a
great, amazing meta-analysis andyou take what they say and,
like that's, you know the finalword on it, and we have to kind
of step back.
Science is nuanced, right?
What is the question you wantanswered?
First, Because I thinkmeta-analysis are important, but
(19:13):
they are not the be all, endall.
If, let's say, you are teachingand this is a very this isn't a
weird scenario that I'm going tobring up I think people might
think like, oh, she's justtrying to find an exception here
.
But if you're a teacher, wekind of know what works for the
majority of students, right, butyou might have that exception
student and I certainly didNothing was working and you know
(19:34):
.
So you type in, you want toknow what works for that student
.
Normal group studies, andespecially meta-analysis, aren't
going to be able to answer that.
Because of the way theyaggregate the studies and the
statistics, they don't even tellyou which is a problem some of
the demographics.
And so I want to get away fromthis idea that meta-analysis are
(19:54):
great.
They are great for when youwant to know general statements
about the average student.
They will never be able to tellyou about those exceptions that
you're trying to figure outwhat works for.
And that's that single case,research design I was talking
about.
And I do think this is my 10year bet that with better ways
of aggregating and figuring out,collating information, maybe
(20:16):
even with AI, machine learning,we're going to be able to get to
a place where you have like adropdown of, like this student
you know these are their scores.
Find me the single case studythat showed that worked for,
like increasing oral readingfluency, for that.
So, but that aside.
So that's the main point.
But let's say you do kind ofwant to know generally what
(20:37):
works.
Meta-analysis take severalstudies and they should all be
alike, similar in design, and ifthey're not, then you want to
figure out.
You can do like moderationanalysis later to figure out the
aspects that made the studies.
You know, maybe, maybe ifthere's variation, but you want
to choose similar studies withsimilar interventions and that
(21:02):
have a control.
So you want high qualitystudies is, I guess, is what I'm
trying to say, and that's, youknow, the inclusion criteria for
those really matters and shouldbe detailed.
There's this great vlog.
So Robert Slavin or Slavin, I'mnot sure how you pronounce this
thing he has these great seriesof blogs on meta-analyses and he
gives these sort of threethings that you wanna pay
(21:24):
attention to, or maybe fouractually.
But I would suggest thatreaders take a look at that
because some of those are kindof obvious, like red flags that
they can check for.
And I think one has to do asample size or inflated effect
sizes.
If you see meta-analyses thatare reporting really high, you
know above, like 0.5, 0.6.
And I think one in his examplehad like 1.2 effect, like given
(21:46):
effect size, which is crazylarge.
You wanna be cautious, you wannago back to those original
studies, see if they belongedthere, because like that one
that had a 1.2, I think he saysit was like an example of like
tennis, it didn't even have todo with like education.
So always go back to theoriginal source and researchers
(22:10):
need to be good about describingthe original studies in there
and the populations too.
But this idea of like kind ofgarbage in, garbage out holds
you really, robert was a bigproponent of high quality
meta-analyses and inclusionstandards and you should have
(22:30):
those, and I think he does as hehas.
Like that he ran the bestevidence in Cyclopedia B that
did high quality meta-analyses,and I think there was like one
other thing I wanted to sayabout oh, meta-analyses now are
becoming popular, but yeah,which is like two steps removed?
Yeah, so it's like yeah, well,I've seen it.
(22:54):
I've seen people kind of doingthese and I'm like, wait a
minute, like what was the con?
Like I always wanna see thedata right, and like the
original studies, like what werethey on?
To give me like a sense.
It's like knowing thepopulation, demographics of the
students in a study, right, well, the population here are
studies, because it's a study ofstudies and so in those you're
(23:16):
even further away, right, causelike you don't know so they're
just like compiling themeta-analyses.
Yes, they're taking several, butit's like a game of telephone,
almost in a way.
Right it gets distorted, so youjust have to be careful.
Meta-analyses are great whendone correctly.
Lori (23:35):
That sounds really great, like imagine if I was, if I
wasn't having this conversationwith you and I had heard that I
would be like, wow, that soundsreally fancy.
Meta, meta.
That must be super, super valid, you know, like even more valid
than the first kind of meta.
Neena Saha (23:51):
Right, not necessarily though.
Right Like education solved.
We can do meta-metas now Likewe know everything that works,
but it's not true.
It's a back to that match oflike the population too.
You have to know all thatinformation to see if it works
for your students.
Lori (24:07):
I am wondering if we just want to kind of leave teachers
with, as they're heading back toschool, right Like sharing some
reading, research, topics ofinterest or current research
that you're seeing.
Nina, we know that you'rereally great about knowing the
latest and greatest research.
Neena Saha (24:25):
Yeah, of course.
So I see so much research but Ithink my sort of the trends
that I'm seeing that I findinteresting are and this might
be because we kind of know nowphonics is explicit, systematic,
cumulative what we have to dofor that, and so I'm seeing a
lot more on morphology, fluencyand comprehension and
(24:48):
specifically morphology.
I find this fascinating and Ithink that's gonna be like a
trend, kind of alert that in thenext few years is gonna become
more important and we're gonnasee sort of more instruction and
programs, products around that,and which is good.
Because English ismorphophonemic.
We're not a true alphabeticlanguage.
(25:08):
Our words are composed of notjust sound information but sound
and meaning information andkind of a phrase that's used.
With morphology, a lot is likeislands of regularity.
Right, they provide thosemorphemes.
So morphemes, if the audiencedoesn't know, are just the
smallest unit of meaning in aword.
So, like with the word jumped,the jump part is the stem.
(25:31):
That, spelled as E-D is, youknow, like the suffix indicate,
like past tense and those.
So that's two morphemes and sothat island of regularity would
be that E-D, because we see thatagain.
In other words, a lot it's asound that can also.
You know that's prettyconsistent.
Sometimes it's the id, like incheat id versus jumped, but they
(25:56):
still kind of give you wordmeaning too.
So morphology is reallyinteresting to me and I've
linked, if the audience isinterested, to three sort of
studies on that of differentflavors.
So the first one is this reallycool kind of basic science
theoretical background modelingpaper and it's called executive
(26:17):
functions and morphologicalawareness explain the shared
variance between word readingand listening comprehension.
I really liked it because nowwe're seeing all these models
that kind of go beyond thesimple view model.
Right, simple view.
We've got, you know, wordreading skills, oral language,
you'll get good readingcomprehension.
But there's a lot more.
(26:38):
Simple is great.
We like parsimmonian science,but sometimes it's too simple
and so now we're trying tofigure out what are the other
factors.
And apparently morphology is abig one.
Actually it explains a lot ofvariants.
When I was reading the paper Iactually thought it was funny,
because you don't usually usewords like striking findings or
(26:58):
something.
And then the author did in thisone which I think is great that
that got through peer review,because usually we're always
hedging.
I wish I had the example ofthis other paper I had just read
, where it was potentially verymodest effects and I was like
that's science free.
But this paper was likestriking findings of morphology.
I don't have the quote in frontof me but it was an interesting
(27:20):
read.
So not an experimental study,it was correlational structural
equation modeling, but stillvery interesting.
Showing the morphology is animportant link between the two.
Melissa (27:33):
That's why I was just going to ask Nina, because I can
see morphology important forboth learning spelling, but also
comprehending.
But you're talking aboutthey're talking about both right
that?
It's like the bridge betweenExactly.
Neena Saha (27:46):
Yeah, and the active view model which is, I believe
and I'll, duke and Matt Burnshad a big meta and I'll
Cartwright OK, cartwright, ok.
And so they also havemorphology playing a big role.
I believe in the bridgingprocesses part, so it's good and
it makes sense why there's both, because it gives information
about meaning, but it's alsothere for sound, so it's like
(28:08):
the bridge.
And then the next study onmorphology that I'd like to
share is one that's a little bit, I hope, more practical for
teachers, because this was acorpus study, so a large
language.
What they did.
This was in the UK.
They took a large body ofchildren's literature and did
(28:28):
all these statistics on it, likecalculating frequency and stuff
.
And this paper is open accessand has a list of suffixes by
frequency, which I think it'salways great.
When the researchers put thatin, it's like let's go check and
see what are the ones I shouldmaybe teach.
It's really nice.
So I wanted to share that onethat's called effects of target
(28:49):
age and genre on morphologicalcomplexity in children's reading
material.
I didn't go do a deep dive onit because I don't think they
had their suffixes ranked byfrequency.
They had it done differently,so I don't know the top ones,
but I would recommend doing thatif you want to teach certain
high important morphologicalunits or suffixes in this case.
(29:09):
And then the last one onmorphology was an RCT.
So I found this one reallyinteresting.
Oh, and this is actually theone that only found very modest
and used very hedging wording.
So this one's an interventionstudy.
What they did was they comparedto third grade students in
Canada, but they were Englishspeaking and the title was
called Contrasting DirectInstruction in Morphological
(29:30):
Decoding and MorphologicalInquiry Analysis.
Interventions in Grade 3Children with Poor Morphological
Awareness.
So what they did is theystarted with about 163 students,
screened them on morphologicalawareness skills and only took
the ones with poor morphologicalawareness skills.
Then they had three conditions,two interventions.
(29:52):
One was this sort of directdecoding, one where it was
print-based, and the other wasinquiry-based where they asked
questions about like is the UNin uninvite important versus the
UN in uncle, kind of liketeaching those strategies.
And then they had the control,which was businesses usual and
(30:14):
that one.
They didn't have any sort ofmorphological training or
instruction in that one.
And a little bit disappointing Ithought they'd find they had
all these post-test measures, sothey had a post-right.
Immediately they tested on wordreading, reading vocabulary,
sentence comprehension, and theythought there were very modest
(30:35):
effects for, maybe, readingvocabulary, for the direct
decoding intervention, I believe, maybe one other, but not kind
of across the board largeeffects, and that the delayed
post-test none of the effectskind of stayed.
So if you spend all this timeand resources, you want to make
sure that the effects of theintervention last and at
(30:58):
post-test, I think none of themwere significant.
So kind of keep your eye onthis, though.
Even though it was just like itwasn't null effects across the
board, there's something there.
So I think we're going tofigure out how to better make
these more effective, thesemorphological awareness
interventions.
Melissa (31:18):
That's what I was just going to ask, nino, when you
were talking about it.
What do you do with that?
Because you just talked aboutother studies that did show that
it's important to teachmorphology, and I think that's
one of those things, too, whereyou're like it just makes sense.
It's like the brushing yourteeth example.
It just makes sense to teachour kids morphology, to make
sense of words, but then a studycomes along that doesn't show
(31:39):
much.
What do you do as a teacher?
Neena Saha (31:42):
Yeah, as a teacher, and that's why translation is so
tricky, because these are thestudies.
Right, we have good, solidtheoretical evidence, we've got
corpus that study about, we knowthere's a lot of morphines, we
know our language, but then whenit comes to doing the
intervention, it's like how comethere weren't great effects?
(32:06):
And I think it's just so hardto find good implementation like
translatable studies, andthat's just the short answer,
the kind of cop out answer thatwas like humans are messy,
there's so many variables goingon, there's so many things that
are tricky, but I think they'llget there.
And this is just one paper.
But the authors did say, eventhough it's very modest effects,
(32:27):
those strategies still work.
Like trying to tease that sortof word inquiry of figuring out
if the un and uninvited is thesame as on an uncle.
No, they're different, right,yeah, so that's a lot about
morphology.
Melissa (32:40):
Oh yeah, I was gonna ask if there was, because you
mentioned fluency andcomprehension.
Do you want to talk about those?
Neena Saha (32:45):
So fluency.
I think I've been seeing a lotabout readers theater and,
admittedly, I didn't really knowwhat that was, but it's pretty
much what it sounds like.
You get up there and you kindof like give Students line, and
you guys could jump in andcorrect me if I'm wrong, but
students get to kind of act outthe dialogue lines right, and so
I I saw a lot about this, Ishould say on, like these
Facebook groups, about you know,transforming, you know
(33:07):
decodable passages into, likereaders theater, things like oh,
that's interesting.
And then I was like, is thereany research on this?
Well, this study kind of cameout right as I was thinking
about this and it's a systematicreview and meta-analysis of the
readers theater impact on thedevelopment of reading skills,
and they found pretty largeeffects.
I didn't go through to see ifthere's red flags, but by
skimming it I think it's.
(33:29):
It looks legitimate to me andit's pure reviewed.
So that's a good one to kind oflook through to maybe see
different flavors ofinterventions for readers
theater Programs that work.
Melissa (33:39):
Yeah that's a nice one, because I think sometimes
people can think it's just liketrying to make it fun yeah, but
to know that it also has somevalidity behind it is good.
Neena Saha (33:50):
Agreed when it's fun and valid.
That's, and I'm especiallyexcited because it's a
meta-analysis.
Lori (33:56):
That is a good example of a valid meta-analysis that we're
giving everyone in the shownotes and I mean it's readers
theater, which is really, Ithink, fairly easy to implement.
So Fun, easy and valid, allreally good things.
Melissa (34:13):
Our friend Chase Young will be very excited.
Neena Saha (34:17):
And then I think I just put I have a few on
comprehension, not gonna go intothat, one's a survey.
Well, I guess I'll go into it alittle bit.
So once a survey study of kids,sorry, of teachers in Australia
, but I love survey studies.
I just love seeing what teacheryou know when people do this,
because we have to know whereteachers are at Before you can
you know, try and sort of quote,unquote, fix things or develop,
(34:37):
and so this one was interesting.
I think they found that andthey found this.
I think in like every surveystudy I have read about, no
matter the topic, that there'scontradicting answers.
So teachers will know a littlebit about something, but then,
like they'll contradictthemselves later, which is just
means I think we're not doing asgood as a job.
As you know, sciencecommunicators or researchers or
(34:59):
what have you getting thatInformation out there?
But so that one's there, Ithink it's open access.
Then I also, you know,sometimes there's with science
of reading, there's this focuson phonics and first graders,
and so for comprehension, I havethis one for fourth grade
students and it's in socialstudies and I think I actually
heard the first version of thisat triplets are last summer.
It's a professional developmentprogram to increase vocabulary
(35:23):
and comprehension in an embeddedlike social studies, fourth
grade.
So that's a cool one, I think.
If you know, teach olderstudents or Want to see how you
can do like readinginterventions and other you know
, not the content, not literacyblocks, so I tried to put
something for everyone.
And then the last thing I justwanted to touch on, which I
think also trend alert, likemorphology, I think is going to
(35:43):
be big, but using data topersonalize instruction.
You know, when you think oflike the five pillars, that's
not one of them, right, but itshould be embedded in like all
of them.
You should be using data, andso I linked to one about how
using targeted readinginstructions is better than you
know, just giving like aone-size-fits-all Right.
You want to use children's datato give them the right
(36:06):
intervention.
Lori (36:07):
So I put that one there.
Yeah, that's so important.
I think, nina, I think you'regoing to win the prize for the
longest show notes in history ofour podcast.
Is there anything else thatyou'd like to share with our
(36:28):
listeners before we, before wecome to a close?
Neena Saha (36:32):
No, I'm just grateful that you guys have me
on here and I think it's soimportant.
You know I may not have donelike the best job in explaining
I think some of these ideas thatare.
So, you know, science is sonuanced right, and I think we
just need more of moreDiscussion on this and more
people talking about the nuance,because it's really hard to
convey some of these no ideasand like causal inference and
(36:55):
what makes a study good and whenshould you trust it?
So no, I'm just grateful to behere and I don't have any.
There might be something laterwhere I add to the show notes
Like here's something I messedup.
Melissa (37:08):
I love that discussion about the nuance because I think
I'm seeing a lot of like I'mright, you're wrong kind of
battles happening versus likethese kind of discussions.
Okay, let's like talk through.
Yeah, let's talk through what,what this actually means.
Well, thank you, yeah of course.
Lori (37:24):
Yeah, this was really fun.
Thank you for being here.
Thanks for listening.
Literacy lovers, to stayconnected with us, sign up for
our email list at literacypodcast comm.
Melissa (37:38):
And to keep learning together.
Join the Melissa and Lori loveliteracy podcast Facebook group
and be sure to follow us onInstagram and Twitter.
Lori (37:48):
If this episode resonated with you, take a moment to share
with a teacher friend or leaveus a five-star rating and review
on Apple podcasts.
Melissa (37:58):
Just a quick reminder that the views and opinions
expressed by the hosts andguests of the Melissa and Lori
love literacy podcast are notnecessarily the opinions of
great minds, pbc or itsemployees.
Lori (38:10):
We appreciate you so much and we're so glad you're here to
learn with us.
You're listening to.
Melissa and Lori Love Literacy.
Today we'll be talking toresearcher Neena Saha.
She will share tips forteachers about how to read
research, includingmeta-analysis, and tell us what
reading research is hot rightnow.
Welcome, teacher friend.
I'm Lori and I'm Melissa.
(00:22):
We are two literacy educatorsin Baltimore.
Melissa (00:25):
We want the best for all kids and we know you do too.
Our district recently adopted anew literacy curriculum, which
meant a lot of change foreveryone, Lori and I can't wait
to keep learning about literacywith you today.
Lori (00:42):
Hi everyone, welcome to Melissa and Lori Love Literacy.
We are ready for today's guestto share tips for educators
reading or reviewing researchand what's hot right now.
Melissa (00:54):
Yeah, I can't wait.
We're here with Neena Saha, whowe found through a reading
research recap, which was anewsletter, turned into a
YouTube channel and gave us alot of information for some of
our podcasts.
Lori (01:06):
Yeah, Neena, welcome.
Thank you for being here.
Neena Saha (01:10):
Thank you for having me.
I'm excited to be here today.
Lori (01:12):
Yeah well, we talk a lot about reading science, research
on this podcast.
So we're thinking, before wedive into research with you, we
should take a step back and kindof ground ourselves in science.
So we're hoping that you, youmight be able to share what is
science for.
I know that's like the biggestquestion, but we thought it was
(01:34):
important to start broadly.
Neena Saha (01:37):
Yeah, well, that's it.
It's almost like a trickquestion, right, because science
is very hard to define, right,and sort of all these
definitions that we have aboutit.
There's always an exception,and I think a really good
example and I have a link ofthis is that there was, you know
, once a theory is, you know,proven false by observational or
(01:59):
experimental data, you shouldget rid of it, right.
But that's not what they didwith, like, the theory of
gravity, and I think it was likeNeptune or Saturn, one of those
planets.
They found an anomaly in it andbut they still held on and a
few years later they discoveredanother planet that was like
affecting it.
And so you know, there's it'svery hard to sort of come to a
(02:21):
consensus on what science is,but I think your question was
what is science for?
Lori (02:26):
so I've already got a little bit off track.
Neena Saha (02:28):
But in education it's to figure out
evidence-based practices.
We want to know what works forchildren, and there are some
things that science you knowcannot answer, and I think it's
important to state that at thestart sort of moral judgments,
ethical things value.
Science can't answer those andnever will be able to.
(02:50):
And then there's sort of thesequestions that science you know
maybe can answer, but we don'treally look to science to answer
them, like there's probablyevidence about, you know,
brushing your teeth.
There's something likerandomized controlled trials on
it.
But like most of us, it'sobvious, right, we don't need
that.
And then I like to you know,for back to education though, in
(03:13):
terms of what practices aremost effective for most students
.
We really do want to know thatand it's often counterintuitive
to our observations.
And if you listen to a Solderstory and the whole Fouches and
Pinot, what happened there?
You know our observations ofstudents aren't always correct.
(03:34):
So it's a great starting point.
I have links in that, you knowI'll share with the audience
here too, that go into that.
It's like pseudoscience oftenleads to really good science.
So we don't want to get rid ofthose.
You know, this sort of sciencefiction, fantasy, old wives
tales.
Those observations are criticalfor coming up with theories or
(03:56):
hypotheses that can then laterbe tested.
So science is for figuring outthose.
You know what works for themost children and even I we
could talk about this laterindividuals.
I think that there's gonna bemore.
There's this whole fascinatingfield of research called single
case research design.
It doesn't mean a single person, though it could still mean you
(04:18):
know groups of kids, but it's.
It tells you what works formore individual students and
it's really fascinating.
Sorry, long-winded answer toyour question that's okay.
Lori (04:28):
I actually expected a really long answer to that
because I feel like it was verybroad, so thank you.
Thank you for yeah.
I think, that's also reallyhelpful to contextualize outside
of education too.
So I appreciate those examplesthat really ground us in like
everyday experiences of you knowwhat is it for and and when is
it helpful.
(04:49):
So I would love to hear fromyou I mean, you took the
research from that, you thatyou've been reading a whole lot
about and you started readingresearch recap and we think
that's so helpful to giveteachers access to research.
Neena Saha (05:07):
So I'm wondering if you might be able to share some
tips you can give busy teacherswho are looking for reading
research, now that we know whatscience is for in education yeah
, and so again, this is like atough question because there's
so many resources out there andactually there's even research
on those resources which I foundand kind of blew my mind when I
(05:29):
found this research articlethat tested all these different
sites and how easy it was to getto the evidence-based practice
and whether you could trust it.
And I think my key tips thatwould be find blogs that you
feel are comfortable like.
If you're comfortable with theauthors, you know their experts
in the field and you like theirstyle of writing, that
(05:54):
translates so the reading leaguejournals a good place to start
for translational research.
Other reading experts havetheir own blogs too that they
share.
But if you really want thedirect, you know primary sources
, the research papers I wouldsay you know.
Obviously, look at the readingresearch recap.
That's the monthly blog put outby MetaMetrics.
(06:14):
It's great.
It collates or curates.
You know the research accordingto like the headings.
You can also go to open sourcesites.
You can use Google Scholar.
Sometimes it's a little bitoverwhelming and because it's
for all science fields you don'talways like if you type in
morphology it might come up withlike biology stuff, so you have
to be a little bit specific,but if there's an open access
(06:37):
article it'll let you know.
Ed archive, psi archive theseare run by the open science
framework, so a lot of these arepreprints.
So if the readers aren't awarewhat that is, it's just a pre.
It's sometimes before it's peerreviewed, before it goes peer
review, you can put up amanuscript and even after it's
peer reviewed you could say thisis, you know, the unformatted,
(06:58):
peer reviewed or in-pressarticle.
But also one thing I heard a lotwith the reading research recap
is how can I go back?
You know what's the recentresearch on?
I don't know morphology, forexample, and even I would have
to kind of go back.
Or I remembered an article andI'd have to search for it by
(07:20):
scrolling through my old blogs.
On Substack, which was where Ioriginally started it,
metametrics acquired it.
They put it on their blog, theMetametrics site, but it was
still very hard to search.
You'd have to click on eachthing.
And so that's what.
Samepage is kind of new sideproject that I'm figuring out.
You know where to take it next,but you can go to SamePage type
(07:42):
in morphology, open access, andyou'll see.
Actually, I think at leastthere should be four or five
papers that are open access.
Lori (07:49):
That's so neat.
Yeah, nina.
So we'll share SamePage a wholelot as this podcast launches,
so also put it in our show notes.
So SamePage reading is it?
I have it up, samepageorg.
Yeah, I was likeorg.
I'm wondering, though, if youmight be able to dive into,
before we kind of get ahead alittle bit more with research,
(08:10):
can you dive into peer review,just like pause, because I think
that's so important, and youand I had a conversation about
this, and I was just.
I learned so much, so I feellike our teacher listeners will
really learn too.
It was, you were, so youexplained it so easily to me.
So what is, why is it importantfor like things to be peer
(08:32):
reviewed and like why does thatmatter?
Neena Saha (08:35):
Yeah, it's a great question and it's not clear to
people kind of outside ofacademia.
And even in academia, I have tosay it's nebulous.
Nebulous in the sense thatdifferent journals will use
different methods, so there'salways kind of an exception.
I'll tell you the general sortof overview.
So peer review is when youwrite up a paper, you've done
(08:57):
your experiment, you write upyour results and you submit it
to a journal.
The first thing that usuallyhappens is the editor of that
journal kind of checks it foroverall.
Is it suitable for our journal,does it fit here?
If it is, they'll send it outto content area experts or
methods experts who will reviewit.
So if it's about three to fivepeople and they will, then the
(09:20):
editor will kind of come back,synthesize those, give you, you
know, the main points.
They'll either reject itoutright, except with major
revisions, or except with minor,some sort of flavor of those
categories.
And it's important and this Ithink this is interesting.
In prepping kind of for thispodcast, I learned a little bit
more too about it.
It's important becausescientists have said it's
(09:42):
important.
I know that that's kind of aweird answer, but hear me out.
So they've decided you knowthat this is the way to get
trusted science out there andtheir research, though
surprisingly, on peer review,doesn't show that it's, you know
, more effective, like there'snot research saying that it's
like the gold standard, the bestway to do it, and there's
(10:03):
actually some detractors in itfor it.
But my, what I think and othershave said this to, I think
Shanahan in one of his blogssays it'll catch major mistakes
or errors and it will check itjust to make sure you have the
right information there.
Science is it has to bereplicated and it's communal
(10:24):
right.
So if you're going through asystem outside a peer review,
scientists, experts in yourfield, aren't going to see it,
because that's not the acceptedway that science works.
As you know, society hasdecided, or the society of
scientists.
It's a social endeavor bypeople seeing the data you know,
testing it out in their lab,trying to replicate it.
(10:46):
And so peer review, basically,is important because scientists
have said it's important.
This is their process theyadopted.
Melissa (10:54):
What does make me feel better, like when I think about
it?
I think you know, if we'reputting so much stock in what's
science right, like this isresearch says the science says
this then we don't want justanyone to be able to come up and
say like, hey, I did thisexperiment or I wrote this
research article and you shouldbelieve me, so I don't know.
It gives me some trust that,like at least some other people
(11:16):
have checked this out Exactly.
Neena Saha (11:19):
Exactly Right.
When you have like threecontent area expertise, you know
experts checking over whatyou've done, you have some level
of confidence that, okay, thisis you know, sort of decent.
Melissa (11:30):
Right, right.
Yeah, that's interesting,though, about the research that
the research about.
Neena Saha (11:36):
I know I found that interesting too.
Melissa (11:39):
Yeah, I don't know what to do with that in my brain.
Think more about it.
Well, we wanted to talk alittle bit about you know you
made this.
We mentioned it already, butwe'll say it again same page
readingorg, which is just.
You could lose yourself forhours on that page I at least I
(12:00):
can and you really made itbecause they were.
You saw, you specifically sawfour ways to improve how we get
research to teachers.
Do you want to dive into thosefour things and tell us about
them?
Neena Saha (12:13):
Yeah, of course, and so I didn't see a good
framework when I was kind ofdoing my research for this.
So I came up with my own and Imight tweak this, you know, over
the next few months.
But there's problems of access,right.
So, getting open access papers,and don't get me started on
that A lot of this research isfunded by federal grants, which
is funded by tax dollars, sothis shouldn't be a problem.
(12:36):
But, you know, because of thepublishers and I've linked to
some, you know stories aboutthat and paywalls, access does
remain a problem.
That's a whole nother.
You know issue so, but accessis the first thing, then
aggregation, let's say eveneverything was, you know, open
access.
How do you go about?
You know there's hundreds ofjournals which are the good ones
(12:58):
.
How do you, you know, bringthose together?
And for me, I've been doing itmanually but at the same page,
we're working on trying toautomate this, so I get you know
so many emails, not hundreds.
Lori (13:10):
I was going to say hundreds, but it's not hundreds
a day.
Neena Saha (13:12):
I get a lot of emails a day of like alerts and
kind of cross-checking, you know, different Google alerts with
like research alerts for onlinefirst articles and triaging
those.
So it's a lot of work right nowbut we're coming up with a
method to make that easier, sothat's.
The second thing is aggregationis a problem and then there's
(13:33):
kind of well, two, maybe threemore.
So understanding, synthesis andtranslation.
Understanding, you know, let'ssay you have the article in
front of you, how do you makesense of it?
Right, there's a lot ofscientific jargon.
How do you understand the statsand the methods?
And I don't have a few tips forteachers that we can get to on
that, but there's that's.
(13:54):
You know this might be anunpopular opinion, but I do
think teachers need you know acourse in scientific design or
logic and causal inference aspart of their teacher prep
programs.
This idea that like no, we needgatekeepers who will always kind
of translate the research andthen give it because teachers
(14:15):
won't understand it.
You look back through history,sharing of information and
knowledge has always been in thefavor of you know, like people
like eventually getting it andlearning it right To people like
the Gutenberg and the press andeverything.
It's like no, keep theinformation away, because there
will be like social uprising and, you know, riots and stuff with
all this information.
Never, this stuff doesn'thappen, and teachers need to
(14:37):
have access to this, even ifthey don't understand it all.
But there are a lot of tools tohelp with understanding.
And then synthesis, like onesingle paper doesn't really mean
anything, and so that's kind ofcrazy, right, you have to take
that paper and see, well, werethere five other papers on the
topic that said something theopposite?
Right?
You have to view that paperwithin the body of the
(15:00):
scientific research on it, andso that's another kind of
problem.
We have methods to deal withthat, such as meta-analysis.
And then I kind of added thislast one, I think, since we
talked, which is translation andthat's, you know, synthesis is
great, but then how do you?
You could have, you know, basicscience papers and synthesizers
that have nothing to do withthe classroom, and how to
(15:21):
translate that.
So that's kind of another issueI'm trying to think through.
And, to be perfectly clear,same page is just now addressing
access and aggregation.
I have ideas to help withunderstanding and synthesis and
translation, but right now theyare very bare bones, such as,
like pulling out, you know, keystatements.
If there's something thattranslates directly to the
classroom, right?
(15:42):
So working on those other ones.
Melissa (15:45):
I was just going to say I love the idea of translation.
I'm really glad you broughtthat up because even you know if
it's is a classroom, it doesn'tmean that classroom looked like
your classroom in the studyright.
So you have to also think doesthis work for me in my classroom
with my students, just becauseit worked for that classroom in
the study?
Neena Saha (16:04):
That is so important , melissa, because that's one of
the tips that you canimmediately hand off to teachers
Check the population, how arethey describing it?
Because if it's not, yourstudents, if you don't see your
student demographics reflectedthere and like the racial
breakdown, the SES status, thebilingual that, you're right,
that study doesn't have, youknow, like the geeky term is
external validity.
(16:25):
It doesn't, you know,generalize to your students, so
that's not one that will behelpful.
Lori (16:30):
That's a good point.
I was thinking a lot about thesynthesis point you made.
Like you know, I think it'seasy to see what we want to see
as we read research and justhave that like confirmation bias
, like oh yeah, okay, so thisthing I am doing is the right
thing to do, instead of seeingthe things that are harder to
see, which are the things thatmaybe we don't want to see, that
(16:53):
it's not, I don't know, it'shard, yeah it's my pet peeve
when I see people evenresearchers will do this where
they'll like post one study onFacebook and it's like, okay,
great, but what else is outthere?
Neena Saha (17:06):
Right, like it may answer your question, but it's
like we know there's other.
So it's always good to give,like you know, some sort of
couching of the background, oflike here's what we know,
because you know you have to.
One study could disagree withanother and, like I think I
linked to a really fascinatingNew York Times article, maybe
two years ago, about these tworandomized controlled trials and
(17:27):
it was for Alzheimer'smedication that you know
directly contrasted each other,like they had results that were
in opposition.
So it's like what do you dothen?
And of course, like you said,laura, you'll, if you'll only
see the one that you kind ofwant to believe with maybe not
on purpose, right, but likeconfirmation bias.
Melissa (17:45):
Yeah, yeah, there's almost always like.
You always feel like if youhave something you want to prove
, you can almost always findsome research study, Exactly
right, you could find something.
There's something to prove thatthis is right.
I'm wondering if, since we'reon this, if we could talk a
little bit about meta-analysisyou brought those up and I know
that I've seen them can talkedabout a bit more, as we're
(18:06):
talking about the science ofreading and sometimes I worry
because it feels like hey, ifthis is called a meta-analysis,
like stamp done, this is the endof the story, like don't ask
any more questions, and I getworried.
But like I don't even quiteknow what a meta-analysis
entails and how they've comealong.
So I'm wondering if you couldjust dive into that and tell us
(18:28):
all you know about those.
Neena Saha (18:30):
Yes, and I'll put the caveat out here, I am not an
expert by any means, but I willshare what I know, because I
did do a meta-analysis, anetwork meta-analysis, which is
like a different type, as partof my graduate work at
Vanderbilt.
So, but I will tell you what Iknow about it.
First, though, I want to backup because I want to make two
points about this.
Meta-analysis are kind of putthere as this, almost above,
(18:52):
like the gold standard of RCTs,right, you know, you have a
great, amazing meta-analysis andyou take what they say and,
like that's, you know the finalword on it, and we have to kind
of step back.
Science is nuanced, right?
What is the question you wantanswered?
First, Because I thinkmeta-analysis are important, but
(19:13):
they are not the be all, endall.
If, let's say, you are teachingand this is a very this isn't a
weird scenario that I'm going tobring up I think people might
think like, oh, she's justtrying to find an exception here
.
But if you're a teacher, wekind of know what works for the
majority of students, right, butyou might have that exception
student and I certainly didNothing was working and you know
(19:34):
.
So you type in, you want toknow what works for that student
.
Normal group studies, andespecially meta-analysis, aren't
going to be able to answer that.
Because of the way theyaggregate the studies and the
statistics, they don't even tellyou which is a problem some of
the demographics.
And so I want to get away fromthis idea that meta-analysis are
(19:54):
great.
They are great for when youwant to know general statements
about the average student.
They will never be able to tellyou about those exceptions that
you're trying to figure outwhat works for.
And that's that single case,research design I was talking
about.
And I do think this is my 10year bet that with better ways
of aggregating and figuring out,collating information, maybe
(20:16):
even with AI, machine learning,we're going to be able to get to
a place where you have like adropdown of, like this student
you know these are their scores.
Find me the single case studythat showed that worked for,
like increasing oral readingfluency, for that.
So, but that aside.
So that's the main point.
But let's say you do kind ofwant to know generally what
(20:37):
works.
Meta-analysis take severalstudies and they should all be
alike, similar in design, and ifthey're not, then you want to
figure out.
You can do like moderationanalysis later to figure out the
aspects that made the studies.
You know, maybe, maybe ifthere's variation, but you want
to choose similar studies withsimilar interventions and that
(21:02):
have a control.
So you want high qualitystudies is, I guess, is what I'm
trying to say, and that's, youknow, the inclusion criteria for
those really matters and shouldbe detailed.
There's this great vlog.
So Robert Slavin or Slavin, I'mnot sure how you pronounce this
thing he has these great seriesof blogs on meta-analyses and he
gives these sort of threethings that you wanna pay
(21:24):
attention to, or maybe fouractually.
But I would suggest thatreaders take a look at that
because some of those are kindof obvious, like red flags that
they can check for.
And I think one has to do asample size or inflated effect
sizes.
If you see meta-analyses thatare reporting really high, you
know above, like 0.5, 0.6.
And I think one in his examplehad like 1.2 effect, like given
(21:46):
effect size, which is crazylarge.
You wanna be cautious, you wannago back to those original
studies, see if they belongedthere, because like that one
that had a 1.2, I think he saysit was like an example of like
tennis, it didn't even have todo with like education.
So always go back to theoriginal source and researchers
(22:10):
need to be good about describingthe original studies in there
and the populations too.
But this idea of like kind ofgarbage in, garbage out holds
you really, robert was a bigproponent of high quality
meta-analyses and inclusionstandards and you should have
(22:30):
those, and I think he does as hehas.
Like that he ran the bestevidence in Cyclopedia B that
did high quality meta-analyses,and I think there was like one
other thing I wanted to sayabout oh, meta-analyses now are
becoming popular, but yeah,which is like two steps removed?
Yeah, so it's like yeah, well,I've seen it.
(22:54):
I've seen people kind of doingthese and I'm like, wait a
minute, like what was the con?
Like I always wanna see thedata right, and like the
original studies, like what werethey on?
To give me like a sense.
It's like knowing thepopulation, demographics of the
students in a study, right, well, the population here are
studies, because it's a study ofstudies and so in those you're
(23:16):
even further away, right, causelike you don't know so they're
just like compiling themeta-analyses.
Yes, they're taking several, butit's like a game of telephone,
almost in a way.
Right it gets distorted, so youjust have to be careful.
Meta-analyses are great whendone correctly.
Lori (23:35):
That sounds really great, like imagine if I was, if I
wasn't having this conversationwith you and I had heard that I
would be like, wow, that soundsreally fancy.
Meta, meta.
That must be super, super valid, you know, like even more valid
than the first kind of meta.
Neena Saha (23:51):
Right, not necessarily though.
Right Like education solved.
We can do meta-metas now Likewe know everything that works,
but it's not true.
It's a back to that match oflike the population too.
You have to know all thatinformation to see if it works
for your students.
Lori (24:07):
I am wondering if we just want to kind of leave teachers
with, as they're heading back toschool, right Like sharing some
reading, research, topics ofinterest or current research
that you're seeing.
Nina, we know that you'rereally great about knowing the
latest and greatest research.
Neena Saha (24:25):
Yeah, of course.
So I see so much research but Ithink my sort of the trends
that I'm seeing that I findinteresting are and this might
be because we kind of know nowphonics is explicit, systematic,
cumulative what we have to dofor that, and so I'm seeing a
lot more on morphology, fluencyand comprehension and
(24:48):
specifically morphology.
I find this fascinating and Ithink that's gonna be like a
trend, kind of alert that in thenext few years is gonna become
more important and we're gonnasee sort of more instruction and
programs, products around that,and which is good.
Because English ismorphophonemic.
We're not a true alphabeticlanguage.
(25:08):
Our words are composed of notjust sound information but sound
and meaning information andkind of a phrase that's used.
With morphology, a lot is likeislands of regularity.
Right, they provide thosemorphemes.
So morphemes, if the audiencedoesn't know, are just the
smallest unit of meaning in aword.
So, like with the word jumped,the jump part is the stem.
(25:31):
That, spelled as E-D is, youknow, like the suffix indicate,
like past tense and those.
So that's two morphemes and sothat island of regularity would
be that E-D, because we see thatagain.
In other words, a lot it's asound that can also.
You know that's prettyconsistent.
Sometimes it's the id, like incheat id versus jumped, but they
(25:56):
still kind of give you wordmeaning too.
So morphology is reallyinteresting to me and I've
linked, if the audience isinterested, to three sort of
studies on that of differentflavors.
So the first one is this reallycool kind of basic science
theoretical background modelingpaper and it's called executive
(26:17):
functions and morphologicalawareness explain the shared
variance between word readingand listening comprehension.
I really liked it because nowwe're seeing all these models
that kind of go beyond thesimple view model.
Right, simple view.
We've got, you know, wordreading skills, oral language,
you'll get good readingcomprehension.
But there's a lot more.
(26:38):
Simple is great.
We like parsimmonian science,but sometimes it's too simple
and so now we're trying tofigure out what are the other
factors.
And apparently morphology is abig one.
Actually it explains a lot ofvariants.
When I was reading the paper Iactually thought it was funny,
because you don't usually usewords like striking findings or
(26:58):
something.
And then the author did in thisone which I think is great that
that got through peer review,because usually we're always
hedging.
I wish I had the example ofthis other paper I had just read
, where it was potentially verymodest effects and I was like
that's science free.
But this paper was likestriking findings of morphology.
I don't have the quote in frontof me but it was an interesting
(27:20):
read.
So not an experimental study,it was correlational structural
equation modeling, but stillvery interesting.
Showing the morphology is animportant link between the two.
Melissa (27:33):
That's why I was just going to ask Nina, because I can
see morphology important forboth learning spelling, but also
comprehending.
But you're talking aboutthey're talking about both right
that?
It's like the bridge betweenExactly.
Neena Saha (27:46):
Yeah, and the active view model which is, I believe
and I'll, duke and Matt Burnshad a big meta and I'll
Cartwright OK, cartwright, ok.
And so they also havemorphology playing a big role.
I believe in the bridgingprocesses part, so it's good and
it makes sense why there's both, because it gives information
about meaning, but it's alsothere for sound, so it's like
(28:08):
the bridge.
And then the next study onmorphology that I'd like to
share is one that's a little bit, I hope, more practical for
teachers, because this was acorpus study, so a large
language.
What they did.
This was in the UK.
They took a large body ofchildren's literature and did
(28:28):
all these statistics on it, likecalculating frequency and stuff
.
And this paper is open accessand has a list of suffixes by
frequency, which I think it'salways great.
When the researchers put thatin, it's like let's go check and
see what are the ones I shouldmaybe teach.
It's really nice.
So I wanted to share that onethat's called effects of target
(28:49):
age and genre on morphologicalcomplexity in children's reading
material.
I didn't go do a deep dive onit because I don't think they
had their suffixes ranked byfrequency.
They had it done differently,so I don't know the top ones,
but I would recommend doing thatif you want to teach certain
high important morphologicalunits or suffixes in this case.
(29:09):
And then the last one onmorphology was an RCT.
So I found this one reallyinteresting.
Oh, and this is actually theone that only found very modest
and used very hedging wording.
So this one's an interventionstudy.
What they did was they comparedto third grade students in
Canada, but they were Englishspeaking and the title was
called Contrasting DirectInstruction in Morphological
(29:30):
Decoding and MorphologicalInquiry Analysis.
Interventions in Grade 3Children with Poor Morphological
Awareness.
So what they did is theystarted with about 163 students,
screened them on morphologicalawareness skills and only took
the ones with poor morphologicalawareness skills.
Then they had three conditions,two interventions.
(29:52):
One was this sort of directdecoding, one where it was
print-based, and the other wasinquiry-based where they asked
questions about like is the UNin uninvite important versus the
UN in uncle, kind of liketeaching those strategies.
And then they had the control,which was businesses usual and
(30:14):
that one.
They didn't have any sort ofmorphological training or
instruction in that one.
And a little bit disappointing Ithought they'd find they had
all these post-test measures, sothey had a post-right.
Immediately they tested on wordreading, reading vocabulary,
sentence comprehension, and theythought there were very modest
(30:35):
effects for, maybe, readingvocabulary, for the direct
decoding intervention, I believe, maybe one other, but not kind
of across the board largeeffects, and that the delayed
post-test none of the effectskind of stayed.
So if you spend all this timeand resources, you want to make
sure that the effects of theintervention last and at
(30:58):
post-test, I think none of themwere significant.
So kind of keep your eye onthis, though.
Even though it was just like itwasn't null effects across the
board, there's something there.
So I think we're going tofigure out how to better make
these more effective, thesemorphological awareness
interventions.
Melissa (31:18):
That's what I was just going to ask, nino, when you
were talking about it.
What do you do with that?
Because you just talked aboutother studies that did show that
it's important to teachmorphology, and I think that's
one of those things, too, whereyou're like it just makes sense.
It's like the brushing yourteeth example.
It just makes sense to teachour kids morphology, to make
sense of words, but then a studycomes along that doesn't show
(31:39):
much.
What do you do as a teacher?
Neena Saha (31:42):
Yeah, as a teacher, and that's why translation is so
tricky, because these are thestudies.
Right, we have good, solidtheoretical evidence, we've got
corpus that study about, we knowthere's a lot of morphines, we
know our language, but then whenit comes to doing the
intervention, it's like how comethere weren't great effects?
(32:06):
And I think it's just so hardto find good implementation like
translatable studies, andthat's just the short answer,
the kind of cop out answer thatwas like humans are messy,
there's so many variables goingon, there's so many things that
are tricky, but I think they'llget there.
And this is just one paper.
But the authors did say, eventhough it's very modest effects,
(32:27):
those strategies still work.
Like trying to tease that sortof word inquiry of figuring out
if the un and uninvited is thesame as on an uncle.
No, they're different, right,yeah, so that's a lot about
morphology.
Melissa (32:40):
Oh yeah, I was gonna ask if there was, because you
mentioned fluency andcomprehension.
Do you want to talk about those?
Neena Saha (32:45):
So fluency.
I think I've been seeing a lotabout readers theater and,
admittedly, I didn't really knowwhat that was, but it's pretty
much what it sounds like.
You get up there and you kindof like give Students line, and
you guys could jump in andcorrect me if I'm wrong, but
students get to kind of act outthe dialogue lines right, and so
I I saw a lot about this, Ishould say on, like these
Facebook groups, about you know,transforming, you know
(33:07):
decodable passages into, likereaders theater, things like oh,
that's interesting.
And then I was like, is thereany research on this?
Well, this study kind of cameout right as I was thinking
about this and it's a systematicreview and meta-analysis of the
readers theater impact on thedevelopment of reading skills,
and they found pretty largeeffects.
I didn't go through to see ifthere's red flags, but by
skimming it I think it's.
(33:29):
It looks legitimate to me andit's pure reviewed.
So that's a good one to kind oflook through to maybe see
different flavors ofinterventions for readers
theater Programs that work.
Melissa (33:39):
Yeah that's a nice one, because I think sometimes
people can think it's just liketrying to make it fun yeah, but
to know that it also has somevalidity behind it is good.
Neena Saha (33:50):
Agreed when it's fun and valid.
That's, and I'm especiallyexcited because it's a
meta-analysis.
Lori (33:56):
That is a good example of a valid meta-analysis that we're
giving everyone in the shownotes and I mean it's readers
theater, which is really, Ithink, fairly easy to implement.
So Fun, easy and valid, allreally good things.
Melissa (34:13):
Our friend Chase Young will be very excited.
Neena Saha (34:17):
And then I think I just put I have a few on
comprehension, not gonna go intothat, one's a survey.
Well, I guess I'll go into it alittle bit.
So once a survey study of kids,sorry, of teachers in Australia
, but I love survey studies.
I just love seeing what teacheryou know when people do this,
because we have to know whereteachers are at Before you can
you know, try and sort of quote,unquote, fix things or develop,
(34:37):
and so this one was interesting.
I think they found that andthey found this.
I think in like every surveystudy I have read about, no
matter the topic, that there'scontradicting answers.
So teachers will know a littlebit about something, but then,
like they'll contradictthemselves later, which is just
means I think we're not doing asgood as a job.
As you know, sciencecommunicators or researchers or
(34:59):
what have you getting thatInformation out there?
But so that one's there, Ithink it's open access.
Then I also, you know,sometimes there's with science
of reading, there's this focuson phonics and first graders,
and so for comprehension, I havethis one for fourth grade
students and it's in socialstudies and I think I actually
heard the first version of thisat triplets are last summer.
It's a professional developmentprogram to increase vocabulary
(35:23):
and comprehension in an embeddedlike social studies, fourth
grade.
So that's a cool one, I think.
If you know, teach olderstudents or Want to see how you
can do like readinginterventions and other you know
, not the content, not literacyblocks, so I tried to put
something for everyone.
And then the last thing I justwanted to touch on, which I
think also trend alert, likemorphology, I think is going to
(35:43):
be big, but using data topersonalize instruction.
You know, when you think oflike the five pillars, that's
not one of them, right, but itshould be embedded in like all
of them.
You should be using data, andso I linked to one about how
using targeted readinginstructions is better than you
know, just giving like aone-size-fits-all Right.
You want to use children's datato give them the right
(36:06):
intervention.
Lori (36:07):
So I put that one there.
Yeah, that's so important.
I think, nina, I think you'regoing to win the prize for the
longest show notes in history ofour podcast.
Is there anything else thatyou'd like to share with our
(36:28):
listeners before we, before wecome to a close?
Neena Saha (36:32):
No, I'm just grateful that you guys have me
on here and I think it's soimportant.
You know I may not have donelike the best job in explaining
I think some of these ideas thatare.
So, you know, science is sonuanced right, and I think we
just need more of moreDiscussion on this and more
people talking about the nuance,because it's really hard to
convey some of these no ideasand like causal inference and
(36:55):
what makes a study good and whenshould you trust it?
So no, I'm just grateful to behere and I don't have any.
There might be something laterwhere I add to the show notes
Like here's something I messedup.
Melissa (37:08):
I love that discussion about the nuance because I think
I'm seeing a lot of like I'mright, you're wrong kind of
battles happening versus likethese kind of discussions.
Okay, let's like talk through.
Yeah, let's talk through what,what this actually means.
Well, thank you, yeah of course.
Lori (37:24):
Yeah, this was really fun.
Thank you for being here.
Thanks for listening.
Literacy lovers, to stayconnected with us, sign up for
our email list at literacypodcast comm.
Melissa (37:38):
And to keep learning together.
Join the Melissa and Lori loveliteracy podcast Facebook group
and be sure to follow us onInstagram and Twitter.
Lori (37:48):
If this episode resonated with you, take a moment to share
with a teacher friend or leaveus a five-star rating and review
on Apple podcasts.
Melissa (37:58):
Just a quick reminder that the views and opinions
expressed by the hosts andguests of the Melissa and Lori
love literacy podcast are notnecessarily the opinions of
great minds, pbc or itsemployees.
Lori (38:10):
We appreciate you so much and we're so glad you're here to
learn with us.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Special Summer Offer: Exclusively on Apple Podcasts, try our Dateline Premium subscription completely free for one month! With Dateline Premium, you get every episode ad-free plus exclusive bonus content.
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!