June 26, 2024 • 22 mins
Dr. Jerry Valentine, a professor emeritus at the University of Missouri. Jerry brings a half century of knowledge and experience and is sharing the instructional practice inventory which involves an opportunity to collect data and communicate with educators about what is happening inside the classrooms to engage students.
For more information on this process, contact Dr. Jerry Valentine at ValentineJ@missouri.edu
For more information about the the topics discussed, contact us at
Bryan Wright: brwright44@gmail.com
Mark McBeth: mark@educationalrelevance.org
If you found value in today’s episode, share it with a colleague, subscribe, and leave us a review. It helps us keep bringing you conversations that matter.
Thanks for listening. Until next time, keep leading with heart—and stay educationally relevant.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
bryan-r-wright_1_04-11-2 (00:00):
Hello! Welcome to Educational
(00:01):
Relevance.
a platform for experiencededucators to give proven,
successful strategies ineducation and share those
thoughts and ideas with today'sleaders in the classroom and
administration and how we canhelp students of today.
My name is Brian Wright.
I am currently the adjunctprofessor at Concordia
University.
(00:22):
I'm alongside two fineindividuals today.
Mark Mcbeth, my partner incrime.
Mark is a former administratorand a leader of educational
circles around Missouri.
And Kansas.
And today we have a guest.
We have Dr.
Jerry Valentine, professoremeritus at, from the university
of Missouri.
(00:42):
Jerry brings a half century ofknowledge and experience to us
today.
And he's going to be discussingIPI and IPI stands for
instructional practiceinventory.
And I'm going to turn this overto my partner, Mark.
Mark, good morning
mark_1_04-11-2024_092147 (00:55):
Good morning.
So it's quite an honor.
Dr.
Valentine and I go back manyyears into the nineties when I
was a first year principal andthought I knew everything and.
Find out I didn't know much ofanything, but Dr.
Valentine introduced me to theinstructional practice
inventory.
it's an opportunity to collectdata and have a conversation
(01:17):
with educators about what we'redoing inside the classrooms to
engage students.
And with that Dr.
Valentine, how do we go aboutcollecting data for student
engagement?
And then how do people go aboutusing that data?
And why is that valuable?
squadcaster-6ia1_1_04-11- (01:36):
Mark, when we, when we built the
process and I had theopportunity to do it with one of
my former doctoral students,Brian Painter and he managed to
sneak a dissertation out of ourearly work too, which was a win
win for him.
And for all of us, Brian and Iput our heads together to see if
we could produce a way ofgetting in and out of classrooms
and having insight about howkids are thinking, how they're
(01:58):
engaged.
was all part of a largecomprehensive systemic school
improvement project that Idirected at Mizzou was designed
to work with elementary, middle,and high schools and help them
look at the big picture ofeverything they're doing in
school culture, school climate,principal leadership, teacher
leadership, curriculuminstruction and assessment, how
kids learn.
(02:18):
we worked with multiple schoolsaround the state for many years.
engaging with those schools andtrying to help them refine what
they were working on so thatwe'd have better outcomes for
kids.
Well, in designing that in 1995,one of the things that I thought
would be really helpful would beto have a process that would
allow us to understand how kidsare thinking as they go through
(02:39):
a class to collectively look atthat from a school wide picture
because all the work we weredoing on school comprehensive
school improvement was designedto be school wide.
We worked at first trying to getin and out of classrooms and
codify student engagementstrategies.
Are kids involved in cooperativelearning?
Are they doing some kind ofinquiry et cetera?
(03:01):
And we found out as we playedwith that for a few months that
that really was we were barkingup the wrong tree.
That really was not it.
Detailed enough.
It wasn't giving us insight.
You expect to put kids in acooperative, collaborative
learning experience.
You expect them to be tackling areal problem of some sort, and
you don't necessarily alwayshave higher or deeper thinking
going on.
we settled in on the notion thatwe were going to have to fine
(03:23):
tune this and we built thataround basically.
Six strategies two of thosefocus on higher deeper thinking,
two of them focus on lowersurface thinking, and one of
them focuses on disengagement.
And to get make that happen.
we had to figure out.
How do we get in and out of aclassroom with a data collector
who is trained to collect thedata and get in and around the
(03:45):
classroom without and be asunobtrusive as possible without
interrupting without, creatingdistractions.
And then slip out of thatclassroom and codify the
majority of the kids werethinking or how all the kids
were thinking, depending onwhether it was the same process
or whether it were differentthought processes going on in
the classroom.
So it took us, a better part ofa few months to really.
(04:08):
Get the protocols down and to beable to figure out how to make
that happen.
But we finally did, we found outthat we could train teacher
leaders to do this, we wanted tomake the process a teacher
driven process, not anadministrator driven process.
We wanted the teachers to be thecollectors that went to team of
teachers to be the collectors ofthe data.
We want a team of teachers to bethe ones who.
Who looked at the data once theyhad collected it and that first
(04:31):
look and decide, how do we wantto engage the faculty with the
data and the same teacherleaders then were responsible
for leading the faculty in thestudy of the data with a whole
notion that if we could look ata big profile of how our kids
are thinking across the schoolday.
Which is something I thoughtmany years earlier about,
wouldn't it be nice as aprincipal to have that?
Wouldn't it be nice to, as ateacher to have that so I could
(04:52):
look at myself and think how Ifit into the impact that we're
having on all the kids in theschool.
we built that process so thatthe teacher leaders would be the
ones who engage their colleaguesin the study of the data,
getting in and out of theclassroom without being
disruptive and having a realgood handle on how kids are
thinking meant that.
collector walking into theclassroom would have to stand
(05:14):
there at the door for just amoment and scan the room and get
a big picture of how everyone'sengaged and then quickly move in
around the room and among thestudents, sometimes leaning over
and visiting with students aboutwhat they're doing and how
they're thinking, sometimessimply looking at the work that
they're processing, sometimesstanding back of the room and
letting the teacher talk to thekids and lead as teachers do,
(05:35):
and as they do quite a bit ofthe time.
But it took a while to figureout that process.
But eventually we felt we had itdown to where we could have
validity, accuracy of the codingprocess.
We could have reliability,consistent accuracy within the
data collector's own time spentcollecting, and iterator
(05:56):
reliability, which was to theThe consistency of accuracy
across data collectors asmultiple teachers help to
collect the data across theschool.
so that's basically what we did.
Now, we put it in the hands ofthe teachers into me with the
faculty and to engage thefaculty on reflective thought
processes about.
The data profile that they werelooking at, or are we
(06:19):
comfortable with the amount oftime our kids spend doing higher
or deeper thinking?
Are we comfortable with theamount of time our kids are
spending doing or our surfacethinking?
Do we really like the percent ofdisengagement that we have here
within the classrooms across ourschool?
Do we really like the headcounts of how many kids are
disengaged and and how they'rethinking and bubbling up those
kind of conversations after eachdata collection, we found we
(06:39):
need to collect data 4 times ayear.
One data collection a yearreally didn't even.
Moved the needle in terms ofgrowth and change and two data
collections started to build avocabulary.
Three data collections moved theneedle and moved it
statistically significantly.
We were seeing significantincreases in higher deeper
thinking, significant decreasesin disengagement.
And yet we also knew that if wetried four, we thought we might
(07:02):
have something better and wedid.
It moved the needle a littlefurther.
we had schools that said, okay,if four good, then let's do 8, 9
a year, every month, why not?
But I found out when I looked atthe data on that, that poor data
collection seems to tap out.
It starts to lose itsmeaningfulness to the faculty
when they're looking at it toooften and having too much to too
(07:23):
much time spent on talking aboutwhen they have other things to
do.
So, 4 data collections a yearseem to move the needle and 3
certainly did as well.
So that's what we've beenrecommending to schools for
some, you know, 2 and a halfdecades now.
bryan-r-wright_1_04-11-2 (07:36):
Jerry, when you say four times zero, is
that quarterly?
And
squadcaster-6ia1_1_04 (07:40):
basically quarterly is what we suggest to
the schools.
Yeah, somewhere about around themiddle of the quarter is usually
the best kind of time to slip in
mark_1_04-11-2024_092147 (07:46):
How many data points are being
collected in a particular day?
squadcaster-6ia1_1_04-1 (07:50):
Depends on the size of the school
because what you'd like to haveis all classrooms.
We collect the data with asystematic proportionate
sampling process that the datacollector starts at a point on
the school map.
Goes down that hall, classroom,classroom, classroom, turns a
corner, classroom, classroom,turns a corner, classroom,
classroom, turns a corner,classroom, climbs the stairs,
and repeats.
(08:11):
And you repeat that pattern, yourepeat that pattern, you repeat
that pattern, you repeat thatpattern all day from the
beginning of the school daytill, till the end of the school
day.
And at the end of a school day,In an elementary school, for
example, maybe a typical midsized elementary school of four
or 500 students, going to havesomewhere in the neighborhood of
120, 130 data points in a largerschool, let's say a large high
(08:34):
school with 1500 kids or 2000kids, you would have multiple
data collectors collecting data,keeping distance from each
other, because now you've got tohave a pool rich enough to be
valid.
And that means you're going toneed 250, 300 data points, 400
data points from a school thatsize.
I can remember in one of thelargest schools, middle schools
(08:54):
in the country, we collecteddata.
We had 5 or 6 data collectorsworking at the same time
because.
Because of the nearly 3, 000students that were in that
school at the time.
So, you've got to have a largedata pool.
And they were collecting 1, 000data points.
But most schools, 125, 150smaller elementary schools,
marching that on up to 150, 175,to 200 or more,
mark_1_04-11-2024_092147 (09:19):
are we looking at middle school,
elementary, high school?
Where does IPI come in at?
What grade level?
squadcaster-6ia1_1_04-11-202 (09:26):
We built the process in the pilot
with elementary, middle, andhigh schools.
We tested it with elementary,middle, and high schools.
And we've been using it since1996, November, when I first did
a collection with elementary,middle, and high schools,
because that project that itcame out of was a support system
for 10 elementary, 10 middle,and 10 high schools.
(09:46):
We were going to be working withthose 30 schools for the next
two and a half years from myoffice at the university do that
comprehensive schoolimprovement.
So we found out it works acrossthe grade level and you get very
different kinds of results toowhen you start thinking about.
Engagement and disengagementzone across those grade levels.
But, but that's what we wantedto do.
And that's what we were able tofigure out how to do with
(10:08):
validity.
bryan-r-wright_1_04-11-202 (10:09):
Now, who are the data collectors
again?
I missed that one, Jerry.
squadcaster-6ia1_1_04-11-20 (10:12):
Now we train teacher leaders to be
data collectors.
teacher leaders who are willingto put in just a little bit of
time to be a part of thisprocess.
And as I tell them when theystart thinking about do they
want to do this or not thatlook, if you, Pitch in and do
this, and you help yourcolleagues look at how kids are
thinking, and you do that for afew years, you're going to look
(10:34):
back after those few years, andyou're going to feel pretty good
about your contribution to wherethat thought process moved from
when you started to where thatprocess ends up whenever you
stop and look back at itlongitudinally.
And the longitudinal data thatwe have supports that as well.
bryan-r-wright_1_04-11-20 (10:50):
added on questions.
So this is not an evaluativetool.
So teachers are prettycomfortable about them coming in
and doing this.
squadcaster-6ia1_1_04-11-20 (10:57):
No, we've told the principals, you
need to know about this process.
understand it very deeply, butyou have to simply stay on the
sidelines and support it it hasto be a teacher driven process.
And it's not about evaluation.
It's not about supervision.
It's not about administrativewalkthroughs.
It's totally about a colleaguegoing into your classroom.
on the kids, not making any kindof code about you as a teacher.
(11:21):
This is all about the kids andhow they're thinking.
A teacher's name is not writtendown.
A room number is not writtendown.
It's an anonymous pool of dataabout kids thinking across the
whole school day.
So that, you really try to takethe, I got you.
Kind of mentality out of it.
I call it the jazz it up effect.
We, we were trying to do ourbest from the beginning to take
(11:41):
the jazz it up effect out ofthis process because the more we
can get teachers to relax andtrust that these are their data.
more we're going to get what'stypically happening day by day
by day in these four datacollections each year, there
mark_1_04-11-2024_092147 (11:55):
why is it valuable?
Above and beyond assessmentdata?
So if we're doing summative orformative type of assessments
with students saying, did theylearn the standard or not?
Why this data in addition tothat?
squadcaster-6ia1_1_04-11-20 (12:11):
are very few ways to get this kind
of data and very few people havegone after this and felt like
they had have done it with alevel of You really have to
train the teachers thoroughly onhow to get, collect good data.
you have good data, now, we geta picture of how much, higher
(12:32):
and deeper thinking is takingplace in the school.
Well, and deeper thinking isreally critical to student
learning.
we've got decades of data fromscholars who've said that as you
increase higher and deeperthinking in the classroom for
the kids, you increase academicsuccess.
it's a straight linerelationship in almost every
study that you look at, andwe've been looking at, and I've
been trying to read all of thatliterature for years.
(12:54):
You know, I come from abackground of school leadership
and preparing principals, but tome, principals have to be
instructional leaders, and thatmeans they have to understand
all of this.
So that's what I focus on in mywhole career.
if we know that we want toincrease higher, deeper thinking
for the kids, then why is thatso important?
Well, number one leads to morepositive academic success.
(13:16):
Number 2, you look at it as away of understanding what is
expected of our students oncethey've been with us for 12, 13
years.
And when they go out into theworkforce, the national reports
that come out of of theworkforce tell us consistently
that, economically, they wantcertain kinds of skills from
their graduates to contribute tothe economy and, to, for
(13:39):
example, let me give you citesome, some examples here from
the National Forum and Forbessurvey, Forbes studies, surveys,
That have come out in the lastseveral years.
In fact, they usually do theseevery four or five, six years.
Here are the top eight skills,in, in the top 10, the ability
to be analytical, criticalthinking, creativity, curiosity,
(14:01):
judgment, innovative.
Complex problem solving, andcontinuous learning.
All of those are linked back tohigher or deeper thinking.
Because when we talk about theconcept of higher deeper
thinking in the InstructionalPractices Inventory, the IPI,
talking about how we help kidsto be more, critical, more
analytical, more creative to bebetter problem solvers, to be
(14:23):
more complex thinkers, to thinkmore deeply, more critically,
more reflectively, moreanalytically, more creatively.
I roll that off of my tongue allthe time, multiple times a day.
It's what our definition hasbeen we started, and it's what
basically the general definitionof higher thinking has been.
Whether you go back to Dewey andin the beginning of the century,
(14:44):
whether you move to Bloom in themiddle of the century, last
century, whether you move toHattie and others who have
really been contributingsignificantly to the literature
in the last, in the last fewdecades, that's basically the
essence of higher, deeperthinking.
that's what we're trying tocodify it with it with our
observational processes in theclassroom.
(15:05):
How much higher deeper thing dowe have?
How do we grow it?
much disengagement do we have?
How do we reduce it?
We need a certain amount oflower order surface thinking to
build fundamental skills and,and, and recall and practice
and, and drive home those innatekinds of skills that, that are
needed in education.
So, 3 big pictures, higher ordeeper, lower order surface,
(15:26):
disengagement, what the IPIprovides for a faculty to study.
bryan-r-wright_1_04-11-2024_ (15:32):
as you're talking about training
teachers, let's go back to thebasics of Getting the right
people on the bus.
As they say, when you talk aboutJim Cullen, getting the right
people on the bus, how do youget the proper teacher leaders
so you can reduce the subjectiveresults that teachers want to
provide and get something a bitmore objective, what you're
trying to attain with students.
squadcaster-6ia1_1_04-11-202 (15:52):
We ask the school leaders to or ask
teacher leaders who have certainkind of qualities.
Really good understanding of theteaching learning process.
Respected by their peers,willing to put in a little bit
of extra time, not a lotrequired, but just a little bit
demonstrated leadership amongthe faculty and, and then
(16:15):
openness, openness to look atand, and, and consider something
like this.
And lastly, and finally, theability to stand up in front of
their colleagues and help themengaged with some meaningful
data.
So to get them to the skilllevel that they can do that
well, we're looking at abasically an eight hour workshop
(16:35):
eight o'clock to four o'clock.
We start that workshop with theoverview with background
reading.
We move straight into codingexamples of kids in the
classroom on the written paper.
by lunchtime, we're finishingthat process up.
And by early afternoon, we'reout in classrooms doing guided
practice and debriefing as wewalk them through how to build
the skills of getting in thatclassroom.
(16:58):
It's a it's a lengthy process.
And at the end of the day, andwe also sneak in an hour also.
Near the middle of the afternoonof how do you lead your
colleagues in the state of thedata?
Because, frankly, that's one ofthe toughest jobs for that team
of teacher leaders to engagetheir colleagues meaningfully in
the data.
And then, at the end of the day,they take a written assessment
and code and code and code forabout 3540 minutes.
(17:21):
And I take the codes back to myoffice here and, and go through
them and send them, give themfeedback about their coding
skills and their codingaccuracy.
That's how we get those peopleto that.
Then when we start them out fortheir first data collection, we
recommend they pair withsomebody.
If the school's already usingthe process and they're being
added to the team, we recommendthey pair with a veteran data
collector, if not, then tworookie data collectors pair
(17:43):
together and they work like thatfor the first data collection
day or two, and then they moveoff solo.
And as soon as they move offsolo, we're off and running
because now we've got that manymore heads out there to get
deeper, richer pools of datafrom each school.
bryan-r-wright_1_04-11-20 (17:59):
Okay.
And then the second questionconnect the dots as it were,
between the summativeevaluations, such as maybe an
assessment for learningstrategies that you may want to
use with kids.
So the kids are getting part ofwhat they're doing with the fire
of higher earning and how theycan define what they're learning
in the classroom.
But then with the summativeevaluations, such as
standardized testing, how doesIPI affect positively the
(18:20):
summative testing strategies inschools?
squadcaster-6ia1_1_04-11-20 (18:23):
had the chance to do a lot of
studies with that, includingmultiple longitudinal looks at
the data.
We also had the chance to lookat the data across multiple
states.
we have consistent, and we'vegone to pretty sophisticated
statistical treatments,hierarchical linear modeling,
structural equation, equationmodeling.
We're way past the notion ofjust simple correlations and
simple regressions in terms oftrying to look at all of this
(18:45):
over the years.
we've written our papers, andWe've presented them at national
conferences and we put them upon the, website, et cetera.
So, we've got the researchgrounding behind that, but, what
we found is basically that ashigher deeper thinking
increases.
So too does academic success inschools.
(19:05):
also found that the big tailwagging the dog for a lot of
student learning isdisengagement and the amount of
time that kids are disengagedand not engaged with the
learning process in theclassroom that disengagement.
As it goes up, academic successgoes down.
And the ratio that we found overthat in our studies has been
(19:27):
about two to one for every twopercent increase in disengaged
learning time in a school.
That school year, The school canexpect to have lost one more
percent of a student bodyfailing the high stakes test,
meaning not meeting standards.
So, the 2 to 1 ratio, as we'vecalled it over the years in our,
(19:48):
in our work with the IPI is, isbasically shown you, you've
addressed this engagement and,you also have to address higher,
deeper thinking, you have to dothem both at the same time.
And we've also foundmathematically that it takes
about four units of higher,deeper thinking.
To neutralize one unit ofdisengagement.
(20:09):
Now, all of that's a good oldstatistical fancy stuff that we
do, but it's, informative and wesee it in the real world too.
When we start looking at theschools, I've had some real fun
with schools by plotting IPIlongitudinal data, their
statewide assessment data andany other kinds of assessment
data that they have.
Doing plot lines and going upand circling Where do you see
(20:30):
the dip?
Where do you see a growth?
Why is that happening?
Let's have a facultyconversation about those kinds
of things went past beyond justour leadership team.
So it's meant to be a real handson tool for schools to use that
puts him in the driver's seat.
Both in terms of collecting thedata and in learning from the
(20:51):
data.
bryan-r-wright_1_04-11-20 (20:53):
Okay.
you,
mark_1_04-11-2024_092147 (20:55):
So the conversation today was about I P
I instructional practiceinventory and how we collect
data.
And what's that data is usedfor.
It's uniquely different datathan our assessment scores, but
your question, Brian, aboutmaking that correlation between.
Using data like this and thendata like student assessments is
(21:16):
that there's a link.
And so that conversation aboutthe research that says, if we're
doing higher, deeper higherorder questioning, deeper
thinking there's going to be acorrelation in our assessment
scores.
We have to have teachers havingthat dialogue.
Sounds like what Jerry's sayingis we, we got to have teachers
have dialogue around that data.
And if they're having dialoguearound that data, then they can
(21:37):
make changes.
Dr.
Valentine, if people want toreach out to you to learn a
little bit about this, go to IPIstudent engagement.
com.
Would that be about right?
squadcaster-6ia1_1_04-11 (21:47):
That's a good place to go and look at
our work and look at ourresearch and all.
And then of course by email thebest place to, to get ahold of
mark_1_04-11-2024_092147 (21:55):
Okay.
squadcaster-6ia1_1_04 (21:55):
certainly
mark_1_04-11-2024_092147 (21:56):
Very good.
We'll, provide that for everyoneas well.
Anything else, Brian?
bryan-r-wright_1_04-11-2024_0 (22:00):
I think for this session, I think
we're about done.
I want to say thank you, Dr.
Valentine,
squadcaster-6ia1_1_04 (22:05):
pleasure.
bryan-r-wright_1_04-11-2024_ (22:05):
in the meantime, I want to say
thank you all.
Thank both of you for yourparticipation today on education
relevance.
Have a good day now.
Bye bye.
Hello! Welcome to Educational
(00:01):
Relevance.
a platform for experiencededucators to give proven,
successful strategies ineducation and share those
thoughts and ideas with today'sleaders in the classroom and
administration and how we canhelp students of today.
My name is Brian Wright.
I am currently the adjunctprofessor at Concordia
University.
(00:22):
I'm alongside two fineindividuals today.
Mark Mcbeth, my partner incrime.
Mark is a former administratorand a leader of educational
circles around Missouri.
And Kansas.
And today we have a guest.
We have Dr.
Jerry Valentine, professoremeritus at, from the university
of Missouri.
(00:42):
Jerry brings a half century ofknowledge and experience to us
today.
And he's going to be discussingIPI and IPI stands for
instructional practiceinventory.
And I'm going to turn this overto my partner, Mark.
Mark, good morning
mark_1_04-11-2024_092147 (00:55):
Good morning.
So it's quite an honor.
Dr.
Valentine and I go back manyyears into the nineties when I
was a first year principal andthought I knew everything and.
Find out I didn't know much ofanything, but Dr.
Valentine introduced me to theinstructional practice
inventory.
it's an opportunity to collectdata and have a conversation
(01:17):
with educators about what we'redoing inside the classrooms to
engage students.
And with that Dr.
Valentine, how do we go aboutcollecting data for student
engagement?
And then how do people go aboutusing that data?
And why is that valuable?
squadcaster-6ia1_1_04-11- (01:36):
Mark, when we, when we built the
process and I had theopportunity to do it with one of
my former doctoral students,Brian Painter and he managed to
sneak a dissertation out of ourearly work too, which was a win
win for him.
And for all of us, Brian and Iput our heads together to see if
we could produce a way ofgetting in and out of classrooms
and having insight about howkids are thinking, how they're
(01:58):
engaged.
was all part of a largecomprehensive systemic school
improvement project that Idirected at Mizzou was designed
to work with elementary, middle,and high schools and help them
look at the big picture ofeverything they're doing in
school culture, school climate,principal leadership, teacher
leadership, curriculuminstruction and assessment, how
kids learn.
(02:18):
we worked with multiple schoolsaround the state for many years.
engaging with those schools andtrying to help them refine what
they were working on so thatwe'd have better outcomes for
kids.
Well, in designing that in 1995,one of the things that I thought
would be really helpful would beto have a process that would
allow us to understand how kidsare thinking as they go through
(02:39):
a class to collectively look atthat from a school wide picture
because all the work we weredoing on school comprehensive
school improvement was designedto be school wide.
We worked at first trying to getin and out of classrooms and
codify student engagementstrategies.
Are kids involved in cooperativelearning?
Are they doing some kind ofinquiry et cetera?
(03:01):
And we found out as we playedwith that for a few months that
that really was we were barkingup the wrong tree.
That really was not it.
Detailed enough.
It wasn't giving us insight.
You expect to put kids in acooperative, collaborative
learning experience.
You expect them to be tackling areal problem of some sort, and
you don't necessarily alwayshave higher or deeper thinking
going on.
we settled in on the notion thatwe were going to have to fine
(03:23):
tune this and we built thataround basically.
Six strategies two of thosefocus on higher deeper thinking,
two of them focus on lowersurface thinking, and one of
them focuses on disengagement.
And to get make that happen.
we had to figure out.
How do we get in and out of aclassroom with a data collector
who is trained to collect thedata and get in and around the
(03:45):
classroom without and be asunobtrusive as possible without
interrupting without, creatingdistractions.
And then slip out of thatclassroom and codify the
majority of the kids werethinking or how all the kids
were thinking, depending onwhether it was the same process
or whether it were differentthought processes going on in
the classroom.
So it took us, a better part ofa few months to really.
(04:08):
Get the protocols down and to beable to figure out how to make
that happen.
But we finally did, we found outthat we could train teacher
leaders to do this, we wanted tomake the process a teacher
driven process, not anadministrator driven process.
We wanted the teachers to be thecollectors that went to team of
teachers to be the collectors ofthe data.
We want a team of teachers to bethe ones who.
Who looked at the data once theyhad collected it and that first
(04:31):
look and decide, how do we wantto engage the faculty with the
data and the same teacherleaders then were responsible
for leading the faculty in thestudy of the data with a whole
notion that if we could look ata big profile of how our kids
are thinking across the schoolday.
Which is something I thoughtmany years earlier about,
wouldn't it be nice as aprincipal to have that?
Wouldn't it be nice to, as ateacher to have that so I could
(04:52):
look at myself and think how Ifit into the impact that we're
having on all the kids in theschool.
we built that process so thatthe teacher leaders would be the
ones who engage their colleaguesin the study of the data,
getting in and out of theclassroom without being
disruptive and having a realgood handle on how kids are
thinking meant that.
collector walking into theclassroom would have to stand
(05:14):
there at the door for just amoment and scan the room and get
a big picture of how everyone'sengaged and then quickly move in
around the room and among thestudents, sometimes leaning over
and visiting with students aboutwhat they're doing and how
they're thinking, sometimessimply looking at the work that
they're processing, sometimesstanding back of the room and
letting the teacher talk to thekids and lead as teachers do,
(05:35):
and as they do quite a bit ofthe time.
But it took a while to figureout that process.
But eventually we felt we had itdown to where we could have
validity, accuracy of the codingprocess.
We could have reliability,consistent accuracy within the
data collector's own time spentcollecting, and iterator
(05:56):
reliability, which was to theThe consistency of accuracy
across data collectors asmultiple teachers help to
collect the data across theschool.
so that's basically what we did.
Now, we put it in the hands ofthe teachers into me with the
faculty and to engage thefaculty on reflective thought
processes about.
The data profile that they werelooking at, or are we
(06:19):
comfortable with the amount oftime our kids spend doing higher
or deeper thinking?
Are we comfortable with theamount of time our kids are
spending doing or our surfacethinking?
Do we really like the percent ofdisengagement that we have here
within the classrooms across ourschool?
Do we really like the headcounts of how many kids are
disengaged and and how they'rethinking and bubbling up those
kind of conversations after eachdata collection, we found we
(06:39):
need to collect data 4 times ayear.
One data collection a yearreally didn't even.
Moved the needle in terms ofgrowth and change and two data
collections started to build avocabulary.
Three data collections moved theneedle and moved it
statistically significantly.
We were seeing significantincreases in higher deeper
thinking, significant decreasesin disengagement.
And yet we also knew that if wetried four, we thought we might
(07:02):
have something better and wedid.
It moved the needle a littlefurther.
we had schools that said, okay,if four good, then let's do 8, 9
a year, every month, why not?
But I found out when I looked atthe data on that, that poor data
collection seems to tap out.
It starts to lose itsmeaningfulness to the faculty
when they're looking at it toooften and having too much to too
(07:23):
much time spent on talking aboutwhen they have other things to
do.
So, 4 data collections a yearseem to move the needle and 3
certainly did as well.
So that's what we've beenrecommending to schools for
some, you know, 2 and a halfdecades now.
bryan-r-wright_1_04-11-2 (07:36):
Jerry, when you say four times zero, is
that quarterly?
And
squadcaster-6ia1_1_04 (07:40):
basically quarterly is what we suggest to
the schools.
Yeah, somewhere about around themiddle of the quarter is usually
the best kind of time to slip in
mark_1_04-11-2024_092147 (07:46):
How many data points are being
collected in a particular day?
squadcaster-6ia1_1_04-1 (07:50):
Depends on the size of the school
because what you'd like to haveis all classrooms.
We collect the data with asystematic proportionate
sampling process that the datacollector starts at a point on
the school map.
Goes down that hall, classroom,classroom, classroom, turns a
corner, classroom, classroom,turns a corner, classroom,
classroom, turns a corner,classroom, climbs the stairs,
and repeats.
(08:11):
And you repeat that pattern, yourepeat that pattern, you repeat
that pattern, you repeat thatpattern all day from the
beginning of the school daytill, till the end of the school
day.
And at the end of a school day,In an elementary school, for
example, maybe a typical midsized elementary school of four
or 500 students, going to havesomewhere in the neighborhood of
120, 130 data points in a largerschool, let's say a large high
(08:34):
school with 1500 kids or 2000kids, you would have multiple
data collectors collecting data,keeping distance from each
other, because now you've got tohave a pool rich enough to be
valid.
And that means you're going toneed 250, 300 data points, 400
data points from a school thatsize.
I can remember in one of thelargest schools, middle schools
(08:54):
in the country, we collecteddata.
We had 5 or 6 data collectorsworking at the same time
because.
Because of the nearly 3, 000students that were in that
school at the time.
So, you've got to have a largedata pool.
And they were collecting 1, 000data points.
But most schools, 125, 150smaller elementary schools,
marching that on up to 150, 175,to 200 or more,
mark_1_04-11-2024_092147 (09:19):
are we looking at middle school,
elementary, high school?
Where does IPI come in at?
What grade level?
squadcaster-6ia1_1_04-11-202 (09:26):
We built the process in the pilot
with elementary, middle, andhigh schools.
We tested it with elementary,middle, and high schools.
And we've been using it since1996, November, when I first did
a collection with elementary,middle, and high schools,
because that project that itcame out of was a support system
for 10 elementary, 10 middle,and 10 high schools.
(09:46):
We were going to be working withthose 30 schools for the next
two and a half years from myoffice at the university do that
comprehensive schoolimprovement.
So we found out it works acrossthe grade level and you get very
different kinds of results toowhen you start thinking about.
Engagement and disengagementzone across those grade levels.
But, but that's what we wantedto do.
And that's what we were able tofigure out how to do with
(10:08):
validity.
bryan-r-wright_1_04-11-202 (10:09):
Now, who are the data collectors
again?
I missed that one, Jerry.
squadcaster-6ia1_1_04-11-20 (10:12):
Now we train teacher leaders to be
data collectors.
teacher leaders who are willingto put in just a little bit of
time to be a part of thisprocess.
And as I tell them when theystart thinking about do they
want to do this or not thatlook, if you, Pitch in and do
this, and you help yourcolleagues look at how kids are
thinking, and you do that for afew years, you're going to look
(10:34):
back after those few years, andyou're going to feel pretty good
about your contribution to wherethat thought process moved from
when you started to where thatprocess ends up whenever you
stop and look back at itlongitudinally.
And the longitudinal data thatwe have supports that as well.
bryan-r-wright_1_04-11-20 (10:50):
added on questions.
So this is not an evaluativetool.
So teachers are prettycomfortable about them coming in
and doing this.
squadcaster-6ia1_1_04-11-20 (10:57):
No, we've told the principals, you
need to know about this process.
understand it very deeply, butyou have to simply stay on the
sidelines and support it it hasto be a teacher driven process.
And it's not about evaluation.
It's not about supervision.
It's not about administrativewalkthroughs.
It's totally about a colleaguegoing into your classroom.
on the kids, not making any kindof code about you as a teacher.
(11:21):
This is all about the kids andhow they're thinking.
A teacher's name is not writtendown.
A room number is not writtendown.
It's an anonymous pool of dataabout kids thinking across the
whole school day.
So that, you really try to takethe, I got you.
Kind of mentality out of it.
I call it the jazz it up effect.
We, we were trying to do ourbest from the beginning to take
(11:41):
the jazz it up effect out ofthis process because the more we
can get teachers to relax andtrust that these are their data.
more we're going to get what'stypically happening day by day
by day in these four datacollections each year, there
mark_1_04-11-2024_092147 (11:55):
why is it valuable?
Above and beyond assessmentdata?
So if we're doing summative orformative type of assessments
with students saying, did theylearn the standard or not?
Why this data in addition tothat?
squadcaster-6ia1_1_04-11-20 (12:11):
are very few ways to get this kind
of data and very few people havegone after this and felt like
they had have done it with alevel of You really have to
train the teachers thoroughly onhow to get, collect good data.
you have good data, now, we geta picture of how much, higher
(12:32):
and deeper thinking is takingplace in the school.
Well, and deeper thinking isreally critical to student
learning.
we've got decades of data fromscholars who've said that as you
increase higher and deeperthinking in the classroom for
the kids, you increase academicsuccess.
it's a straight linerelationship in almost every
study that you look at, andwe've been looking at, and I've
been trying to read all of thatliterature for years.
(12:54):
You know, I come from abackground of school leadership
and preparing principals, but tome, principals have to be
instructional leaders, and thatmeans they have to understand
all of this.
So that's what I focus on in mywhole career.
if we know that we want toincrease higher, deeper thinking
for the kids, then why is thatso important?
Well, number one leads to morepositive academic success.
(13:16):
Number 2, you look at it as away of understanding what is
expected of our students oncethey've been with us for 12, 13
years.
And when they go out into theworkforce, the national reports
that come out of of theworkforce tell us consistently
that, economically, they wantcertain kinds of skills from
their graduates to contribute tothe economy and, to, for
(13:39):
example, let me give you citesome, some examples here from
the National Forum and Forbessurvey, Forbes studies, surveys,
That have come out in the lastseveral years.
In fact, they usually do theseevery four or five, six years.
Here are the top eight skills,in, in the top 10, the ability
to be analytical, criticalthinking, creativity, curiosity,
(14:01):
judgment, innovative.
Complex problem solving, andcontinuous learning.
All of those are linked back tohigher or deeper thinking.
Because when we talk about theconcept of higher deeper
thinking in the InstructionalPractices Inventory, the IPI,
talking about how we help kidsto be more, critical, more
analytical, more creative to bebetter problem solvers, to be
(14:23):
more complex thinkers, to thinkmore deeply, more critically,
more reflectively, moreanalytically, more creatively.
I roll that off of my tongue allthe time, multiple times a day.
It's what our definition hasbeen we started, and it's what
basically the general definitionof higher thinking has been.
Whether you go back to Dewey andin the beginning of the century,
(14:44):
whether you move to Bloom in themiddle of the century, last
century, whether you move toHattie and others who have
really been contributingsignificantly to the literature
in the last, in the last fewdecades, that's basically the
essence of higher, deeperthinking.
that's what we're trying tocodify it with it with our
observational processes in theclassroom.
(15:05):
How much higher deeper thing dowe have?
How do we grow it?
much disengagement do we have?
How do we reduce it?
We need a certain amount oflower order surface thinking to
build fundamental skills and,and, and recall and practice
and, and drive home those innatekinds of skills that, that are
needed in education.
So, 3 big pictures, higher ordeeper, lower order surface,
(15:26):
disengagement, what the IPIprovides for a faculty to study.
bryan-r-wright_1_04-11-2024_ (15:32):
as you're talking about training
teachers, let's go back to thebasics of Getting the right
people on the bus.
As they say, when you talk aboutJim Cullen, getting the right
people on the bus, how do youget the proper teacher leaders
so you can reduce the subjectiveresults that teachers want to
provide and get something a bitmore objective, what you're
trying to attain with students.
squadcaster-6ia1_1_04-11-202 (15:52):
We ask the school leaders to or ask
teacher leaders who have certainkind of qualities.
Really good understanding of theteaching learning process.
Respected by their peers,willing to put in a little bit
of extra time, not a lotrequired, but just a little bit
demonstrated leadership amongthe faculty and, and then
(16:15):
openness, openness to look atand, and, and consider something
like this.
And lastly, and finally, theability to stand up in front of
their colleagues and help themengaged with some meaningful
data.
So to get them to the skilllevel that they can do that
well, we're looking at abasically an eight hour workshop
(16:35):
eight o'clock to four o'clock.
We start that workshop with theoverview with background
reading.
We move straight into codingexamples of kids in the
classroom on the written paper.
by lunchtime, we're finishingthat process up.
And by early afternoon, we'reout in classrooms doing guided
practice and debriefing as wewalk them through how to build
the skills of getting in thatclassroom.
(16:58):
It's a it's a lengthy process.
And at the end of the day, andwe also sneak in an hour also.
Near the middle of the afternoonof how do you lead your
colleagues in the state of thedata?
Because, frankly, that's one ofthe toughest jobs for that team
of teacher leaders to engagetheir colleagues meaningfully in
the data.
And then, at the end of the day,they take a written assessment
and code and code and code forabout 3540 minutes.
(17:21):
And I take the codes back to myoffice here and, and go through
them and send them, give themfeedback about their coding
skills and their codingaccuracy.
That's how we get those peopleto that.
Then when we start them out fortheir first data collection, we
recommend they pair withsomebody.
If the school's already usingthe process and they're being
added to the team, we recommendthey pair with a veteran data
collector, if not, then tworookie data collectors pair
(17:43):
together and they work like thatfor the first data collection
day or two, and then they moveoff solo.
And as soon as they move offsolo, we're off and running
because now we've got that manymore heads out there to get
deeper, richer pools of datafrom each school.
bryan-r-wright_1_04-11-20 (17:59):
Okay.
And then the second questionconnect the dots as it were,
between the summativeevaluations, such as maybe an
assessment for learningstrategies that you may want to
use with kids.
So the kids are getting part ofwhat they're doing with the fire
of higher earning and how theycan define what they're learning
in the classroom.
But then with the summativeevaluations, such as
standardized testing, how doesIPI affect positively the
(18:20):
summative testing strategies inschools?
squadcaster-6ia1_1_04-11-20 (18:23):
had the chance to do a lot of
studies with that, includingmultiple longitudinal looks at
the data.
We also had the chance to lookat the data across multiple
states.
we have consistent, and we'vegone to pretty sophisticated
statistical treatments,hierarchical linear modeling,
structural equation, equationmodeling.
We're way past the notion ofjust simple correlations and
simple regressions in terms oftrying to look at all of this
(18:45):
over the years.
we've written our papers, andWe've presented them at national
conferences and we put them upon the, website, et cetera.
So, we've got the researchgrounding behind that, but, what
we found is basically that ashigher deeper thinking
increases.
So too does academic success inschools.
(19:05):
also found that the big tailwagging the dog for a lot of
student learning isdisengagement and the amount of
time that kids are disengagedand not engaged with the
learning process in theclassroom that disengagement.
As it goes up, academic successgoes down.
And the ratio that we found overthat in our studies has been
(19:27):
about two to one for every twopercent increase in disengaged
learning time in a school.
That school year, The school canexpect to have lost one more
percent of a student bodyfailing the high stakes test,
meaning not meeting standards.
So, the 2 to 1 ratio, as we'vecalled it over the years in our,
(19:48):
in our work with the IPI is, isbasically shown you, you've
addressed this engagement and,you also have to address higher,
deeper thinking, you have to dothem both at the same time.
And we've also foundmathematically that it takes
about four units of higher,deeper thinking.
To neutralize one unit ofdisengagement.
(20:09):
Now, all of that's a good oldstatistical fancy stuff that we
do, but it's, informative and wesee it in the real world too.
When we start looking at theschools, I've had some real fun
with schools by plotting IPIlongitudinal data, their
statewide assessment data andany other kinds of assessment
data that they have.
Doing plot lines and going upand circling Where do you see
(20:30):
the dip?
Where do you see a growth?
Why is that happening?
Let's have a facultyconversation about those kinds
of things went past beyond justour leadership team.
So it's meant to be a real handson tool for schools to use that
puts him in the driver's seat.
Both in terms of collecting thedata and in learning from the
(20:51):
data.
bryan-r-wright_1_04-11-20 (20:53):
Okay.
you,
mark_1_04-11-2024_092147 (20:55):
So the conversation today was about I P
I instructional practiceinventory and how we collect
data.
And what's that data is usedfor.
It's uniquely different datathan our assessment scores, but
your question, Brian, aboutmaking that correlation between.
Using data like this and thendata like student assessments is
(21:16):
that there's a link.
And so that conversation aboutthe research that says, if we're
doing higher, deeper higherorder questioning, deeper
thinking there's going to be acorrelation in our assessment
scores.
We have to have teachers havingthat dialogue.
Sounds like what Jerry's sayingis we, we got to have teachers
have dialogue around that data.
And if they're having dialoguearound that data, then they can
(21:37):
make changes.
Dr.
Valentine, if people want toreach out to you to learn a
little bit about this, go to IPIstudent engagement.
com.
Would that be about right?
squadcaster-6ia1_1_04-11 (21:47):
That's a good place to go and look at
our work and look at ourresearch and all.
And then of course by email thebest place to, to get ahold of
mark_1_04-11-2024_092147 (21:55):
Okay.
squadcaster-6ia1_1_04 (21:55):
certainly
mark_1_04-11-2024_092147 (21:56):
Very good.
We'll, provide that for everyoneas well.
Anything else, Brian?
bryan-r-wright_1_04-11-2024_0 (22:00):
I think for this session, I think
we're about done.
I want to say thank you, Dr.
Valentine,
squadcaster-6ia1_1_04 (22:05):
pleasure.
bryan-r-wright_1_04-11-2024_ (22:05):
in the meantime, I want to say
thank you all.
Thank both of you for yourparticipation today on education
relevance.
Have a good day now.
Bye bye.
Popular Podcasts
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.