September 18, 2024 • 27 mins
Today, we’re going to be taking a closer look at predicted grades – dispelling misconceptions and giving advice for how your school can create an effective predicted grades policy.
Our host is Laura Kahwati, Education Manager for Thought Leadership at Cambridge. She is joined by two special guests. Simon Child is the Head of Assessment Training at the Cambridge Assessment Network and Kevin Ebenezer is Head of Global Recognition at Cambridge.
Show notes:
An investigation of A Level teachers’ methods when estimating student grades - https://www.cambridgeassessment.org.uk/Images/631315-an-investigation-of-a-level-teachers-methods-when-estimating-student-grades.pdf
Bias in teacher assessment results - https://ofqual.blog.gov.uk/2021/05/17/bias-in-teacher-assessment-results/
Cambridge Centre for Evaluation and Monitoring baseline asssessments - https://www.cem.org/
Predicted grades: A guide for schools - https://www.cambridgeinternational.org/Images/682063-predicted-grades-a-guide-for-schools.pdf
Cambridge Assessment Network - https://www.cambridgeassessment.org.uk/the-network/
The What, Why and How of Assessment: A guide for teachers and school leaders - https://www.amazon.co.uk/What-Why-How-Assessment-teachers/dp/1529752558
A new Cambridge guide to support schools in issuing predicted grades fairly and equally - https://blog.cambridgeinternational.org/a-new-cambridge-guide-to-support-schools-in-issuing-predicted-grades-fairly-and-equally/
UCAS predicted grades – what you need to know for entry - https://www.ucas.com/advisers/help-and-training/guides-resources-and-training/application-overview/predicted-grades-what-you-need-know-entry-year
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
(upbeat music)
- Hello, welcome to theBrighter Thinking Pod
from the International Education Group
of Cambridge UniversityPress & Assessment.
I'm Laura Kahwati,
Education Manager for Thought Leadership
(00:20):
at Cambridge International,
and I'll be your host today.
We created our Brighter Thinking Pod
to support teachers around the world.
Each episode brings you helpful advice
and interesting conversationfrom some of our authors,
teachers, and academics.
Today, we're going tobe taking a closer look
at predicted grades,dispelling misconceptions,
(00:42):
and giving advice for howyour school can create
an effective predicted grades policy.
For this episode, we arejoined by two special guests.
Returning to the podcast,we have Simon Child
who featured in the last serieson our assessment episodes.
Simon is the Head of Assessment Training
at the Cambridge Assessment Network
(01:03):
and co-course Director
for the Postgraduate Advanced Certificate
in Educational Assessment atthe University of Cambridge.
He's the author of the book,
"The What, Why and How of Assessment:
A guide for teachers and school leaders."
Hello, Simon.
- Hi, Laura.
- He is joined by Kevin Ebenezer.
(01:24):
Kevin is Head of Global Recognition
at Cambridge International Education.
In his role, he collaboratesextensively with ministries,
universities, governments,
and various stakeholdersto secure recognition
for Cambridge qualifications.
So, hello, Kevin.
- Hi, Laura.
(gentle upbeat music)
(01:58):
- Remember all the links
and the info that we discuss are available
in the show notes for your ease.
And if you want to get yourvoice heard on the show,
you can get in touchon Twitter or Instagram
at CambridgeInt.
We begin each show with an icebreaker
to help our listeners getto know our guests more.
For this episode, keepingwith our predictions theme,
(02:21):
our question is,
please tell us about yourworst-ever prediction.
And Simon, if it's okay,I'd like to start with you.
- Thanks, Laura.
But I'd like to caveatthis as I was very young
at the time when I madethis bad prediction
and bad predictions often emerge
from some form of naivetyor blind faith in something.
(02:43):
And my prediction actuallyis no different to that,
to be honest.
It's football-based.
So when I was very young,
I predicted that my footballteam, Blackburn Rovers,
who had just won the leaguein 1995, fired to victory
by Alan Shearer amongstsome other players as well.
I predicted that they wouldgo on to retain their title
for the next five years.
(03:04):
And not only did thatspectacularly not happen,
the football team were actually relegated
from the Premier League within
that five year period.- Oh, no.
- So it went spectacularly badly.
My prediction on that front.
- My goodness.
Well, I'm sorry to hear that,
but thank you
for such a confessionalicebreaker answer. (laughs)
- Thank you.
- Kevin, please, can we move on to you?
(03:27):
Could you tell us aboutyour worst-ever prediction?
- Well, Simon spoke about football.
The other prediction is the weather.
And mine is about the weather.
Again, blind faith,
because I trusted the BBC,
more than I trusted the Belgian news.
And I was living in Belgium at that time,
and I was at a conference,
(03:49):
I was at a hotel, listenedto the news in the morning
and saw the weather.
The Belgian news predicted hurricanes,
the British news, BBCpredicted strong winds.
So I chose to drive home
and at that night there was a hurricane.
I had to stop the car
(04:09):
and sleep in a lay way on the road.
- Oh, no.
- So yes, (laughs) bad prediction in my-
- I think it was the worst storm
in UK history at the time, wasn't it?
In terms of it-- Yeah, I think it was.
And yeah, the weatherman was Michael Fish
and he became an iconic figure.
- (laughs) For making bad predictions.
(04:30):
- For making bad predictions. Yes.
- That was really unfortunately linked
to the word icebreaker, actually.
I'm so sorry to hear that.
That's quite a disastrous bad prediction.
However, I think you can forgive yourself
that it was sort of indirect.
I mean, you were followinga trusted source.
- Yeah.
(all laughing)That's true.
- But I guess it doesn't always go right.
(04:52):
Do you keep pillows and ablanket in your car from now on?
- No, I just, (laughs)
I don't trust the BBC, whatever.
(all laughing)
- Oh, dear.
Thank you both very much forthose icebreaker answers.
Moving on to more seriousbusiness of predicted grades,
(05:12):
please, can I start with you,Kevin, with my first question?
Can you explain to our audiences
what we mean when we talkabout predicted grades?
- Well, from a recognition point of view,
predicted grades refer to theestimated academic performance
of a student in a particular subject
as forecasted by teachers,departments or schools.
(05:36):
These grades are oftenused for various purposes,
mainly for admission into universities,
but also for scholarship applications
or even guide students onthe educational journeys.
Because if you have predicted low grades,
you may not want to apply toa certain university where,
you know, you don't havea chance of getting in.
(05:57):
So it is a really important thing
that we try to get the predictions right.
- So you are always thinkingabout that next step
beyond predicted grades
and how much they caninfluence those steps.
- Yeah.
And they influence the outcomesfor those students as well.
- Yeah, Kevin's definition,I agree with, you know,
that it's sometimescalled a predicted grade,
(06:19):
sometimes called a forecast grade.
And there's lots of differentreasons why they are useful
or potentially useful to students,
teachers at universities and so on.
So I think in some ways,
it's often a requirement aspart of the application process
as Kevin mentioned, for a student applying
for further learning.
(06:39):
But I also think that there'sthe idea of the information
is requested by boards,
such as Cambridge for thepurpose of maybe dealing
with unusual circumstances for students
who are taking examinations.
Maybe, they're ill on a particular day.
Predictive grades canhelp with that support
that Cambridge offers.
(06:59):
But also, it can be used in some cases
as a kind of motivational inspiration
for the student as well.
So thinking about howpredictive grades function
in that respect issomething that hopefully,
we'll explore later onin the podcast as well.
And largely that's partof the idea of reflecting
on a student's current progress.
(07:20):
So that can often bea reason, for example,
school reporting or accountability,
seeing the progression of a student
between one stage of learning and another.
- That's really interesting
because obviously, there'sthe importance of the accuracy
around what kind ofprogress we might assume
a child to make, but those more emotional
(07:41):
and psychological side of things,
like the idea that astudent might be inspired
to do something like Kevinsaid in terms of the next step
for the future of their education.
- Yeah, it's often,
there's sometimes the predictedgrade is actually turned
to the student as an aspirational grade.
And that's an interestinglabeling that's given to that.
(08:02):
So it's the idea of that ifyou do, well, if you work hard,
you can achieve this thing.
And for students that are
what we'd call extrinsicallymotivated, you know,
need to have that kind of,
that push and that idea ofwhat they could achieve,
predictive grades do servethat purpose in many cases.
- So some children may beintrinsically motivated,
(08:22):
whereas others might actually
have such a difference made to them
by having that external aspiration.
- Exactly. Yeah.
- Can I ask you both?
And I will start withyou for this one, Simon.
How important is it to make sure
that predicted grades are fair
and equal given all of those things
(08:42):
that we've just talked about
that have to be taken into account?
In other words, I mean,
why not just predict everyone in A*?
- (laughs) Well, in some ways,
the importance of the predictedgrades generally depends
on the reasons for whythey're being generated
in the first place.
So when I was a student, way back in 2004,
(09:03):
when I was doing my A Levels,
we submitted our project grades to UCAS,
and that was actually partof the application process.
So that was giving theuniversities information
about where I sat withinthe cohort in some way.
So the high stakes nature ofthose predicted grades meant
(09:23):
that the grades themselveswere often overestimated.
And some research from Cambridge
would suggest that up to 65%
of predicted grades madeby teachers are after
usually an overestimationof the actual performance
that a student has atthe end of their studies.
And this might not actuallybe too much of a problem
(09:44):
in terms of fairness and so on,
as long as we're awareof that particular trend
of why teachers may be overover predicting their students.
But what's more, perhaps moreproblematic is when we delve
into how those predicted grades are made.
And that's where the fairnesspoint comes in a little bit.
So off call did some research
(10:05):
and they found,
it was around 2021
that they found a bitof evidence of biases
in terms of gender,
in terms of people withspecial educational needs
or for people in fromdisadvantaged groups.
So if it's a high stakes typeprediction, like A Level,
(10:27):
those biases may actuallybe disadvantaging
some subsets of students.
So making sure that therisk fair is possible
and that the processes in placeare as fair as they can be,
is really important forthose different groups.
- Absolutely, yes.
No groups should be treatedany differently to any others.
And I presume that highlights even more
why it's important for schoolsto know who the groups are
(10:47):
that might be disadvantagedso that you can think ahead
before you get too late in the process
of predicting their grades.
- Exactly. Yeah.
- Kevin.
- Yeah, I mean, Simon,just to let you know,
you can still uses predicted grades
and they make offers on it,
and these are conditional offers
(11:08):
and if a student doesn'tmake those grades,
in some cases they willnot get that place.
And this is really important for students,
especially our students overseas,
because it's really importantthat they do get the place,
so they can work ontheir visas, et cetera.
So, you know, going into clearingat a later stage makes it
(11:29):
a lot harder for students.
So getting those predictionsright is very important
for students, especiallycoming from overseas.
And what we find alsois that students that
if students are grades are over predicted
in some courses, thenCambridge students' reputation
(11:55):
may go down by universities
because if everybody overpredicts, then they'll say,
"Oh, this is from Cambridge,
so it can actually damage thegrading system for Cambridge."
I think fairness andpredicted grade essential
for fostering competitive educational
and a competitive educational environment.
(12:19):
And no, everyone is predicting an A*
to diminish the credibility of that A*.
- I see.
So actually the credibilityof a predicted grade
goes way beyond the process of predicting
rather than just leading up towards it.
- Yeah.
- Simon, you just picked up on bias
(12:39):
and that really highlights the importance
of getting predicted gradesright and making them credible.
What information can teachersuse in order to make sure
that their predictedgrades and the processes
that lead up to them areas robust as possible?
- In most cases, Laura,
there's a mix of what we'd call data-based
(13:00):
and professional judgment
in making predicted grades for students.
So the first general area
would be some kind ofstatistical information
that we've generated.
And that kind of data may come
from things likebenchmarking type assessments
or more recent assessmentsadministered by teachers.
And one important thing to consider
(13:23):
when making predicted grades
using that kind of information
is the weighting ofthese different sources
of statistical evidencethat may be produced
or used by schools.
So for example, you may be tempted
to use a large-scalebenchmarking assessment,
such as the ones offered
by the Cambridge Centerfor Evaluation & Monitoring
to make a prediction for a student.
And so let's call them StudentX, just for for instance,
(13:47):
those types of benchmarking assessments
are very effective at giving a sense of
how students like Student X are likely
to perform in a later assessment.
But there are individual differences
that need to be considered specific
to that particular student.
You could also use assessment performance.
(14:08):
So for example, previous grades in GCSE
or other formalized assessmentscould be useful information.
Generally, giving moreweight to recent performances
and progressive students
is key for using that kind of information.
So for example,
some research from theresearch team in Cambridge
found that most teachersuse some kind of form
(14:29):
of mock examination to confirm predictions
that may be coming out
from assessment performance data.
And then finally, it'sthe kind of the sense
of making some kind of in-class judgment.
So for example, as a teacheryou may be getting a perception
or an understanding ofthe motivation of students
and their interest in the subject
and area that they're studying,
(14:51):
and also the day-to-dayquality of their work.
So those are really the three main areas
that I can think of that teachers can use
to make predicted grades.
- It really reminds us how allof those assessment processes
that go on all the timeat various school stages
are so important, whetherit be mock examinations
or class tests.
And not only that, but thatteachers are consistent as well.
(15:15):
Kevin, you have a lot of correspondence
with schools globally in this area.
What have you found, basedon what Simon just said?
- Yeah, I think, Simonlisted quite a few things
that I see from schools.
You know, mainly schoolswill use class performance,
mock exam results, homeworkIGCSE results for A Levels,
(15:40):
but also I think that more holistic idea
when they know the student,
so they can actually look
at things like the student's work ethic,
the engagement, all these sort of things
will contribute to the teachers making
that professional judgment
to provide a good predictedgrade for the student.
(16:03):
- And just add to that,Kevin, there's the idea of
that teachers from theresearch anyway indicates
that teachers are oftenthinking about students
and how they'd perform
on their best day.- Yeah.
- And that's an interesting part
when we're making predicted grades,
and that was part of the explanation
for why they're often over predicted is
'cause we're optimistic about our students
and how they will perform
(16:23):
in the moment so.- Yeah.
- All that information about
how they do in things likeclassroom-based assessments
and so on is useful informationin terms of giving the sense
of how they wouldperform on any given day,
which is really interesting.
- It's such a difficultcall to make I guess,
unless you have enough information there
to be really confident
because otherwise, you'rehaving to weigh up so much
(16:46):
in terms of what is accurate,what is aspirational.
Of course, we all want to be optimistic,
but we also have to be reliable as well.
So such a careful balance.
In terms-
- Yeah, it's not anexact science, you know,
it is professional judgment.
- Yes. Yeah.
And we need as much information as we can
(17:07):
to fuel that professional judgment too.
When we are trying tonavigate things like this.
What are some of the commonpitfalls that we do come across?
I know you've mentioned some already.
Simon, is there anythingelse you've noticed
that can be a problem in this process?
- Hmm. Yes.
There's a few common pitfallsthat I thought about.
(17:27):
So the first one is really the idea of
not actually being clear
as a school or as an individual teacher
as to why the predicted gradesare actually being created
in the first place andwhat their purpose may be.
So we've talked a lotabout it being related
to some later high-stakes assessment
and motivational purposes and so on.
(17:47):
But defining what that purpose is,
inefficiently can actually be an issue
'cause it takes time as a teacher to say,
build up that evidence and that confidence
in the predicted grade.
And there's an opportunity cost to that
in terms of other teachingthat could be done.
So that's something to be aware of.
I'll say as well, the idea of safeguards
(18:09):
and specific processes
to try and remove the potentialof the biases, for example.
So having some kind ofstandardization or moderation policy
or process within a school
is a way of guarding againstthose particular issues.
And finally, the idea ofclass-to-class variability
(18:30):
in the evidence used.
So again, you could havesomething at a school level
to determine which are thebest sources of evidence
for a teacher to actuallymake their judgments on
and which ones are most reliable,
perhaps externally sourced and so on.
And that can actually makea much stronger prediction
and reduce that class-to-class variability
(18:52):
in the quality of the predictions.
- So really in order to helpthat professional judgment
of everybody involvedthat Kevin talked about,
we need a school-wide policy
and culture around this sothat there's consistency.
- Yeah, I agree with that.
I think sometimes limiteddata is also used.
(19:13):
So a school may just predictit on one assessment,
like a mock exam.
And in those cases, that'sgonna be a major pitfall
because it all depends onthe student on that day
and their performance.
- And we can't, with allthe opportunity that we have
for summative assessmentsthroughout the years
(19:33):
that a student's in education,
we can't afford to just relyon one single mock examination.
- Yeah.
- So in terms of creating a clear
and effective policy forpredicted grades in schools,
Kevin, how can schools go about this?
- Well, I think again, you know,
(19:54):
they have to create thatclear and effective policy
and there's a lot of guidancewe have provided on that.
But, you know,
they have to regularlyreview their processes.
For me that is really important
because what they can do iscollect data longitudinally
and actually see, okay,
(20:16):
how are predictions arewe getting it accurate?
And that for me would bethe best way for schools
to look at it and reflect on their policy.
- Kevin, I agree.
And just to add to that a little bit more,
that kind of post-hoc orpost prediction analysis
that you would do is I thinka really important part
(20:38):
of the overall policy structure
that you have within a school.
So even in cases where youwould have a predicted grade
that was aspirational tothe student, you know,
somehow trying to motivatethem to do as well as they can,
if that's the actual intentionof the predicted grades.
And then you should see some success
(21:01):
in terms of how thepredicted grade correlates
with the actual performanceof the students.
And so that's a reallyimportant thing to consider.
But in terms of the things
that I would also think about
in terms of creating an effective policy,
as I mentioned before,
the idea of thinking deeplyabout the opportunity cost
for teacher time and developing the data
(21:24):
and the assessments themselves
for making good predicted grades.
It may be in some circumstances
that they're not necessarily required
and time could be spent on something else.
So there is sometimes thatanalysis needs to be done in
as part of creating the policy,
but also whilst you aremaking the predicted grades
(21:46):
and generating the differentsources of evidence
and analyzing them, theidea of trying to create
and maintain a standardizedapproach within departments
and across schools as well.
So in many ways,
those approaches mayresemble the sorts of things
that you may observe in astandardization meeting.
The idea of getting groupsof teachers together
(22:07):
to look at the differentsources of evidence
and the different grades.
And also,
almost having a kind ofinter-teacher reliability element
where you look at one teachermaking the predicted grade
for one student
and another teacher makingthe same predictor grade
about the same student
and seeing if there is acorrelation of those two
(22:28):
that can give an indicationof any potential biases
or disagreements.
And that can actuallystrengthen the overall processes
in place in terms of theoverall policy for schools.
- It sounds to me likethe best thing to do,
as with many things in education,
is to make sure that whenyou've got something in place
that you do always comeback and reevaluate it
(22:49):
before you go round again.
And also, like you say, having a culture
of dialogue amongst teachersand amongst staff members.
I think not only does that help the pupils
who are receiving the predicted grades,
but it also helps theteachers with their judgment.
- I think another thing
is we have to look at is transparency.
(23:09):
So transparency in the policy,
and that's really importantfor students and for parents.
And if schools are nottransparent about the policy
of predicted grades,there could be problems
for the schools down the line.
So making your policy wellknown to students and parents
I think is really important.
- Absolutely, keeping everyoneinformed along the way
(23:31):
and remembering that thisisn't just about teachers
and students, but it isalso about parents as well
and other stakeholders.
Simon, where can teachers
and school leaders go tofind out more information?
- Well, last year, CambridgeInternational created
a really useful factsheet on predicted grades,
(23:52):
which you can access freely
on the Cambridge International website.
So it's called "Predictedgrades: A guide for schools."
So you can hopefully findthat relatively easily.
But one thing to add is that
if you are using somekind of form of assessment
that you are generatingor creating yourselves,
(24:13):
one thing that's really important
is to establish the validity
and the reliability of thoseindividual assessments.
So if you want to considerdeveloping general
and translatable expertisein assessment design,
we do offer a range, of courses,
at the Cambridge AssessmentNetwork that can help
with this process.
And overall, the aims ofthose kinds of courses
(24:36):
is to help people makebetter predicted grades
for their students
and to also give them thebest chance of success
in their later careers and lives.
- And of course, in theassessment processes,
we can also remember your book,
"The What, Why and How ofAssessment" for teachers too.
Kevin, where would you say thatteachers and school leaders
(24:59):
to go to find out more information?
- Well, our website, as Simon mentioned,
I also wrote a blog to accompany that,
so that's on our Cambridge blog pages.
And also I think schoolscan look at webinars.
There's lots of webinarson how to predict grades.
There's also guidanceavailable on the UCAS website.
(25:24):
Now that's really important
because UCAS does use predicted grades
and the guidance theygive is really valuable.
They can also attend workshops,webinars, conferences,
and organizations,
such as the Community forInternational Schools.
IACAC and UCAS actuallydo conferences about this
(25:47):
because predicted gradesare really important
for universities.
So if schools attend those conferences,
go to one of those sessions.
- That's really helpful advice.
Thank you.
And I do remember readingyour blog on predictive grades
and how interesting it was to consider
the international landscapeof predictive grades
as well as whatever regionyou happen to be in.
(26:07):
(gentle music)
Well, that is all thatwe've got time for today
on our "Predicted Grades" episode.
Thank you to Kevin and Simonfor being such fantastic guests
and sharing some really useful insights.
I hope I can predict that everyone
will find it informativeand helpful (laughs)
and as enjoyable as I have.
(26:27):
Don't forget to tell yourfriends and colleagues about us
and rate our show on whateverplatform you are listening on.
Our show notes have lots of useful links
that we've discussedthroughout the episode,
so be sure to take a look at them.
And you can also follow and contact us
on Twitter or Instagram at CambridgeInt.
(26:49):
Thanks for listening.
We hope you join us again soon.
(gentle upbeat music)
(upbeat music)
- Hello, welcome to theBrighter Thinking Pod
from the International Education Group
of Cambridge UniversityPress & Assessment.
I'm Laura Kahwati,
Education Manager for Thought Leadership
(00:20):
at Cambridge International,
and I'll be your host today.
We created our Brighter Thinking Pod
to support teachers around the world.
Each episode brings you helpful advice
and interesting conversationfrom some of our authors,
teachers, and academics.
Today, we're going tobe taking a closer look
at predicted grades,dispelling misconceptions,
(00:42):
and giving advice for howyour school can create
an effective predicted grades policy.
For this episode, we arejoined by two special guests.
Returning to the podcast,we have Simon Child
who featured in the last serieson our assessment episodes.
Simon is the Head of Assessment Training
at the Cambridge Assessment Network
(01:03):
and co-course Director
for the Postgraduate Advanced Certificate
in Educational Assessment atthe University of Cambridge.
He's the author of the book,
"The What, Why and How of Assessment:
A guide for teachers and school leaders."
Hello, Simon.
- Hi, Laura.
- He is joined by Kevin Ebenezer.
(01:24):
Kevin is Head of Global Recognition
at Cambridge International Education.
In his role, he collaboratesextensively with ministries,
universities, governments,
and various stakeholdersto secure recognition
for Cambridge qualifications.
So, hello, Kevin.
- Hi, Laura.
(gentle upbeat music)
(01:58):
- Remember all the links
and the info that we discuss are available
in the show notes for your ease.
And if you want to get yourvoice heard on the show,
you can get in touchon Twitter or Instagram
at CambridgeInt.
We begin each show with an icebreaker
to help our listeners getto know our guests more.
For this episode, keepingwith our predictions theme,
(02:21):
our question is,
please tell us about yourworst-ever prediction.
And Simon, if it's okay,I'd like to start with you.
- Thanks, Laura.
But I'd like to caveatthis as I was very young
at the time when I madethis bad prediction
and bad predictions often emerge
from some form of naivetyor blind faith in something.
(02:43):
And my prediction actuallyis no different to that,
to be honest.
It's football-based.
So when I was very young,
I predicted that my footballteam, Blackburn Rovers,
who had just won the leaguein 1995, fired to victory
by Alan Shearer amongstsome other players as well.
I predicted that they wouldgo on to retain their title
for the next five years.
(03:04):
And not only did thatspectacularly not happen,
the football team were actually relegated
from the Premier League within
that five year period.- Oh, no.
- So it went spectacularly badly.
My prediction on that front.
- My goodness.
Well, I'm sorry to hear that,
but thank you
for such a confessionalicebreaker answer. (laughs)
- Thank you.
- Kevin, please, can we move on to you?
(03:27):
Could you tell us aboutyour worst-ever prediction?
- Well, Simon spoke about football.
The other prediction is the weather.
And mine is about the weather.
Again, blind faith,
because I trusted the BBC,
more than I trusted the Belgian news.
And I was living in Belgium at that time,
and I was at a conference,
(03:49):
I was at a hotel, listenedto the news in the morning
and saw the weather.
The Belgian news predicted hurricanes,
the British news, BBCpredicted strong winds.
So I chose to drive home
and at that night there was a hurricane.
I had to stop the car
(04:09):
and sleep in a lay way on the road.
- Oh, no.
- So yes, (laughs) bad prediction in my-
- I think it was the worst storm
in UK history at the time, wasn't it?
In terms of it-- Yeah, I think it was.
And yeah, the weatherman was Michael Fish
and he became an iconic figure.
- (laughs) For making bad predictions.
(04:30):
- For making bad predictions. Yes.
- That was really unfortunately linked
to the word icebreaker, actually.
I'm so sorry to hear that.
That's quite a disastrous bad prediction.
However, I think you can forgive yourself
that it was sort of indirect.
I mean, you were followinga trusted source.
- Yeah.
(all laughing)That's true.
- But I guess it doesn't always go right.
(04:52):
Do you keep pillows and ablanket in your car from now on?
- No, I just, (laughs)
I don't trust the BBC, whatever.
(all laughing)
- Oh, dear.
Thank you both very much forthose icebreaker answers.
Moving on to more seriousbusiness of predicted grades,
(05:12):
please, can I start with you,Kevin, with my first question?
Can you explain to our audiences
what we mean when we talkabout predicted grades?
- Well, from a recognition point of view,
predicted grades refer to theestimated academic performance
of a student in a particular subject
as forecasted by teachers,departments or schools.
(05:36):
These grades are oftenused for various purposes,
mainly for admission into universities,
but also for scholarship applications
or even guide students onthe educational journeys.
Because if you have predicted low grades,
you may not want to apply toa certain university where,
you know, you don't havea chance of getting in.
(05:57):
So it is a really important thing
that we try to get the predictions right.
- So you are always thinkingabout that next step
beyond predicted grades
and how much they caninfluence those steps.
- Yeah.
And they influence the outcomesfor those students as well.
- Yeah, Kevin's definition,I agree with, you know,
that it's sometimescalled a predicted grade,
(06:19):
sometimes called a forecast grade.
And there's lots of differentreasons why they are useful
or potentially useful to students,
teachers at universities and so on.
So I think in some ways,
it's often a requirement aspart of the application process
as Kevin mentioned, for a student applying
for further learning.
(06:39):
But I also think that there'sthe idea of the information
is requested by boards,
such as Cambridge for thepurpose of maybe dealing
with unusual circumstances for students
who are taking examinations.
Maybe, they're ill on a particular day.
Predictive grades canhelp with that support
that Cambridge offers.
(06:59):
But also, it can be used in some cases
as a kind of motivational inspiration
for the student as well.
So thinking about howpredictive grades function
in that respect issomething that hopefully,
we'll explore later onin the podcast as well.
And largely that's partof the idea of reflecting
on a student's current progress.
(07:20):
So that can often bea reason, for example,
school reporting or accountability,
seeing the progression of a student
between one stage of learning and another.
- That's really interesting
because obviously, there'sthe importance of the accuracy
around what kind ofprogress we might assume
a child to make, but those more emotional
(07:41):
and psychological side of things,
like the idea that astudent might be inspired
to do something like Kevinsaid in terms of the next step
for the future of their education.
- Yeah, it's often,
there's sometimes the predictedgrade is actually turned
to the student as an aspirational grade.
And that's an interestinglabeling that's given to that.
(08:02):
So it's the idea of that ifyou do, well, if you work hard,
you can achieve this thing.
And for students that are
what we'd call extrinsicallymotivated, you know,
need to have that kind of,
that push and that idea ofwhat they could achieve,
predictive grades do servethat purpose in many cases.
- So some children may beintrinsically motivated,
(08:22):
whereas others might actually
have such a difference made to them
by having that external aspiration.
- Exactly. Yeah.
- Can I ask you both?
And I will start withyou for this one, Simon.
How important is it to make sure
that predicted grades are fair
and equal given all of those things
(08:42):
that we've just talked about
that have to be taken into account?
In other words, I mean,
why not just predict everyone in A*?
- (laughs) Well, in some ways,
the importance of the predictedgrades generally depends
on the reasons for whythey're being generated
in the first place.
So when I was a student, way back in 2004,
(09:03):
when I was doing my A Levels,
we submitted our project grades to UCAS,
and that was actually partof the application process.
So that was giving theuniversities information
about where I sat withinthe cohort in some way.
So the high stakes nature ofthose predicted grades meant
(09:23):
that the grades themselveswere often overestimated.
And some research from Cambridge
would suggest that up to 65%
of predicted grades madeby teachers are after
usually an overestimationof the actual performance
that a student has atthe end of their studies.
And this might not actuallybe too much of a problem
(09:44):
in terms of fairness and so on,
as long as we're awareof that particular trend
of why teachers may be overover predicting their students.
But what's more, perhaps moreproblematic is when we delve
into how those predicted grades are made.
And that's where the fairnesspoint comes in a little bit.
So off call did some research
(10:05):
and they found,
it was around 2021
that they found a bitof evidence of biases
in terms of gender,
in terms of people withspecial educational needs
or for people in fromdisadvantaged groups.
So if it's a high stakes typeprediction, like A Level,
(10:27):
those biases may actuallybe disadvantaging
some subsets of students.
So making sure that therisk fair is possible
and that the processes in placeare as fair as they can be,
is really important forthose different groups.
- Absolutely, yes.
No groups should be treatedany differently to any others.
And I presume that highlights even more
why it's important for schoolsto know who the groups are
(10:47):
that might be disadvantagedso that you can think ahead
before you get too late in the process
of predicting their grades.
- Exactly. Yeah.
- Kevin.
- Yeah, I mean, Simon,just to let you know,
you can still uses predicted grades
and they make offers on it,
and these are conditional offers
(11:08):
and if a student doesn'tmake those grades,
in some cases they willnot get that place.
And this is really important for students,
especially our students overseas,
because it's really importantthat they do get the place,
so they can work ontheir visas, et cetera.
So, you know, going into clearingat a later stage makes it
(11:29):
a lot harder for students.
So getting those predictionsright is very important
for students, especiallycoming from overseas.
And what we find alsois that students that
if students are grades are over predicted
in some courses, thenCambridge students' reputation
(11:55):
may go down by universities
because if everybody overpredicts, then they'll say,
"Oh, this is from Cambridge,
so it can actually damage thegrading system for Cambridge."
I think fairness andpredicted grade essential
for fostering competitive educational
and a competitive educational environment.
(12:19):
And no, everyone is predicting an A*
to diminish the credibility of that A*.
- I see.
So actually the credibilityof a predicted grade
goes way beyond the process of predicting
rather than just leading up towards it.
- Yeah.
- Simon, you just picked up on bias
(12:39):
and that really highlights the importance
of getting predicted gradesright and making them credible.
What information can teachersuse in order to make sure
that their predictedgrades and the processes
that lead up to them areas robust as possible?
- In most cases, Laura,
there's a mix of what we'd call data-based
(13:00):
and professional judgment
in making predicted grades for students.
So the first general area
would be some kind ofstatistical information
that we've generated.
And that kind of data may come
from things likebenchmarking type assessments
or more recent assessmentsadministered by teachers.
And one important thing to consider
(13:23):
when making predicted grades
using that kind of information
is the weighting ofthese different sources
of statistical evidencethat may be produced
or used by schools.
So for example, you may be tempted
to use a large-scalebenchmarking assessment,
such as the ones offered
by the Cambridge Centerfor Evaluation & Monitoring
to make a prediction for a student.
And so let's call them StudentX, just for for instance,
(13:47):
those types of benchmarking assessments
are very effective at giving a sense of
how students like Student X are likely
to perform in a later assessment.
But there are individual differences
that need to be considered specific
to that particular student.
You could also use assessment performance.
(14:08):
So for example, previous grades in GCSE
or other formalized assessmentscould be useful information.
Generally, giving moreweight to recent performances
and progressive students
is key for using that kind of information.
So for example,
some research from theresearch team in Cambridge
found that most teachersuse some kind of form
(14:29):
of mock examination to confirm predictions
that may be coming out
from assessment performance data.
And then finally, it'sthe kind of the sense
of making some kind of in-class judgment.
So for example, as a teacheryou may be getting a perception
or an understanding ofthe motivation of students
and their interest in the subject
and area that they're studying,
(14:51):
and also the day-to-dayquality of their work.
So those are really the three main areas
that I can think of that teachers can use
to make predicted grades.
- It really reminds us how allof those assessment processes
that go on all the timeat various school stages
are so important, whetherit be mock examinations
or class tests.
And not only that, but thatteachers are consistent as well.
(15:15):
Kevin, you have a lot of correspondence
with schools globally in this area.
What have you found, basedon what Simon just said?
- Yeah, I think, Simonlisted quite a few things
that I see from schools.
You know, mainly schoolswill use class performance,
mock exam results, homeworkIGCSE results for A Levels,
(15:40):
but also I think that more holistic idea
when they know the student,
so they can actually look
at things like the student's work ethic,
the engagement, all these sort of things
will contribute to the teachers making
that professional judgment
to provide a good predictedgrade for the student.
(16:03):
- And just add to that,Kevin, there's the idea of
that teachers from theresearch anyway indicates
that teachers are oftenthinking about students
and how they'd perform
on their best day.- Yeah.
- And that's an interesting part
when we're making predicted grades,
and that was part of the explanation
for why they're often over predicted is
'cause we're optimistic about our students
and how they will perform
(16:23):
in the moment so.- Yeah.
- All that information about
how they do in things likeclassroom-based assessments
and so on is useful informationin terms of giving the sense
of how they wouldperform on any given day,
which is really interesting.
- It's such a difficultcall to make I guess,
unless you have enough information there
to be really confident
because otherwise, you'rehaving to weigh up so much
(16:46):
in terms of what is accurate,what is aspirational.
Of course, we all want to be optimistic,
but we also have to be reliable as well.
So such a careful balance.
In terms-
- Yeah, it's not anexact science, you know,
it is professional judgment.
- Yes. Yeah.
And we need as much information as we can
(17:07):
to fuel that professional judgment too.
When we are trying tonavigate things like this.
What are some of the commonpitfalls that we do come across?
I know you've mentioned some already.
Simon, is there anythingelse you've noticed
that can be a problem in this process?
- Hmm. Yes.
There's a few common pitfallsthat I thought about.
(17:27):
So the first one is really the idea of
not actually being clear
as a school or as an individual teacher
as to why the predicted gradesare actually being created
in the first place andwhat their purpose may be.
So we've talked a lotabout it being related
to some later high-stakes assessment
and motivational purposes and so on.
(17:47):
But defining what that purpose is,
inefficiently can actually be an issue
'cause it takes time as a teacher to say,
build up that evidence and that confidence
in the predicted grade.
And there's an opportunity cost to that
in terms of other teachingthat could be done.
So that's something to be aware of.
I'll say as well, the idea of safeguards
(18:09):
and specific processes
to try and remove the potentialof the biases, for example.
So having some kind ofstandardization or moderation policy
or process within a school
is a way of guarding againstthose particular issues.
And finally, the idea ofclass-to-class variability
(18:30):
in the evidence used.
So again, you could havesomething at a school level
to determine which are thebest sources of evidence
for a teacher to actuallymake their judgments on
and which ones are most reliable,
perhaps externally sourced and so on.
And that can actually makea much stronger prediction
and reduce that class-to-class variability
(18:52):
in the quality of the predictions.
- So really in order to helpthat professional judgment
of everybody involvedthat Kevin talked about,
we need a school-wide policy
and culture around this sothat there's consistency.
- Yeah, I agree with that.
I think sometimes limiteddata is also used.
(19:13):
So a school may just predictit on one assessment,
like a mock exam.
And in those cases, that'sgonna be a major pitfall
because it all depends onthe student on that day
and their performance.
- And we can't, with allthe opportunity that we have
for summative assessmentsthroughout the years
(19:33):
that a student's in education,
we can't afford to just relyon one single mock examination.
- Yeah.
- So in terms of creating a clear
and effective policy forpredicted grades in schools,
Kevin, how can schools go about this?
- Well, I think again, you know,
(19:54):
they have to create thatclear and effective policy
and there's a lot of guidancewe have provided on that.
But, you know,
they have to regularlyreview their processes.
For me that is really important
because what they can do iscollect data longitudinally
and actually see, okay,
(20:16):
how are predictions arewe getting it accurate?
And that for me would bethe best way for schools
to look at it and reflect on their policy.
- Kevin, I agree.
And just to add to that a little bit more,
that kind of post-hoc orpost prediction analysis
that you would do is I thinka really important part
(20:38):
of the overall policy structure
that you have within a school.
So even in cases where youwould have a predicted grade
that was aspirational tothe student, you know,
somehow trying to motivatethem to do as well as they can,
if that's the actual intentionof the predicted grades.
And then you should see some success
(21:01):
in terms of how thepredicted grade correlates
with the actual performanceof the students.
And so that's a reallyimportant thing to consider.
But in terms of the things
that I would also think about
in terms of creating an effective policy,
as I mentioned before,
the idea of thinking deeplyabout the opportunity cost
for teacher time and developing the data
(21:24):
and the assessments themselves
for making good predicted grades.
It may be in some circumstances
that they're not necessarily required
and time could be spent on something else.
So there is sometimes thatanalysis needs to be done in
as part of creating the policy,
but also whilst you aremaking the predicted grades
(21:46):
and generating the differentsources of evidence
and analyzing them, theidea of trying to create
and maintain a standardizedapproach within departments
and across schools as well.
So in many ways,
those approaches mayresemble the sorts of things
that you may observe in astandardization meeting.
The idea of getting groupsof teachers together
(22:07):
to look at the differentsources of evidence
and the different grades.
And also,
almost having a kind ofinter-teacher reliability element
where you look at one teachermaking the predicted grade
for one student
and another teacher makingthe same predictor grade
about the same student
and seeing if there is acorrelation of those two
(22:28):
that can give an indicationof any potential biases
or disagreements.
And that can actuallystrengthen the overall processes
in place in terms of theoverall policy for schools.
- It sounds to me likethe best thing to do,
as with many things in education,
is to make sure that whenyou've got something in place
that you do always comeback and reevaluate it
(22:49):
before you go round again.
And also, like you say, having a culture
of dialogue amongst teachersand amongst staff members.
I think not only does that help the pupils
who are receiving the predicted grades,
but it also helps theteachers with their judgment.
- I think another thing
is we have to look at is transparency.
(23:09):
So transparency in the policy,
and that's really importantfor students and for parents.
And if schools are nottransparent about the policy
of predicted grades,there could be problems
for the schools down the line.
So making your policy wellknown to students and parents
I think is really important.
- Absolutely, keeping everyoneinformed along the way
(23:31):
and remembering that thisisn't just about teachers
and students, but it isalso about parents as well
and other stakeholders.
Simon, where can teachers
and school leaders go tofind out more information?
- Well, last year, CambridgeInternational created
a really useful factsheet on predicted grades,
(23:52):
which you can access freely
on the Cambridge International website.
So it's called "Predictedgrades: A guide for schools."
So you can hopefully findthat relatively easily.
But one thing to add is that
if you are using somekind of form of assessment
that you are generatingor creating yourselves,
(24:13):
one thing that's really important
is to establish the validity
and the reliability of thoseindividual assessments.
So if you want to considerdeveloping general
and translatable expertisein assessment design,
we do offer a range, of courses,
at the Cambridge AssessmentNetwork that can help
with this process.
And overall, the aims ofthose kinds of courses
(24:36):
is to help people makebetter predicted grades
for their students
and to also give them thebest chance of success
in their later careers and lives.
- And of course, in theassessment processes,
we can also remember your book,
"The What, Why and How ofAssessment" for teachers too.
Kevin, where would you say thatteachers and school leaders
(24:59):
to go to find out more information?
- Well, our website, as Simon mentioned,
I also wrote a blog to accompany that,
so that's on our Cambridge blog pages.
And also I think schoolscan look at webinars.
There's lots of webinarson how to predict grades.
There's also guidanceavailable on the UCAS website.
(25:24):
Now that's really important
because UCAS does use predicted grades
and the guidance theygive is really valuable.
They can also attend workshops,webinars, conferences,
and organizations,
such as the Community forInternational Schools.
IACAC and UCAS actuallydo conferences about this
(25:47):
because predicted gradesare really important
for universities.
So if schools attend those conferences,
go to one of those sessions.
- That's really helpful advice.
Thank you.
And I do remember readingyour blog on predictive grades
and how interesting it was to consider
the international landscapeof predictive grades
as well as whatever regionyou happen to be in.
(26:07):
(gentle music)
Well, that is all thatwe've got time for today
on our "Predicted Grades" episode.
Thank you to Kevin and Simonfor being such fantastic guests
and sharing some really useful insights.
I hope I can predict that everyone
will find it informativeand helpful (laughs)
and as enjoyable as I have.
(26:27):
Don't forget to tell yourfriends and colleagues about us
and rate our show on whateverplatform you are listening on.
Our show notes have lots of useful links
that we've discussedthroughout the episode,
so be sure to take a look at them.
And you can also follow and contact us
on Twitter or Instagram at CambridgeInt.
(26:49):
Thanks for listening.
We hope you join us again soon.
(gentle upbeat music)
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Special Summer Offer: Exclusively on Apple Podcasts, try our Dateline Premium subscription completely free for one month! With Dateline Premium, you get every episode ad-free plus exclusive bonus content.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.