Using performance data to help with assessments in 2022
Blog by Graham Cummings - Former Head of Maths for Edexcel. He is currently an adviser to mathematics exam boards. @MathsNot
With the cancellation of examinations once again in 2021, teachers across the country are trying to find ways to assess their students and give them meaningful current attainment and projections for 2022.
Past papers might be used for this process – but in some cases, students will have found the papers and mark schemes on the internet; or else the past papers will contain material students haven’t been taught yet.
An alternative would be to use unseen material – but you’d then need to know how well students should be performing to help assign them a grade. How do we know which questions are those you’d expect most to be able to answer against those that are really tough? Perhaps that’s the easy bit – it’s the questions in the middle that you’ll have to wrestle with.
One way to make an assessment of your students with the various tests and assessments either supplied by exam boards or compiled yourself is to use the performance data which Pearson have been collecting via Results Plus for over ten years. This data gives you the average performance on every question of a student awarded each grade available.
For example, a student awarded Grade 4 might have, on average, scored 2.5 out of 5 on question 17. As a single question, it’s a broad measure, but when aggregated over a collection of questions, the performance data can provide useful guidance of the level a student is working at.
National Results Plus data is available via Pearson’s results Plus website – we’ll look at how to find that later. Luckily for users of the emporium, it has all been compiled over the years and is available in the Results Plus Skills Map cabinet.
As an example, let’s look at the data for GCSE Mathematics (1MA1) Paper 1H (Higher tier) from Summer 2019.
Yes, it’s tiny in this picture but not to worry for now.
What you might see is that a student gaining a nominal grade 4 on this paper averaged 24.61 out of 80 marks (just over 30%). The minimum mark for a 4 was 18 and the minimum for a Grade 5 was 27, so as you might expect the average is around the middle (not always exactly). Likewise, a student gaining a nominal grade 7 on this paper averaged 50.51 out of 80 marks (just over 63%). The minimum mark for a 7 was 46 and the minimum for a Grade 8 was 56, so once again the average is around the middle of those boundaries.
But what if you had set this paper to students who hadn’t, as a result of the pandemic, been taught any of the statistics and probability topics? This would be an extreme and unlikely example, of course, but for illustration, you could edit out those questions from this paper (Pearson make them available as Word files to make this easy) and edit out the performance data for those questions as well. Then you’ll find a student gaining a nominal grade 4 on this revised paper would have averaged 18.85 out of 67 marks (just over 28%). This will give you an idea of how well a student compares to a national figures for students who did sit the exams to decide a likely grade.
You’ll find the Practice sets 1-17 provided on the emporium have the performance data attached, and from this it has been possible to suggest grade boundaries for those papers. Once again, you can edit out questions and data as required.
What should you do if you want to make up your own assessments? Pearson have provided, via the emporium, a spreadsheet which includes performance data by grade for the last seven GCSE examination sessions. Whichever questions you choose for your own assessments, you can compile the data to give a you a sense of how well students performed nationally on that aggregate set, whether you have chosen questions by topic, difficulty or assessment objective.
The same principles apply for other qualifications too – AS/A level and International GCSE for example.
Skills maps and data for Mathematics qualifications can be found in the Results Plus Skills Maps cabinet on the emporium.
You can also find the data (for all subjects) on the Results Plus website by logging in using centre number 99999 and choosing the option for Global Results Analysis. Follow the options through to GCSE Mathematics June 2019, for example, and you’ll be able to download a .csv file with all the data.