Using Computers to Predict Exam Results – A Data Law Tightrope…

I took a day of leave from work yesterday, which also happened to be A Level results day. Due to the current COVID-19 situation, I found myself in the house for most of the day with a news channel on in the background. As you would expect the majority of the coverage was about A Level results. In previous years the debate had always been ‘are exams getting easier?’ This year the debate was dominated by the fairness surrounding how the results were calculated.

It was heartbreaking to see students on TV who have worked so hard and through no fault of their own were not given the opportunity to sit the exams or have the benefit of face-to-face teaching for that matter. Equally, students who had been awarded low grades despite being predicted good ones.

One of my former law lecturers taught me how ‘to do’ exams. She said don’t go into an exam worried – see it as an opportunity to show off your knowledge. Sadly the cohort of 2020 was denied that opportunity.

In order for me to provide an analysis of the results process it would be helpful to explain how the results were calculated this year. Traditionally of course it would be marks on the paper as a percentage providing a grade A-E.

The Office of Qualifications and Examinations Regulation or “OFQUAL” as they are more commonly known are the regulators of exams in the UK. OFQUAL have produced a fairly detailed set of documents explaining the methodology. I will attempt to summarise in a few paragraphs.

Initially teachers were asked to provide predicted grades. This was termed ‘centre assessed grade’ (CAG). Teachers were then asked to rank their pupils this is known as ‘rank order.’ OFQUAL explain the rank order as follows:

“We also asked teachers to provide a rank order of students for each grade for each subject. There were several reasons for this. First, we know from research evidence that people are better at making relative judgements than absolute judgements and that teachers’ judgements tend to be more accurate when they are ranking students rather than estimating their future attainment. The research literature suggests that, in estimating the grades students are likely to achieve, teachers tend to be optimistic (although not in all cases). That is not surprising, teachers want to do the best for their students, and the analysis we carried out immediately after CAGs were submitted bears this out.”

From what I can gather when OFQUAL analysed all of the data they were unhappy with the increase in pass rates. They deemed them to be ‘implausibly high.’

This is when OFQUAL or the Government or both decided to implement a system of ‘statistical standardisation’ Instead. This is called the Direct Centre Performance model (DCP). There is an explanation of DCP in one of the OFQUAL issued documents:

“(P)redicting the distribution of grades for each individual school or college.

That prediction is based on the historical performance of the school or college in that

subject taking into account any changes in the prior attainment of candidates

entering this year compared to previous years. This was fine-tuned to take account

of known issues such as centres with small cohorts of students, small-entry subjects,

and tiered subjects. Decisions were also made on the number of years of historical

data included in the model. The details of these decisions are set out in this report

and are formalised in the regulations we put in place for summer 2020.”

In a nutshell, DCP is a statistical model. Is this then an automated process and does it engage data law?

The General Data Protection Regulation (GDPR) describes “profiling” as “any form of automated processing of personal data consisting of the use of personal data to evaluate certain personal aspects relating to a natural person, in particular to analyse or predict aspects concerning that natural person’s performance at work, economic situation, health, personal preferences, interests, reliability, behaviour, location or movements;” Although exams results are not specifically mentioned in this section, the students data is being processed (used) to evaluate potential academic performance.

Article 22 of the GDPR deals with automated decision making. This is what Article 22(1) says:

“The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.”

In plain English, my reading of Article 22(1) is that an A Level or GCSE student has the right not to be subject to a decision based solely on automated processing. If the system is found to be an automated system then clearly it will affect his or her because life opportunities for University and or work will be impacted by poor results.

The Information Commissioner’s Office (ICO) is the UK Regulator of data protection. The ICO gives an example of an automated process being an online decision to award a loan. In that process I may provide data such as my name, employment status, income and expenditure in order for the bank to decide my creditworthiness. I then receive a decision. In the case of exam results, the teacher, school or college has provided certain information about the students and therefore the process may be automated, following the loan example. Raw data is provided and a computer system makes a decision. Others may argue that the processing isn’t solely automatic because there is teacher input. This point is no doubt open to debate but it does appear to be automated.

Article 22(2)(c) of the GDPR says that automated processing is allowed if an individual has consented to the automated processing. As far as I am aware, pupils hadn’t expressly consented, certainly the ones on TV didn’t give me that impression. An automated system is also allowed if there is a contract or a specific law.

Article 22(3) does provide some comfort for individual pupils. There is an obligation for data controllers (in this case the exam board) to provide some human intervention. My suspicion is the human intervention would be the right to appeal the automated decision and, further, students do have the opportunity to take written examinations in autumn. At the time of writing this blog the appeals process was unclear (like many things in the current Covid world) and it appears that the school or college would appeal rather than the individual student.

The Secretary of State for Education, Rt. Hon. Gavin Williamson MP, explained the new ‘triple lock system’. He said that the school or college would appeal:

“Students who would like to use a valid mock result will be able to do so through the appeals process, with individuals notifying their school or college who will provide evidence of their mock results to their exam board.”

The triple lock may be sufficient to satisfy Article 22(3), but if a student is denied the opportunity to appeal then there may be a breach of the GDPR.

In conclusion, an individual would probably need to show that OFQUAL automatically processed data without a contractual right, explicit consent (in other words permission) or any legal obligation and that the individual was not given the opportunity to have any sort of human intervention into the process in order to bring a successful claim under the GDPR. OFQUAL may argue that by giving an opportunity to appeal under the Government’s triple lock, they satisfy Article 22(3) and, further, the system wasn’t wholly automated because the raw data was obtained from teachers. If pupils do successfully demonstrate a breach of the GDPR, the likely remedy would be compensation for the upset and distress caused.

One thing is for sure: this certainly is not the last we have heard about the exam results of 2020. GCSE results are due to be published next week and some teachers, their unions and of course the students are already unhappy with the Government / OFQUAL’s approach. Ultimately, it will in all likelihood be for the Courts to decide whether Article 22 has been breached and I do expect to see this point litigated in the coming months. Students may also wish to complain formally to the ICO.