Why AE students have so many proctored exams

Online proctoring is not new. TU Delft has been doing it since 2014. But the use of the anti-cheating software has spiked given the corona restrictions and has given rise to a heated debate. Supporters point to the fact that exams can go ahead as they normally would without too much disruption thanks to the anti-cheat software. Students do not have to worry about study delays and teachers do not have to worry about cheating. That is the idea anyway. Detractors see online proctoring as a breach of privacy given that students are watched during their exam through their webcam and microphone. On top of this, they argue that the way exams are run is quite stressful, they worry about the percentage of students that experience technical problems during proctoring, and they point out that students can still cheat even with online proctoring.

The Dutch Data Protection Authority decrees that online proctoring can only be used if there are no alternative exam options. TU Delft complies with this directive. However, one thing stands out for anyone looking through the Faculty of Aerospace Engineering’s (AE) exam schedule: apart from a few other forms of testing, the option chosen most frequently is online proctoring. This semester (Q2), nine of the 11 bachelor subjects were online proctored. For the master subjects, this was 16 of the 31 exams. This puts AE well above the average of other faculties. TU Delft wide, there are 43 exams using anti-cheat software on the programme, of which AE alone accounts for 37%. What is going on?

Freedom of choice
The teachers are the usually the ones who choose how exams will be taken. The corona restrictions meant that they again had to pore over numerous considerations. They asked themselves whether students only need to reproduce the material or if they need to demonstrate how to use it creatively? How easy is it to cheat, which subjects are most vulnerable to cheating and, by extension, which exam method should they use? Alternatives to online proctoring are easier to find for some subjects than others. In master subjects that have a relatively small number of students, one option is to do oral exams. This is impossible for bachelor subjects that have hundreds of students. Teacher René van Alderliesten, who added a proctored exam for the Introduction to Aerospace Engineering II to the schedule this semester, explains. “This is an introduction to a discipline so we have no options such as a particular assignment. Furthermore, given the high number of students we could not go for oral or physical exams on campus.”

While teachers may have some degree of freedom of choice, in reality they are not completely free to choose. Their plans need to be approved by the Board of Examiners and they have to comply with the exam regulations and directives that are defined by that same Board. This is the crux of the matter. AE’s Board of Examiners made itself strongly heard this year.

Clear preference
It is the job of the Board of Examiners to monitor the quality of exams and, in doing so, ensure the quality of AE courses. On 25 September, AE’s Examination Board emailed all the Faculty’s teachers about the requirements for the exams to be held later that semester (Q1). It was the usual type of brief, formal email that is sent every semester. Except for one point. Page one of the annex in which the requirements were given for the exam period stated ‘Remote proctored exams are the preferred replacement for written exams in Q1 that cannot be converted to assignments or orals’ and ‘A limited number of remote non-proctored exams are permitted in Q1’.

Officially, these two sentences contravene the core directive of TU Delft and are thus not allowed. A statistical investigation into cheating by teacher and Chair of the Examination Board, Dr Richard Dwight, for his bachelor subject Applied Numerical Analysis serves as an example of contravening the directives. Dwight found that about 5% of the students worked together on the entire exam. “For us, the exams at the end of the last academic year were a real-life experiment on fraud. All on-campus tests were converted into non-proctored online versions as quickly as possible. My research shows that one group of students gave identical answers. This shows the likelihood of this group working together on the entire exam.” Does this statistic apply to other subjects? “Good question,” says Dwight now, “but these are the only figures we have so we have to work with them.”

While the Faculty’s Student Council protests the Q1 directives, the Board of Examiners refuses to make major changes. Looking back, Dwight admits that the Board may have responded too strictly. “We may have been too extreme in our policy in Q1, but it arose from genuine concerns.” The concerns are for both the outcomes of the fraud investigation as well as the quality of the degree.  “Our ultimate nightmare is that the students who graduate during the corona crisis earn a ‘corona degree’, a paper of little value because potential employers think that they cheated in their exams.”

‘We may have been too extreme in our policy’

It was only after the Exam Task Force led by Willem van Valkenburg and the Examination Board pointed to the main rules that students had a greater say. “This semester, Q2, we have been involved in the exam regulations from the start,” says Gijs Vugts of the Faculty’s Student Council. “That is good as, officially, the Student Council does not have a monitoring role vis-à-vis the Board of Examiners and Dwight did not have to contact us.”

The current exam regulations no longer state that the Faculty prefers online proctoring, yet, teachers still have limited space to choose. Teachers who choose non-proctored exams are required to submit a detailed anti-cheating plan. Even teachers that use the controversial anti-cheating software need to justify why they opted for it to the Board of Examiners. The difference now is that a brief explanation is enough. Alderliesten says that “The guidelines leave little space to choose anything other than online proctoring.” Teacher Julien van Campen, who chose a combination of six non-proctored module assignments and one proctored final exam for his Dynamics subject, agrees. “Proctored exams are the default option in the guidelines.”

Aftermath
On top of this, this semester (Q2) is dealing with the aftermath of the stringent requirements imposed in the last semester (Q1). The figures of the cheating investigation are still in the minds of the teachers. Van Campen interprets an email from the Board of Examiners dated 10 November, which is in the possession of Delta, as saying that online proctored exams are still the preferred form of examination. “I have repeatedly been told that non-proctored exams are less desirable,” he says. He finds this a pity. “In the last academic year, I offered a non-proctored resit for Dynamics and that was much more relaxing as I, as the teacher, had much more control. But this time? The communication around the online proctoring software is very limited for us teachers. My colleague and I had to beg for extra instructions the week before the exam. And we notice that we are not the only ones that are more nervous this time, the students are too.”

Earlier, Delta had announced that technical problems had occurred in about 30% of the online proctored exams. He does not see this figure in his own exam. “I would guess that about 10% of the 435 students had problems.” However, there were two students who were thrown out of the exam and did not come back. “That is terrible.” The Board of Examiners will have to decide how to proceed with the exams. The Faculty is offering technical resits for students who incur major problems during their proctored exams.

Misunderstandings
Dwight recognises the tension around proctored exams. “But I also see that there are many misunderstandings. Students think that they will be flagged (flagging means that a screenshot of a student is taken during suspicious behaviour, eds.) if they look away even for a second. This is not the case. Furthermore, there is a huge gap between being flagged and officially being accused of fraud. There are countless checks and balances between these two.”

There are no hard figures on fraud rates in online proctoring

The Examination Board and the Faculty’s Student Council will soon review the regulations for Q3. Neither party has changed its position. For Dwight and his colleagues, online proctoring is still the best way to avoid fraud, while Vugts and the other Council members point out that students can still cheat even when under the surveillance of anti-cheat software. There are no hard figures on fraud rates in online proctoring, but the statistics of the Dynamics exam may be something to go by. “In our exam, we saw that some of the answers in the proctored version were strikingly better than in previous years when on-campus tests were the norm,” says Van Campen. He does not want to use the word ‘cheating’, but he does say that the figures are ‘striking’.

Despite the difference in opinion, Vugts and Dwight are confident that they will reach agreement. “The Board of Examiners is involving us a lot better in defining the guidelines,” says Vugts. In turn, Dwight says that “Every semester is a learning opportunity, and these lessons are reflected in the guidelines.”