Eight questions about anti-cheating software

The subject of online proctoring has been discussed in the national media and in House of Representative debates since the outbreak of the corona crisis. Among the political parties that raised questions about the surveillance method were the SP, D66 and the Party for the Animals. For those for whom the subject is hazy, online proctoring software films students during exams with their own cameras. The intention is to prevent cheating. The software also records what was done on their laptop so that students do not copy their answers from Google or other sources. There are various suppliers of anti-cheating software at Dutch universities such as ProcturU, the controversial (in Dutch) Proctorio, ExamSoft and RPNow. TU Delft works with RPNow. While Proctorio is constantly subject to outcry, little is heard about RPNow.

Online proctoring and terms such as privacy objections and technical problems are usually mentioned in one breath. Students from Erasmus University are thinking about going to court to fight against the use of a second camera. Students at the University of Amsterdam went to court last year. They lost. A petition at TU Delft about privacy objections was signed 2,500 times last year. The faculty student councils at TU Delft mostly raised the issue of technical problems. At the start of the academic year – during the Q1 exam period – about 30% of TU Delft students said that they had had technical problems with proctored exams.

Students are now asking Delta what the situation is. We asked Willem van Valkenburg of the Exam Taskforce the following and seven more questions.

1. How many students had techical issues during Q2?
“The percentage was miniscule. In the last exam period, TU Delft’s exam helpdesk received 180 reports from students, of which 27 were because of technical problems. One student reported three times during one exam.” As about 6,000 students do one proctored exam during each exam period, so the percentage of 0.45%.
A strikingly large drop that Van Valkenburg says has several reasons. “The capacity of the call centre which students could contact for help during proctored exams was greatly expanded. The manual for students was also made more concise and clear.” Earlier this year, students and student councils said that some of the technical issues were caused by the ways in which teachers arranged their exams. For example, one teacher at the Faculty of Civil Engineering and Geosciences asked for the explanations behind the answers to be mailed, while the anti-cheating software did not allow this. “Teachers now know better how to test their own exams in advance.”

2. How does RPNow run the images?
There are three methods of proctoring. The method that is quickly becoming the most popular uses artificial intelligence to track ‘suspicious behaviour’ of students. Proctorio is one of the softwares that uses artificial intelligence. A second method, live proctoring, involves an invigilator watching the live images of students who are taking an exam. As this is labour intensive, most proctoring companies work with the third method. Here the images are saved and later watched by an invigilator. RPNow uses this third method.

3. What about the number of proctored exams?
According to the Dutch Data Protection Authority’s national guidelines, teachers may only opt for proctoring if all other forms of examinations – such as oral exams or open book exams – are not possible. In this exam period, TU Delft will hold 540 exams, of which 52 will be proctored. This amounts to 9% of the exams. At the start of this academic year, this percentage was 15%. The difference is partly because Q3 has fewer ‘knowledge subjects’ than Q1. “At the beginning of the year, TU Delft ran more large scale exams in which students were tested on their factual knowledge.  This type of testing uses more online proctoring.” Another issue is that this time last year, online proctoring was not yet widely available. “Because of the lockdown, teachers en masse had to convert their exams to digital exams. And a year later, you cannot convert these to proctored versions without a good reason.”

4. Who will look at my films and how long will they be kept?
Webcam images will first be watched by an employee of RPNow. “This is really the first step. If a student moves a lot, the employee may wonder what he/she was doing.” The RPNow employee sends images that remain suspicious after the initial review to TU Delft’s exam desk. This is staffed by the invigilators who walked among the tables of quiet students before the corona crisis. “Only if the invigilator deems the images as suspicious too, are they sent to the teacher of the subject. Ultimately, the faculty’s Board of Examiners decide if there really has been cheating.”
Webcam images are kept on the RPNow servers for three months. “This is the agreement that we made with the company. It allows time to watch the exams and gives students time if they wish to lodge a complaint.”

5. Where is my data kept?
It is kept in several places. RPNow stores the proctoring data – the webcam images – in the United States (USA). All the answers given by students are stored on European servers. In June of last year, European judges declared the so-called Privacy Shield between Europe and the USA invalid because the agreement would not sufficiently protect the privacy of European citizens. This verdict had far reaching consequences, including for TU Delft.

6. What agreements have TU Delft and RPNow made about my data?
TU Delft already had a processor agreement with RPNow which stated how long data could be stored and who had access to it. After the Privacy Shield was dropped, the Executive Board decided that all current contracts with American companies could continue. “We then made additional agreements and updated other agreements contractually with RPNow,” explains Van Valkenburg. “One of these was that the data could only be used for proctoring and not for other purposes.” He adds that “European and Dutch proctoring companies often store their data in Amazon Cloud (in Dutch). This means that the Privacy Shield is invalid.”

7. Hundreds to thousands of exams were declared suspicious at other universities. Was this the case at TU Delft?
This happened to 1,400 students at Wageningen University (in Dutch) whose exams were done using RPNow. Of these, 10 were actually accused of cheating. More than 100 students at Erasmus University in Rotterdam and about 110 students at the Vrije Universiteit Amsterdam (Free University) were also accused of cheating. Of the Erasmus students, one was proven guilty. The figure for the Free University is not known. At the end of December, no exam was declared invalid at TU Delft. Whether this is still the case, Van Valkenburg does not know. “We do not have the figures from all the Board of Examiners from the last few months.”

8. Can you still choose an opt out exam?
Yes, anyone not wishing to do their exam at home, can do it on campus. For Q3, 26 students signed up for this opt out. “We do not impose a maximum number of registrations anymore, as we did in previous semesters. From Q4, we can thus have more than 30 students per day.”