Online proctoring: ‘I was in tears at my laptop’

Computer Science and Engineering bachelor student Kevin Charm is sitting at his laptop in his room in Delft. His hands are resting on the keyboard, his eyes staring at the screen in front of him. He does not dare look away. He has just started on a new question in a proctored exam for his Airport of the Future minor. He thinks, fills in his answer and clicks to the next question. This time the question loads much more slowly. A lot slower. “As long as nothing goes wrong,” he thinks. When he finishes the last question and wants to read everything through, his screen freezes. He is thrown out of the exam and cannot submit his answers. Charm feels his heart race and tries not to panic. “Contact the helpdesk quickly,” he thinks while watching time slip away. He also gets onto Brightspace, the recommended place for help during exams. Half an hour later, having received extra time from his teacher, he is finally able to submit his answers. Two days later it goes wrong again during another proctored exam. And this time it is not only him, but his fellow students too. “You saw everyone in the shared chat start to panic,” he says.

A Delta questionnaire among students and student councils showed that exam software is often unstable during exams. About 30% of the respondents in a Student Council survey at the Faculty of Aerospace Engineering (A&E) experienced technical problems during online proctoring. TU Delft too sees the same percentage emerge from surveys, says the Chair of the Exam Taskforce Willem van Valkenburg. The Student Council at the Faculty of Mechanical, Maritime and Materials Engineering (3mE) asked students in a questionnaire about the type of technical problems that they experience. The most common problems were:

• being thrown out of an exam, for example the exams that RP Now combines with Maple TA (Möbius);
• being denied access to the exam itself because the exam is not ready in the RP Now software;
• having problems with the webcam or microphone in RP Now while they can work without problems with other software.

The solutions are often just as varied as the problems. At times a TU Delft teacher can help right away, at other times, it takes a few phone calls to a helpdesk set up specially for proctoring before a solution can be found. Van Valkenburg says that “Problems often arise when RP Now is combined with TU Delft’s exam environment. This always makes it complicated.”

Directions are not clear 
Surprisingly, it is not always clear what students need to do if something goes wrong during exams. A&E Student Council member Gijs Vugts says that “Sometimes students only get a helpdesk email address that is only checked sporadically. They also do not know if they may open their mail, check Brightspace or use their phone to ask for help if something goes wrong. If TU Delft would give a telephone number of a helpdesk for each exam, it would make a huge difference.” Eva Slingerland (Lijst Bèta) also raised this same point at the Central Student Council’s monthly consultation meeting with the Executive Board. “Students need a telephone number, support and clear directions.”

Over the last few months, TU Delft has taken measures to minimise the technical problems. Exams may only be held during the day and may not take more than one and a half hours. “We saw that students were thrown out of the software more frequently in the evenings and in exams that lasted more than one and a half hours, and there were more problems with the internet connections,” explains Van Valkenburg. TU Delft has also established a helpdesk for students with technical problems. Van Valkenburg and his team are also trying to minimise the number of proctored exams. “We are doing this even though we are seeing that the questions of non-proctored exams appear on sites such as studeersnel.nl soon after the exams. This means that teachers continually need to put time and effort into thinking up new exams. If this continues, they will opt for proctoring more often.”

Extra stressful
And few students want this. Many students experience greater worry, tension and uncertainty with proctoring. Charm says that “Time passes while you’re sitting and wondering if your problem will be solved. It’s terrible. I was completely stressed out.” Even when proctored exams go well, students say that the stress level is higher than the ‘ordinary’ exams on campus. Of the 200 respondents in the Faculty of A&E, 140 say that they find proctoring more to a lot more stressful.

Student Council members Raoul Bruens of the Faculty of Electrical Engineering, Mathematics and Computer Science and Joyce Rijs of 3mE see that this is also an issue in their faculties. “You used to think,” says Rijs, “that you’ve finished studying and only need to get through the exam. But with proctoring you now need to spend your time on all sorts of other issues.” As proctoring software checks rooms for means of cheating, students may not leave books or papers lying around. Walls need to be bare and other screens covered. Going to the toilet or eating and drinking during exams is also not permitted. Bruens adds that “Students are affected by the mental pressure that this causes. This is on top of the normal exam stress.” During the first quarter, Computer Science and Engineering bachelor student Mika Turk only had non-proctored exams. A relief. “Finally not needing to worry if the connection will work or if it remains stable.”

Van Valkenburg regularly consults student councils about problems that arise. And these discussions go well, both he and the student councils say. “Proctoring is a process of continuous learning and adjusting,” he says. Van Valkenburg and his team are currently putting the following measures in place to ensure that proctoring will go more smoothly in the forthcoming quarters.

• Expanding the number of helpdesk staff so that students are helped more quickly.
• Giving the helpdesk a dedicated telephone number. “We hope that this will be done before the second quarter’s exam week. We cannot guarantee this as we first have to take on more helpdesk staff,” explains Van Valkenburg.
• Potentially spreading the start times of exams. “We see a rise in the need for support shortly before exams start.”
• An infographic that clearly explains the steps that students should take in case of technical problems.

While technical problems rarely have far-reaching effects, there are exceptions. Of the 6,000 students that worked with online proctoring this quarter, a small number were unable to start their exams because of technical problems. “That number is very small,” says Van Valkenburg. One of the students who was unable to take his exam was Aerospace Engineering master student Mauryze Brug. He had to try to reinstall the RP Now software on the day of his exam as he kept getting the same error message. “The error message was about my microphone and you may not start an exam without a microphone.” He tried everything, but the error message kept coming back. Even the helpdesk staff could not get the software to work. And he was unable to borrow a laptop from friends on time to do his exam. “I doubt if it would have made a difference if I had tried to install it earlier,” says Brug. “And anyway, it should not be my problem in the first place. If we didn’t have to do online proctoring, I could simply have done the exam.”

Exams on campus
Another student that Delta spoke to was unable to access her exam anymore after the RP Now software broke the connection. Contact with the helpdesk did not help and while her teacher thought that the problem lay with RP Now, the helpdesk thought it lay with TU Delft. “I couldn’t bear it and I sat at my laptop in tears,” she says. As her case is being looked into by a faculty’s exam committee, she does not want her name published (her name is known to the editorial board). Her story is supported by screenshots that are now in the possession of the editorial board. “I studied hard for this exam and wanted to get this subject done at any cost. But now those five credits have gone up in smoke.” In this case, the faculty’s exam committee will determine how to proceed.

Students who object to proctoring this quarter may, as in the first quarter, apply for an opt-out. In this case they do not do the exam from home, but under the supervision of invigilators on campus. The deadline for applying for an opt-out for the exams in the second quarter is 10 January. “We designed the opt-out in line with the national restrictions of the strict lockdown,” explains Van Valkenburg. “It is up to the Executive Board and the Mayor of Delft if the opt-out option will be permitted.”

Tips from students and from Willem van Valkenburg.

• For teachers: make sure that, on top of an email address, a telephone number for the helpdesk is included in the exam information. “This will help students get help much more quickly should anything go wrong during the exam,” says Student Council member Vugts.
• For teachers: always make sure there is a community chat option during the exams. Rijs explains that “Should anything go wrong, it is comforting to be able to see on the chat that you are not the only one.”
• For students: make an extra profile on your laptop that you only use for proctored exams. Bachelor student Charm explains that “This helps avoid pop-ups from other programmes suddenly appearing. Pop-ups can be seen as cheating.”
• For students: try to install the RP Now software a few days before the exam instead of on the day itself. “This gives the helpdesk enough time to solve any problems,” says Willem van Valkenburg.