Anonymous student questionnaires expose lecturers to aggression and discrimination. And what do they really say about the quality of education? The Works Council wants change.
(Artwork: Xiao Feng & Mathieu, XF&M)
“The lecturer often gave the impression of not being familiar with the material, and sometimes I felt that she had no idea of what she was talking about.” Marleen Keijzer cited this comment that a second-year student wrote about her on an evaluation form. “Another student wrote that I come across as denigrating and laugh at students.”
Pure nonsense, but the comments affected her, said Keijzer, who has been happily teaching mathematics at TU Delft for over 30 years and joined the Works Council a year ago. “While cycling home, I thought: They can’t be talking about me. I never laugh at students, I’m not that kind of person. But I had a bad day anyway.”
Sleepless nights because of unfounded criticism that students pour into the questionnaires intended to evaluate the education are experienced widely in higher education. Dissatisfied students use the anonymous questionnaires, TU Delft uses those from Evasys, to spit out their bad mood bluntly, and the lecturer is fair game. Hurtful remarks about their lack of didactic and specialist training can be supplemented by critical comments on their clothing, appearance or English accent.
‘Sometimes a lecturer is trampled all over’
“There are also valid arguments among them, but some students just vent their frustrations on the lecturer,” explained Coba van der Veer, pabo (primary school teacher training) lecturer at the Ede Christian University for Applied Sciences and sector manager hbo (higher vocational education) at the AOb (General Union of Education).
She not only receives comments about her own performance, she also sees those that her colleagues get. “Sometimes a lecturer is trampled all over.”
“Quite recognisable,” agreed Marijtje Jongsma, senior lecturer at Radboud University and sector manager at the AOb. “I have been teaching the same subject for 20 years, always with good reviews, but every year there is one or two students with very cutting criticism. I can’t sleep properly for a week after that. And you realise that the worst attacks had already been filtered out.”
The questionnaires lead to social insecurity, according to the lecturers. “It is anonymous criticism that we cannot defend ourselves from,” said Keijzer. “Experienced lecturers know to take student evaluations with a grain of salt, but what if you are just starting in your career? You might stop teaching altogether.”
“No rebuttal is possible, students can say anything without consequences. That turns the work climate ugly,” expressed Jongsma. “The anonymous questionnaires stimulate mistrust between the student and lecturer.”
“There are also statements that make no sense. For example, I got nothing from this subject, I never even attended a class,” said Van der Veer. “I want to take the students seriously, but that is difficult to do like this.”
Chocolate cookies
If the questionnaires produced reliable information about the quality of the education, lecturers could possibly ignore the rude comments. But the standard questionnaires in which students assign scores for the organisation of the education, the educational material or the level of the subject content, which are indeed called course evaluations, more reliably measure student satisfaction, as revealed by dozens of scientific studies.
Two years ago, three pedagogic advisors of the University of Utrecht listed the recent insights in a journal for managers in higher education. The disconcerting conclusion from a study in 2018 was that students evaluated a course significantly better if they were served chocolate cookies during the lectures. And on sunny days the students gave higher scores than on rainy days, as was evident from a comparable study from 2014.
‘The response to online questionnaires is often below 10%’
A high score thus pretty much stated that the students enjoyed themselves, but says little about the quality of the lecturer. That’s only logical, felt the Utrecht education advisors. A student can possibly evaluate a lecturer’s clarity and availability, but it is difficult for them to judge didactic qualities and specialist expertise.
Something is wrong with the standard questionnaires. Evaluation scores are not associated with the learning outcomes, which was revealed by an analysis of all available studies in this field from 2017. A high score does not necessarily mean that the education is good. Other unpleasant aspects: female lecturers receive lower scores than their male colleagues, especially from male students. The women’s lower score refers to the form of knowledge transfer and the amount of knowledge they possess in the students’ opinion, even if their level is comparable.
And then there is the notorious low response rate. That problem is particularly evident with online questionnaires as they have a much lower response than written evaluations in the lecture hall. Research revealed that only very committed or extremely dissatisfied students complete the digital questionnaires, which of course produces a distorted image. “That is why we have occasionally brought the students together in one room with free confectionery and soft drinks to ensure that they completed the questionnaires,” laughed pabo-lecturer Coba van der Veer.
“At TU Delft the response to online questionnaires is often below 10%, even after we sent lots of reminder e-mails,” described Marleen Keijzer. “But the outcome is still presented as the students’ opinion of a subject.”
Fixation on numbers
Although the evaluations are not intended to be an assessment instrument, low scores do have consequences for lecturers. “Managers are aware of the relativity of these scores, but as the lecturer, you have to account for them,” explained Van der Veer. “If you score poorly a couple of times, you have to be able to show improvement.”
Marleen Keijzer sees the same pattern at TU Delft. “The numbers are shared with the students and lecturers in the Education Committee, with 30 to 50 immediate colleagues and with your programme director. These are the people who decide whether you will continue to teach a subject, whether your subject should be scrapped, whether you get a promotion or a permanent contract.”
Keijzer already presented this dissatisfaction with the student questionnaires to the Works Council last year, and on April 15 the topic was back on the agenda. The Council unanimously requested the Executive Board to revise the student questionnaires drastically. “With their anonymity, the questionnaires release aggression, they discriminate against women, and the outcomes do not say anything about the quality of the education. They also appear too late, once the course has been concluded. In other words, there is no benefit from this instrument,” concluded Keijzer about the OR’s position.
“The Works Council believes that they must no longer be anonymous,” proposed OR-chair Menno Blaauw. “Students can give very useful feedback, but anonymous questionnaires do not contribute to a sound discussion of the quality of education. Given the nature of the questions and the fixation on collecting figures, lecturers can do little with the outcomes. There are more useful ways to gather feedback. A good comment from a student who has thought about the education is worth far more than a pile of numbers.”
Honest feedback
Executive Board member and vice rector Rob Mudde sees it differently. “I feel it is important to hear the opinions about the education of all 27,000 students. In the open questions they do sometimes give blunt critique, but personal attacks are filtered out by the quality care staff. Once in a while a nasty comment slips through, that is unfortunate. I have also been a lecturer and occasionally encountered nasty comments or obscene language, but I just shrugged my shoulders. They are just incidental, and I do not want to lose the possibility of hearing the opinions about the education of all students as a consequence.”
‘We must offer students a form of security’
Mudde attaches great value to the anonymity. “The lecturer is the one who evaluates the student, so there is the risk that a student will not feel free to criticise. As a result, students may not honestly say what they think about the education; instead they may say what they think I want to hear. We must offer students a form of security to receive honest feedback.”
The Executive Board member recognises the tension between protecting lecturers against unfounded criticism and giving students the room to express their opinions. “I understand that lecturers do not benefit from being sworn at, but we are not stopping the questionnaires. We feel they are too valuable. If I were to get rid of them, I would immediately have a problem with the Student Council.”
Mudde believes that improvements are possible, even if he doesn’t know what they are. “It is a search for an instrument that gives all students the chance to express their opinion, but in the form of well-supported feedback and not blunt criticism. A lecturer has to be able to cope with criticism, but that doesn’t mean listening to insults.”
Princes and princesses
“Universities consider it highly important to have satisfied students,” stated Job van Luyken, ombudsman at TU Delft. “But the average student is not afraid to speak up, and there are quite a few princes and princesses among them who are used to being pampered.”
The ombudsman feels that consumer behaviour is stimulated by the student questionnaires. The problems noted by the Works Council can be found all over TU Delft in his opinion. “The majority of the lecturers never mention it, but we hear regularly from lecturers who are bothered by the anonymous reactions. They have a large impact, students can really demolish lecturers.”
Van Luyken is definitely not against evaluation, and he is happy for students to judge the education, but the Executive Board should make more of an effort to counteract the anonymous ‘rude remarks’ of students ‘that target the person and not the content of the course’. “It is quite serious that a moderator has to be employed to filter out the unacceptable comments.”
That problem should be discussed in the Student Council, he feels, but that is not being done. “It is filtered out and ignored. The problem is made invisible. This leads to an unsafe work climate. It’s not so much the content of the comments, but the lack of action taken when a student goes too far. That makes it unsafe for the lecturers.”
The Works Council will not let the matter go, as the meeting of 15 April showed. The council plans to present a plan with alternatives for the Evasys surveys before the summer.
This article previously appeared in Onderwijsblad magazine. We have later added the paragraphs from the Works Council meeting of 15 April. The Student Council’s response to this article we requested did not come in before the deadline. Keep an eye on our website for a follow-up.
Artwork by Xiao Feng & Mathieu, XF&M.
Translation by Alison Fisher.
Do you have a question or comment about this article?
delta@tudelft.nl
Comments are closed.