Some TU Delft teachers fear that AI is steadily eroding their students’ critical thinking skills. The worst-case scenario: an entire generation of engineers with poor analytical skills. How are instructors trying to turn the tide?
(Photo: Justyna Botor)
Assistant Professor Bahareh Adbi (Faculty of Electrical Engineering, Mathematics and Computer Science, EEMCS) is still visibly puzzled when she reflects on the AI antics of her Electrical Engineering bachelor students.
It started to dawn on her during a second-year project course. “ChatGPT had just been released – I don’t think I even had an account myself yet – and from one day to the next, half the reports were written very neatly.” Jokingly she said “I could suddenly read them without suffering.”
That might sound like a win, but Abdi and her colleagues soon found themselves in an unworkable situation. Students use AI language models for absolutely everything.
“If I point to AI usage now, I sometimes get an AI-generated response thrown back at me. I’m then talking directly to ChatGPT. Students don’t think for themselves anymore.”
AI dependence is huge
Many teachers were caught off guard by the rapid rise of ChatGPT and, consequently, of other AI chatbots. Many secondary school students (in Dutch) rely on generative AI to do their homework, and many bring this practice to university with them.
This is also the case at TU Delft, as evidenced by a survey sent out by SPICE, the Teaching Academy initiative: nearly 50% of EEMCS respondents admitted to suffering from some form of AI dependency. At Mechanical Engineering (ME) and Technology, Policy and Management (TPM), that figure was as high as 80% and 85% respectively. In a recent Delta interview, several AI enthusiasts painted a similar picture.
Thus, university educators are in a bind. On top of their regular time consuming research and grading work, they have a new challenge on their hands. They constantly have to monitor students for chatbot use, point out the pros and cons, and devise new AI-proof assessment methods. How are TU Delft instructors managing this new reality?
More readable results
“Many of our students struggle with writing,” Architecture teacher Jurjen Zeinstra notes matter-of-factly. He was thus highly enthusiastic at first. Since the introduction of ChatGPT, students writing essays under his supervision produce much more readable results.
‘Writing is, of course, always a form of thinking’
“That’s great, because although writing skills are important, I don’t see it as my job to constantly correct students’ language. I was also immediately interested in AI myself. I set up a course with two PhD students during which we experimented with floor plans, ‘stitching them together’ using AI.” The models weren’t quite up to the task yet, but Zeinstra believes it’s only a matter of time.
Still, the downsides of widespread AI use quickly became clear to him as well. His biggest concern is that “Writing is, of course, always a form of thinking. You try to put something into words, and then you polish the text to gradually make it better.” That thought process is often lost now because falling back on AI is easier, Zeinstra argues.
Write better, think less
It touches on a fundamental issue that seems to affect all faculties: critical thinking skills are rapidly declining among students. Electrical Engineering lecturer Adbi has also noticed this trend, she explains in an office on the lower level of the EEMCS building. Abdi is part of the educational team for the Electrical Engineering bachelor’s programme, and shares her story alongside colleagues Seyedmahdi Izadkhast and Ilke Ercan.
As evidenced by the suddenly easy-to-read texts, ‘their’ aspiring electrical engineers were early adopters. But according to Abdi, things were really turned upside down during the following academic year when, in addition to English, ChatGPT began to master Python, the programming language.
‘Suddenly, no one was crying’
“In the first quarter of the first year, our students have to learn the basics of Python. Some assignments are so difficult, that some students would burst into tears.” But then ChatGPT came along. “Suddenly, no one was crying or asking questions, everyone dutifully turned in all their assignments, and the results were excellent.”
“Until they had to take a programming exam without AI,” adds Ercan, her colleague and an associate professor. “The grades were disastrous. The gap was immediately apparent: formative assignments went exceptionally well, but performance in the classroom and during exams plummeted.”
Programming skills built on quicksand
The result: the programming skills of the first-year students are built on quicksand. Abdi explains that “They do write code and complete the task, but that’s as far as it goes. They can’t critically evaluate what ChatGPT presents to them. They lack analytical thinking – a key part of what we’re trying to teach them here.”
When that realisation sank in, the Electrical Engineering Department went overdrive. Banning AI would be impossible, that much was clear. So teaching and assessment methods needed to be overhauled in order to prevent students from breezing through their bachelor’s degrees.
Comprehensive, crucial and enhancing
One method was assessing written work for use of typical AI language. Ercan points to a study in the Informatics journal that examines abstracts of masters’ theses, carried out by three colleagues from the Mechanical Engineering Faculty. The results were clear: since the launch of ChatGPT, words like comprehensive, crucial, and enhancing appeared much more frequently. Meanwhile, the use of standard terms like using and the remained constant.
“Since then, we’ve advised instructors to pay extra attention to such terms,” says Izadkhast, Associate Professor of Electrical Engineering. Should there be any suspicion of AI use, students are asked to take an oral exam. “Through questions that delve more deeply into the subject matter, we verify whether they truly understand the answers they give.”
No more take-home writing assignments
At his TPM Faculty, Ofke Teekens, an educational advisor, is also trying to keep the AI revolution at bay. “Most likely, nowhere else on campus is writing more prevalent than here. So we knew right away we had to do something.”
‘Instructors are free to determine how much AI use they allow – within the rules set by the Board of Examiners’
Even before most students discovered ChatGPT, he and his three fellow educational advisors began looking for ways to nip chatbot fraud in the bud. With relative success, says Teekens. “We try not only to look at the end result, but also to keep a close eye on things during assignments by talking to students about their work.”
There are no strict, faculty-wide rules about AI in education. “Instructors are free to determine how much AI use they allow – within the rules set by the Board of Examiners, which explicitly do not allow plagiarism. But they do so with support and guidance from us, and from Teaching Support.” In most reports, for example, students are usually required to mention where and how they used AI. And some writing assignments, which formerly could be taken home, are now completed under supervision in an exam hall.
Postcard to grandma
Educational Advisor Teekens does however emphasise that many students do realise that relying too heavily on chatbots is a bad idea. “They’re smart people, after all.” And by no means do all courses require adjustment, he says. “If an exam is the only assessment method for a course, students will simply have to study, with or without AI.”
‘An AI ban is pointless and undesirable. It would be like having banned Google 30 years ago’
Because enforcement is all but impossible, a complete ban on AI would be a futile endeavour, Teekens believes.
Architecture instructor Zeinstra goes a step further. “An AI ban is pointless and undesirable. It would be like having banned Google 30 years ago.” He also questions the practice of writing essays in exam rooms, something which Architecture master’s students recently had to put up with. “You’re forcing students to do something they normally never do, except when they send a postcard to their grandmother.”
Moreover, there is a vast legal grey area surrounding AI fraud, within which both instructors and exam committees must operate. Yes, some cases are crystal clear – think of the UvA (University of Amsterdam) student who copy-pasted the phrase ‘thought for 4 seconds’ into her report. But proving a suspiciously vague and polished sentence was written by AI? No chance.
Electrical Engineering instructor Abdi explains that “A colleague once argued this in front of the exam committee and got into serious trouble. Some students fiercely fight against these kinds of accusations.” Ercan adds, “That’s why I don’t punish them with low grades, but with extra work. I ask them: can you rewrite this, but in your own words, without AI?”
Climate and Big Tech
And then there are the ethical issues surrounding generative AI, which education advisor Teekens believes deserve more attention. “There are also people at TU Delft who say ‘just embrace it, and give everyone a paid licence’.” This would benefit students who cannot afford licences, says the Educational Advisor. “But then you’re ignoring AI’s impact on the climate and its dependence on Big Tech.”
‘Is AI handing over our data to the U.S. Department of Defense?’
Ercan: “We worry every day. About education, but even more so about the societal consequences of AI. Is AI handing over our data to the U.S. Department of Defense? Is AI helping our students plan suicide?”
The red light issue – the declining cognitive skills of TU students – isn’t resolved either. Although the decline worries Teekens too, he does see teachers in his faculty growing accustomed to the new reality. The debate about whether AI will be reined in or fully embraced at TU Delft will persist for some time, he believes.
Policy document
Abdi and her colleagues aren’t quite out of the woods yet. “It’s almost impossible to keep up with the developments. For example, most of our students have now switched to Claude, which makes it even harder to spot AI generated work. But we’ll adapt no matter what. Maybe in the future a good engineer will be very good at writing prompts.”
Electrical Engineering colleague Ercan thinks help from TU Delft’s top administrators is long overdue. “We have to figure a lot of things out on our own. Teachers from different faculties do somewhat support each other, but there’s hardly any campus-wide guidance or direction. In fact, given the budget cuts, we’re actually losing much needed manpower.”
Abdi says that one bright spot is that an AI policy document is in the works. Way too late, but it would still be “nice to be able to rely on general guidelines, rather than having to figure out everything ourselves.” Exactly what the document entails, and whether it will be of value to Abdi and her colleagues, remains to be seen.
This part 2 of a short series. Part 1 was about AI use by students. Part 3, about the (lack of) policy by TU Delft, will follow.
Do you have a question or comment about this article?
nathan@hgsmk.nl
Comments are closed.