Humans of TU Delft: student Jan Wehner on the risks of AI

English only

As Artificial Intelligence (AI) systems offer exciting new capabilities, they also pose a lot of potential risks. Through the Delft AI Safety Initiative (DAISI), Jan Wehner and a group of students hope to help students and academics at TU Delft reduce those risks. 

“I’m in the second year of a master’s in Computer Science and I’m doing the track on Artificial Intelligence. At first, I was really excited about this new technology that could do all these new things. Then I became more and more interested in and concerned about some of the risks that come with this technology and that made me want to learn more about it. I want to be able to help make this technology more safe and more beneficial to humans. 

The Delft AI Safety Initiative, or DAISI, is a student led initiative. We’re very new, just about four months old right now. Our goal is to help students and academics at TU Delft to reduce some of the risks posed by AI, especially the more advanced AI that we might develop in the future. We do this in several ways. First, we try to educate ourselves and to help each other understand the problem and approaches we can take to solve it. We explore some of the research directions to develop knowledge in the field and try to develop relevant skills. We also want to stay connected to the community in this field, to have opportunities to network. 

In addition, we organise different activities. We have an AI safety fundamentals programme where, through readings, discussions and a mini project, people learn about this problem and the key directions in this area. We also organise hackathons where, in a fun competitive setting, people can gain experience on this topic but also might win some prize money. And we also have weekly deep dives where we go in depth into a topic that we want to investigate further. We’re in our second iteration of the AI safety fundamentals programme at the moment and have already organised two hackathons which were really fun.  

‘It’s hard to specify what we want AI to do, to give AI good goals’

One of the hackathons was about how to test AI systems, whether they are safe or not. This kind of testing is crucial to determine whether a system is safe to be deployed or not. But we don’t really have any good tests yet to determine this so it was about coming up with ideas for how we could test these systems and then try to implement these ideas to have a ready-made test at the end. 

There are four amazing people who are organising this initiative with me. Three of us are master’s students and two of us are bachelor’s students. We connected because of our mutual interest in this area and we’re all convinced that this is a very important topic to work on. We wanted to learn about it together and bring even more people in. Right now, I think we have roughly 20 to 30 people who are actively engaged with us. I hope that other people will be keen to join us. You can learn more about us on our website, and we also have a WhatsApp announcement chat where people can get information about our upcoming events. 

One of the key risks, I think, is that it’s really hard for us to specify what we want AI to do, to give AI good goals. For example, you might give an agent a goal of getting a really good score on a video game and you hope this will lead the AI to play the video game well. But instead, it might learn a little bug in the game and start to exploit that and it gets a really high score, but it never really learns to play the game. This is a classic case where technically it’s meeting the goal, but it’s not really what you intended. As we get more powerful systems which are very effective at achieving their goals, this might be a catastrophic problem if they have goals that are not actually what we intended them to do. 

Personally, I want to work on this topic in the future, to do research towards making AI safer. I think there’s huge positive potential for AI, but I think to really grasp this positive potential we should first look at the risks. We need to be cautious about employing this new technology and make sure that as we develop a more and more powerful technology that we can actually control it and it’s safe and aligned with human values.” 

Want to be featured in Humans of TU Delft? Or do you know someone with a good story to tell? Send us an e-mail at humansoftudelft@gmail.com 

• Last week, numerous people from science and tech sectors wrote an open letter in which they call on all AI labs to immediately pause the training of AI systems for at least 6 months. Among the more than 1,800 signatories are numerous academics, also from TU Delft. Professor of Ethics and Technology Jeroen van den Hoven (TPM) is one of them. In an interview in the Dutch magazine De Ingenieur (the engineer) he said: “You should not unleash these systems on society without any kind of control.”

Heather Montague / Freelance writer