How an internal TU Delft corona research became a political battlefield

It was quite an easy calculation, a sort of finger exercise. When Bas Kolen and Pieter van Gelder of the Safety and Security Science Research Group at the Faculty of Technology, Policy and Management (TPM) wrote their piece entitled ‘COVID-19 risico voor evenementen, Addendum’ (COVID-19 risks for events: Addendum) in September, they could not have guessed the level of attention the piece would get and the level of discussion that it would bring about.

Their calculation was intended to give an initial impression of the number of corona infections that you could expect if you organised a large event. It was a sort of quick scan, an internal piece what’s more, written after last spring’s Field Lab experiments. It did not have any margins of error They were not really necessary, thought the researchers. After all, the colleagues with whom the researchers brainstormed also understood the limitations of the calculations.

TU Delft’s risk assessment model
But the document did not remain internal. Far from it. At the request of the House of Representatives, all Fieldlab documents were made public in September. The TPM piece played a major role in the political discussions about the wide introduction of the 2G restrictions. 2G policy denies entry to locations with a high risk of corona infections to those refusing vaccination, It was used by the Cabinet and the Outbreak Management Team (OMT) as part of their scientific argumentation.

When the permanent Parliamentary Committee for Health, Welfare and Sport asked (in Dutch) the Minister of Health Hugo de Jonge to share the complete calculations of the model concerning the different applications of the corona pass (3G, 2G, 3G+1G, 2G+1G) with the House of Representatives, he sent the House Kolen and Van Gelder’s report.

In the correspondence betweeen the Cabinet and the House of Representatives, the Addendum is frequently referred to as the ‘TU Delft’s risk assessment model’ or as ‘TU Delft’s research on 2G’. The TPM researchers have become the football in an unpredictable political process that sometimes loses sight of the context.

‘We would rather have debates through scientific channels’

“It is fine that the knowledge and outcomes of the research are used,” says Kolen. “Given the number of uncertainties it is important to stop and consider the options in models and conclusions that you connect to outcomes. Ideally these discussions take place in scientific channels, but with a sensitive topic such as corona this is hardly possible. People take parts of your research and discuss them in different contexts.”

The TU Delft saga
The TU Delft saga actually started last spring when the researchers delivered their first contribution to the Fieldlab experiments organised by the events sector. Jointly with Radboud UMC they used the findings to design a risk model to assess how events could be organised safely during the pandemic.

An update followed in September. They then also looked at the possible effects of vaccinations and the impact of the Delta variant. One example is worked out for the purpose of illustration in this version, the much discussed Addendum. It is a fictitious unlocated seven hour dance event with hundreds of thousands of visitors. Delta reported on it in mid-November. The 2G entry policy at events would lead to 25% fewer infections and 93% fewer hospital admittances than a 3G policy, was the conclusion.

In November the TPM researchers used their model to scrutinise infections during the Amsterdam Dance Event and the Formula 1 in Zandvoort. They also carried out a sensitivity analysis on the potential impact of a lowered vaccine effectiveness. Here too the model shows that you can avoid more hospital admissions with 2G than with 3G, but the advantage curve flattens as more people are protected through vaccination or recovery from an infection.

‘The foundations are very shaky’

These may be interesting figures but they are absolutely not intended to be the foundation on which a national corona policy is built. The studies say nothing about either the chance that infected people infect other people after an event or the risk that people may falsely feel safe under 2G and therefore follow the corona regulations less strictly. Kolen and Van Gelder have always been clear (in Dutch) about this, in vain so it seems.

At the end of November the Parliamentary Committee responded critically to the Cabinet’s input. A decision that is so sensitive in society whereby the 2G policy denies entry to locations with a high risk of corona infections to those refusing vaccination, must be substantiated very carefully.

And this was not the case. Afterwards, Lisa Westerveld, GroenLinks Member of Parliament, said (in Dutch) to the investigative journalism platform Follow the Money: “The foundations are very shaky. We have asked critical questions about the application of the models and adopted assumptions, but received no or limited answers.”

‘The Cabinet is sticking tightly to the corona pass’

Faced with a long list (in Dutch) of questions and notes from the House of Representatives – that not only refer to the TU Delft research – on Monday 6 December Minister of Health De Jonge retracted the proposal for a change in the law that would make the 2G ruling possible. But he says (in Dutch), “the Cabinet is still resolved to include the CTB (corona pass, Eds.) in the repertoire of measures to be adopted on the basis of 2G.”

New research
At the beginning of next year, the issue will be the subject of debate in the House of Representatives and the Cabinet again. Prior to this the House of Representatives will be sent more scientific findings. These findings will be derived from several sources including a follow-up study that the people from TPM are now carrying out with statisticians, virologists and epidemiologists of Utrecht University, the Erasmus UMC, Erasmus University and the Delft start-up Populytics. This research is being done in connection with a motion (in Dutch) submitted on 3 November by GroenLinks Member of Parliament Westerveld.

“We will look at the effects that different types of corona entry passes could have on the reproduction figures in the Netherlands,” says Kolen. “People who have been at certain locations can infect people. We will look at how this works and what the uncertainties are. We will thus refine our models. We will also look at the effects of the age of the people who attend events on the spread of the virus.”

The researchers have to tread carefully as their work is being closed followed. While Westerveld abhors the actions of the Cabinet, she praises (in Dutch) the work of the TU Delft researchers. However, other feelings prevail elsewhere in Parliament and doubts are cast over the scientific merit of the researchers. Nicki Pouw-Verweij, Member of Parliament for the JA21 party submitted Parliamentary Questions (in Dutch) to De Jonge. One of her questions was whether the Minister believed the TU Delft research to be reliable.

Pouw-Verweij submitted the Parliamentary Questions after Follow the Money published an article in which a statistician, a health economist and a professor of integrity criticised the Addendum.

‘You should see the margins of uncertainty’

One of them, Medical Statistician Maarten van Smeden of the UMC Utrecht said to Delta that he thinks the TU Delft model is ‘messy’. “The researchers pile up assumption after assumption about infection risks and infectiousness. It is not always clear if there is any basis to these assumptions.”

One of the other researchers quoted in the Follow the Money article is Professor of Integrity at the Free University of Amsterdam Rob van Eijbergen who talks about ‘vague assumptions’. He says that he looked at the arguments of the Cabinet ‘with surprise’. He even talks about ‘volksverlakkerij’ (a sham).

Van Smeden also finds it bizar that the researchers hardly carried out any sensitivity analyses to show how far the outcomes would change if the parameters were adjusted. “You want to see margins of uncertainty for the ultimate figures. You can hardly call this document scientific research,” he concludes.

Kolen is all ears when it comes to substantive criticism. “But preferably on the full context of the study. You can of course have differences in opinion about assumptions in models, such as those in the Addendum. It is good to discuss them. But this looks at just one calculation sample in a long sample set.”

Kolen believes that if the Ministry refers to the TU Delft research, it is referring to the entire research and not just one calculation sample. “And, as already stated, we did not look at all at the impact of the QR policy on society as a whole. We will look at this in the follow-up research.”