AWTI: despite teething problems, ‘recognise and reward’ is the wave of the future

So says chairman Eppo Bruins of advisory board AWTI in an interview with the Higher Education Press Agency (Hoger Onderwijs Persbureau).

The Dutch government spends billions of euros on scientific research. But how do we actually know if this money is ending up in the right places, the House of Representatives wondered. How do you evaluate research quality?

In June, the House requested a letter from the Advisory Council for Science, Technology and Innovation (AWTI) on this subject. This letter was presented Monday the 19th of december to Speaker of the House Vera Bergkamp.

One of the reasons for the request was the recently introduced course change in Dutch academia, under the motto ‘recognise and reward’. The idea behind this is that researchers should focus on other things besides their research output, and that they should be rewarded for their other responsibilities. These include teaching, leadership, contributing to team science and creating societal impact.

Critics fear that this new course will damage the Netherlands’ strong international position in academia. But according to Eppo Bruins, it is too early to tell what the result will be. “And more importantly, there’s no stopping it: ‘recognise and reward’ is a global development.”

The House of Representatives wants to know how scientific quality should be assessed. Were you able to answer that question?
“We’ve helped people clarify their thoughts on this subject. The underlying question is: what is the purpose of science? We hardly think about that in the Netherlands, but how people answer that question depends on all kinds of values they might hold. Some point to the inherent value of science, others mention economic benefits, and still others point to the major societal issues that science can help solve. These are all different values, so there are also different kinds of scientific quality, which require their own approach.”

That would be in line with the ‘recognise and reward’ trend.
“If you’re going to assess the quality of science, you need a broad perspective. And that’s what ‘recognise and reward’ offers.”

So why these concerns about the standing of Dutch science in the rest of the world?
“Some feel that things are moving quite quickly, and no one really knows what the new approach will look like yet. The higher education institutions are all handling the shift to ‘recognise and reward’ in different ways, which also comes with risks. If you don’t align the way researchers are assessed, it could have a negative impact on their professional mobility. A good rating from one university wouldn’t automatically open doors somewhere else.”

What new forms of assessment are receiving criticism?
“There have been experiments with ‘narrative’ CVs, which basically had researchers talk about their own work. But they moved on from those fairly quickly. The Dutch Research Council, which funds a lot of research, now asks for evidence-based CVs. That means you get to back up your story with data on the impact of your work in the scientific community. So the requirements keep changing. That’s why we believe that the institutions should develop their assessment procedures together instead of trying to invent the wheel separately.”

But aren’t those differences interesting?
“In a sense, sure. We’ve previously argued for sharper distinctions between knowledge institutions, and if a university has a clear identity it makes sense for that to be reflected in its assessment of researchers. But then you have to be clear about that and make choices that are well thought out.”

Will the Netherlands drop in global rankings if scientists are assessed less on their scientific ‘scores’, such as how often they’re cited by peers?
“We don’t know that yet, but we have compared this shift to the move to open access in science, where scientific articles can be accessed for free by anyone. The Netherlands is also leading the way in that area. The most prestigious journals were hesitant to join the movement at first, so critics feared that open access would come at the expense of the Netherlands’ position in international rankings. But that hasn’t been the case. Besides, you could ask yourself what those rankings are worth exactly, given the enormous diversity in science.”

But publication and citation figures do show how other scientists see your work.
“The core of science is peer review. In our letter, we outline a few drawbacks to this. Reviewers, for example, find it difficult to assess high-risk and interdisciplinary research. They can also disagree with each other, which shows how subjective their judgement is. But peer review still remains the gold standard – we don’t question that at all. All we’re saying is that publishing in journals isn’t the only thing that matters.”

So you’re saying ‘recognise and reward’ can’t be stopped. Is that a good enough reason to go along with it?
“This isn’t just happening in the Netherlands. It’s a global movement. That doesn’t mean every country has to clear its own path, but it’s good to be one of the frontrunners. Because that presents certain opportunities. For example, the Netherlands could become more attractive to international researchers if they know that we take a nuanced view of quality here. It also puts you in a position where you’re able to help determine the course.”

HOP, Bas Belleman

Translation: Taalcentrum-VU