Administrators who think they have to jump through hoops, failing inspectors and a fainthearted minister. Welcome in the circus that is known as the quality agreements.
Among other things, the quality agreements were supposed to lead to more study places. (Photo: Dalia Madi)
By Henk Strikkers
In this part of a series of investigative stories, we examine how the revenues from the loan system are being scrutinised to see if they are being spent properly. This is what you will read:
- Universities and higher education institutions are required to write a plan in which they explain how they will spend the money.
- Gaining approval for these plans became a battlefield: almost 40 per cent were rejected in the first round.
- Administrators whose plans were rejected spoke of ‘jumping through hoops’ and ‘system dominance’.
- Education Minister Ingrid van Engelshoven twice changed a negative recommendation into a positive decision after a discussion with the executive board of a university and a university of applied sciences.
- “It undermines the trust of students”, says Professor Henriëtte Maassen van den Brink.
After the drama surrounding the pre-investments, everyone thought that spending the revenues of the loan system was bound to go well. The report by the Netherlands Court of Audit on pre-investments at universities and higher education institutions is damning. In hindsight, it could only be established that a third of the investments that were made were pre-investments. Members of parliament and student unions are incensed and will do whatever it takes to prevent this from happening again with the revenues from the loan system.
Minister Ingrid van Engelshoven (Education, Culture & Science; D66) concurs. Clear goals must be set, and the plans of universities and higher education institutions need to be tested for feasibility and to ascertain whether the money is being spent on worthwhile projects.
The Accreditation Organisation of the Netherlands and Flanders (NVAO) will be responsible for checking up on this. “The NVAO is highly independent and knows how to oversee quality in higher education,” said the minister in the parliamentary debate following the Netherlands Court of Audit’s report. Nevertheless, the NVAO was not the first choice. The Inspectorate of Education, the Van Vught Commission (which assessed the performance agreements two years earlier) and an entirely new higher education authority had been considered first, but after protests from universities and higher education institutions, were not approached in the end. These institutions have been campaigning for years against what they see as excessively strict regulation, and are extremely concerned that their autonomy will be curtailed.
Illustration: Rueben MillenaarFRAME: Performance Agreements
In 2012, then-Minister Bussemaker negotiated performance agreements with universities and other higher education institutions. This followed on from the policy of the Rutte I cabinet (VVD, CDA & PVV). Within four years, universities and higher education institutions had to meet targets on educational quality and, if they failed to do so, their funding was to be cut by up to 7 per cent. Ultimately, it was mainly the universities of applied sciences that did not meet the performance agreements. A number of them (who did not achieve these by a long shot) were fined, albeit less than the previously agreed 7 per cent.
Form of compromise
The NVAO is normally concerned with assessing the quality of studies but has been passed the buck this time as a form of compromise. Not much later, the so-called Sector Agreement on quality agreements was ceremoniously signed by students, administrators, and Education Minister Ingrid van Engelshoven. “The performance agreements triggered a lot of chagrin and mistrust”, the minister said during the ceremony in April 2018. “It took a while to regain that trust, but it all worked out very well in the end.”
In the Sector Agreement, it states that every university and higher education institution must draw up a quality plan wherein it outlines what it intends to do with the revenues from the loan system. A panel of the NVAO, made up of former education administrators, students and staff from other universities and higher education institutions, examines this plan and decides which people from the relevant university or university of applied sciences they would like to speak to. They then send them an invitation.
Bart Hekkema.“Then you end up spending a day sitting in a small room, and every few hours a few people walk in and talk to you”, says Bart Hekkema. Hekkema is a history Master’s student at the University of Groningen and a member of the NVAO panels that review the quality agreements. These panels assess the quality plans of universities or higher education institutions on the basis of three criteria:
1 Are the plans well-founded and do they contribute to improving education?
2 Were students involved in the development of the plan?
3 Are the plans realistic and is it possible to monitor the effects?
“Personally, I tend to pay particular attention to the question of whether students are actively involved in the plans”, says Hekkema. He sees that the quality agreements have led to universities and higher education institutions doing more to promote student participation. “They often do this in innovative ways, with students who are not members of elected representative bodies.”
Ron Bormans.Rotterdam University of Applied Sciences is one such higher education institution that has adopted an innovative procedure, says Ron Bormans, chair of the Executive Board there. “The quality of the process is as important to us as the outcome for that process. We invested a lot of energy in this. During various stages, we have been in contact with students and faculty staff within the representative body, with our Supervisory Board, and with stakeholders such as the Rotterdam Port Authority and the Erasmus Medical Centre.”
“The quality of the process is as important to us as the outcome for that process. We invested a lot of energy in this. During various stages, we have been in contact with students and faculty staff within the representative body, with our Supervisory Board, and with stakeholders such as the Rotterdam Port Authority and the Erasmus Medical Centre.” Moreover, Bormans proudly mentions initiatives started by students from the representative body themselves, including the campaign ‘What should we do with €26 million?’ This is the amount that Rotterdam University will receive annually from the quality funds in 2024 and the years thereafter.
This yielded a plan in which academic success and intensive, small-scale education are extremely important. “Our strategy is: The teacher is the one doing it, so we want more good teachers. We all share the realisation that academic success can best be achieved twit the help of high-quality teachers.”
(Below the infographic you can read more about how the Rotterdam University of Applied Sciences was not the only university of applied sciences that had a problem with its quality plan).
One of the criteria that the panels assess, and on which the plan submitted by the Rotterdam University of Applied Sciences was initially rejected, is whether the plan makes a substantiated contribution to improving education. According to Hekkema, the panels do not assess whether there will be any quantifiable quality improvements by 2024, but only whether the rationale behind the plan can be understood. “The assessment of the plans therefore has no predictive value for the actual realisation of an improvement in the quality of education.”
“As a panel, it is really quite complicated to fully assess educational improvements. You cannot say that with any certainty about a lot of plans. Having more teachers does not necessarily mean better education”, he says. “Some universities or higher education institutions are doing it right. They say, for instance, that they are hiring extra teachers so that fewer hours of study group sessions are held per teacher and there is more time for personal tuition, for example, when it comes to dissertations. At the end of the day, you don’t know if that actually means better education, although it tends to be a strong indicator.”
Illustration: Rueben Millenaar.FRAME: Almost €600 million per year
In a few years’ time, the quality agreements will amount to almost €600 million. In 2021, all of the universities of applied sciences will receive €239 million in total while universities will receive €146 million. On account of the coronavirus, the Minister has decided that universities of applied sciences that do not yet have an approved plan will also receive their money. The total amount is set to rise to €356 million for universities of applied sciences and €207 million for universities in 2024.
Yet things went sideways anyway. “When the NVAO panel first came to us, I could already see in their eyes that it wasn’t going to work out”, says Bormans. “We were a bit too single-minded. It suited us, but the committee was clear: It had to be exactly within the lines of the national themes.” The panel rejected the plan from the Rotterdam University of Applied Sciences. The NVAO upheld that, and the Minister adopted the NVAO’s advice.
Bormans cites an example of one such criticism: “We had written down that we wanted to hire more teachers. That should have been: ‘Rotterdam University of Applied Sciences is investing in more intensive, small-scale education’, he says as if he is dictating it. “In my view, something like that constitutes system dominance”, says Bormans. “System reality triumphs over actual reality. I find that disappointing.”
That disapproval meant that the institution was subjected to a second round, just like sixteen other universities of applied sciences. “We used the second round to hone the plan and flesh it out for the longer term, but it was mainly about changing the wording”, says Bormans. “I also think that the plan would have been approved the first time around if we had written it differently. To some extent, it does feel like you have to jump through a few hoops, but we still jumped through that hoop in the second round in our own way.”
Henriëtte Maassen van den Brink.Moeilijk criterium
Henriëtte Maassen van den Brink, professor of Business Administration at the University of Amsterdam and up until 2018, professor of Evidence-Based Education at Maastricht University, also finds ‘quality’ a tricky criterion to measure. Like Hekkema, she has on several occasions been a member of the panels that assess the quality agreements.
She sees another problematic aspect, namely monitoring how the millions from the quality agreements are spent in practice. Although universities of applied sciences will have to account for how the money is spent in 2024, Maassen van de Brink says that this is simply ‘impossible to do for plenty of institutions’. “This is especially the case if they adopt a lot of different measures and keep on reserving small amounts for them.”
Juggling with figures
Eindhoven University of Technology is a good example of this. The annual report for 2019 shows that half of the available funds (€3.5 million) were not spent. Of this, €3 million of student loan funds came from the government and half a million came from the university’s reserves. The remainder of the student loan budget (€1.2 million) was added to the earmarked reserve for student loans. This already included €1.4 million that was not spent in 2018. In fact, TU Eindhoven did not spend all the money that was available that year either; at the time, it received €2.9 million from the government and had also added funds of its own.
This makes it difficult to tell where that €2.6 million in the earmarked reserve actually comes from. Is that money from the revenues of the loan system? Is it money from their own reserves? Or is it a bit of both, and if so, in what proportion? Now, this is not necessarily a problem, but if universities have not spent all the money by 2024, then they will have to give it back. Maassen van den Brink foresees a major problem when she looks at the plans. “To what extent will panels in 2024 be able to check how effectively the money has been spent?”
TU Delft
It is precisely external control that is essential for this, according to Maassen van den Brink. “Scrutiny from outside forces universities to properly account for and provide insight into how public funds are spent”, she feels. Maassen van den Brink is well aware of this, as she was also a member of the committee that assessed whether universities of applied sciences were complying with the performance agreements.
Maassen van den Brink also sees the same trend amongst the performance agreements. Where there used to be a heavy fine for not meeting the targets, this was watered down during the process. “As a committee, you get fed up with it in no time. When we subsequently visited universities of applied sciences and universities, we noticed that some had a bit of a laissez-faire ‘let it go, it’ll all work out’ attitude. They knew that they could simply smooth over the commission’s more negative remarks with the minister later on. The same thing is now at risk of happening again with the quality agreements.”
She is referring to Minister Van Engelshoven’s decision to ignore the negative advice from a panel in the cases of the Delft University of Technology and the Hanze University of Applied Sciences, and still award them money from the quality agreements. The minister did this after she had invited the administrators of these universities and universities of applied sciences to the ministry and they had been able to convince her of their plans in person, bypassing the independent NVAO.
According to Maassen van den Brink, this poses the risk that the process will once again become ‘ineffective’. “This undermines both the agreements that we made and the trust of students who had to forfeit their student grants.”
With the cooperation of Altan Erdogan, Yvonne van de Meent and Laura ter Steege.
This story is part of a series made possible by the Dutch Journalism Fund (SVDJ) and various editors at higher education institutions and universities in the Netherlands.
Also read the other stories:
This is how your basic grant died
In search of millions, buried between promises and practices
How the hundreds of millions of euros from the basic student grant evaporated
Pre-investments: a recipe for failure
Do you have a question or comment about this article?
delta@tudelft.nl
Comments are closed.