Spain’s much-maligned system for evaluating scientists, in which the sole criterion for career advancement is the publication of papers, is set to be overhauled under new proposals from the country’s National Evaluation and Accreditation Agency (ANECA).
The reforms, announced earlier this month, would for the first time see researchers at Spain’s public universities evaluated on a range of outputs besides papers, and would also encourage the distribution of findings via open-access platforms. Many scientists are welcoming the move, saying it will help academia move on from a system that has been described as establishing a “dictatorship of papers.”
Currently, ANECA assesses the “research performance” of academics every 6 years. To win a modest salary increase and be eligible for promotion, researchers must show that they have published a minimum of five papers during that period in high-impact journals indexed in Journal Citation Reports (JCR), a database produced by the publishing analytics company Clarivate. Meeting the goal also enables scientists to supervise doctoral students and be listed as principal investigators at their universities, which gives them access to a larger budget.
The Spanish government introduced this system in 1994 in an effort to increase the productivity of the country’s researchers. And it succeeded: By 2021, the country ranked 11th globally in scientific output, producing more than 100,000 publications annually, according to the Ministry of Science and Innovation.
But many academics say the system has had a negative impact on the quality of their work and on science in general. It unleashed “an insatiable paper publishing machine,” says Ángel Delgado Vázquez, head of the Learning and Research Support Service at Pablo de Olavide University. Researchers can work frantically to meet the goal, with some publishing an article every 2 days. Others have cut corners, publishing low-quality papers or fraudulently seeking co-authorship; some have admitted to receiving payments from researchers in Saudi Arabia or India to co-author papers they did not contribute to. Critics say the system has also encouraged researchers to prioritize publishing over teaching, and to use public funds to pay increasingly high article-processing charges imposed by some journals.
Under the new system, ANECA wants assessments to consider a broader range of research outputs, including “publications, patents, reports, studies, technical works, artistic works, exhibitions, archaeological excavations, [and the] creation of bibliographic records.” Assessors will no longer consider only the impact factor of the journals in which scientists publish, but also details such as whether the research reaches nonacademic audiences through news reports or government documents. Papers will also score more highly when coproduced with local communities or other nonacademic authors. And in an attempt to reduce the level of public funds being spent on publication costs, assessors will take into account papers published on noncommercial, open-access publishing platforms that don’t charge author fees, such as Open Research Europe.
The changes are an attempt to fix a system that has prioritized “quantity over quality” and to “recogniz[e] that are different ways of doing science,” says Pilar Paneque, ANECA’s director. She also hopes the new system will help reduce fraud in publishing. (Delgado Vázquez, for one, thinks it will “undoubtedly contribute to stamping out unethical practices.”)
The proposals are a “bold gamble,” says Núria Benítez Monforte, a research administrator at the Catalan Institute of Nanoscience and Nanotechnology. But others say even more radical reform is needed. For example, information scientist Eva Méndez, who serves on the Steering Board of the Coalition for Advancing Research Assessment, says the 6-year evaluation cycles should be abolished altogether, because they “only serve to enrich major scientific publishers, adding to the frustration of younger researchers.”
ANECA is currently reviewing some 600 comments it has received on its proposed changes, with a goal of finalizing the new assessment system by the end of the month. If all goes as planned, it would go into effect on 1 January 2024.
