Beyond Closed Research Metrics: Rethinking Research Discovery and Rankings (part 2 of 2)
Traditionally, academic publishing has been closed, which means that readers can only access research outputs if you or the institution you are affiliated with pay for them through subscriptions. However, research metrics – approximate measures of how many papers have been published and how often they have been cited – are created by the same companies that publish and index the research. These commercial interests mean that research metrics such as Clarivate’s “Journal Impact Factor” and the Scopus “CiteScore” are biased in such a way that they reward publishing and citing sources within these same companies. This means that university ranking systems based on these metrics are also biased; therefore, proprietary research metrics and commercial university rankings are all part of the same closed research infrastructure. Recently, many institutions have started to withdraw from this closed, commercially-driven research infrastructure, as detailed in part 1 of this post, which also discusses the University of Galway’s position.
This closed research environment has been enormously profitable for companies such as Elsevier. Since this profit largely comes from publicly funded money via two routes—through university salaries to fund the writing of the research, and also through university library budgets to fund the subscriptions to read the research—it is also worth considering where this profit is going.
One would expect the profits to lead to increased integrity and publishing efficiency; however, “peer review rings, paper mills, and AI-generated fraud [are] triggering mass retractions and raising concerns about editorial oversight” (Beigel et al., 2025). Instead of improving its academic products, Elsevier/RELX has invested millions of euro as venture capital in Palantir (Heles, 2016; Menn, 2019), which has signed multiple contracts to supply data and software to the aggressive US Immigration and Customs Enforcement agency, ICE (Bhuiyan, 2025). Furthermore, the LexisNexis/RELX product “Accurint” aggregates public and non-public information (e.g., phone records, addresses, government ID numbers) with real-time location data to enable ICE agents to detain and deport people without due process (Needle and Rubel 2025).
Given the problems associated with the profits that these companies make, it is worth thinking about open research alternatives. In particular, this post looks at:
- the publish-review-curate model, APC-free open access publishing as an alternative to commercial author-pays, subscription-based publication outlets
- the fully transparent bibliographic database OpenAlex as an alternative to proprietary commercial databases
- the open edition of the CWTS Leiden Ranking as an alternative to commercial rankings.
Each of these options is an incremental step on the journey toward a more open ecosystem.
Publish-review-curate, APC-free Open Access
In Ireland, HRB Open Research is a funder-sponsored publish-review-curate platform: “Articles are published rapidly as soon as they are accepted, after passing a series of prepublication checks to assess originality, readability, author eligibility, and compliance with HRB Open Research’s policies and ethical guidelines. Peer review by invited experts takes place openly after publication. An article remains published regardless of the reviewers’ reports” (HRB). The parallel platform for European Commission-funded researchers is Open Research Europe.
Diamond open access is a publishing model where the publisher provides immediate, licensed open access to the final published version on the publisher’s website, and there is no fee to the author for publishing. This means it is free to write, read, and reuse the research output, with standard attribution requirements. Diamond open access publications and platforms are often funded by organisations, institutions, or governmental initiatives. The Publish OA NORF project has compiled a directory of Irish Diamond Open Access Journals (2025). These non-profit platforms and publications are less likely to be indexed in the commercial databases that drive rankings (Simard et al., 2025); however, there is a new, open knowledge graph: OpenAlex.
OpenAlex
Since its launch in 2022, OpenAlex has challenged the primacy of commercial databases through being more inclusive and transparent. The sheer volume of records indexed by OpenAlex leads to better representation, particularly for non-English language records and institutions. This increased representation means that it is substantially larger than commercial alternatives. The Scopus and Web of Science corpora are relatively similar in size (65,642,377 records and 71,280,830 records respectively), with an overlap of approximately 90%; meanwhile, OpenAlex has 243,053,925 records, with only 21% overlap with the other two databases (Torres-Salinas and Arroyo-Machado, 2026).
The CWTS Leiden Open Ranking
The CWTS Leiden rankings use OpenAlex to counter some of those biases. Two things are notably absent from the CWTS rankings: opaque “reputation survey” data; and a single, overall score for any institution.
Both THE and QS solicit researchers to fill out “reputation surveys” for their rankings. The THE survey, which “targets only experienced, published scholars, who offer their views on excellence in research and teaching within their disciplines and at institutions with which they are familiar” is available in only 13 languages (THE, 2025). This approach favours Western institutions and “accentuate[s] global, regional, and national inequities” (UN University International Expert Group, 2023). The QS survey invites universities to submit up to 400 contacts, which QS then retains for two years, further consolidating the echo chamber (QS, 2026). The CWTS does not use a survey at all, which makes its process more transparent, more objective, and better aligned with the principles of open research.
Rather than reducing the complexities of universities to a single score, CWTS has 14 indicators that can each be filtered by time period, field, region, and publication output. This means that people have to choose which indicator matters to them, and they can then put together their own basket of statistics to evaluate their choices in a deliberate, thoughtful way. In Ireland, the Aontas na Mac Léinn in Éirinn (AMLÉ, formerly USI–Union of Students in Ireland), which represents the student stakeholders using league tables to choose where to study, has called on universities to “re-evaluate their participation in the university rankings system” (2024 AA CN 2 Motion to Cut Ties with University Rankings).
Open Research and You
While it is not possible for individuals to overturn the traditional closed model of publishing, there are certainly actions you can take to support open research.
- Consider whether diamond open-access journals are a good fit for your next publication.
- Be critical when colleagues discuss the importance of for-profit research metrics such as the “Impact Factor”. Even apart from the fact that these metrics are biased in ways that favour the publishers who have created them, there is no causal relationship between the metrics of a journal and the quality of any individual paper contained in the journal.
- Check out our suggestions for journal editors and book authors for more ways to contribute to sustainable research publishing.
If you have any questions or would like to chat further about any of these topics, please stop by my drop-in clinics (Open Practice, Wednesdays, 3-4pm in the Library Foyer), or send me an email.
Please join us for our next Open Research Forum on Wednesday, 20 May: University of Galway's Research Publishing Policy: Empowering our Research Community.
Comments