Do we need transparency for trust in peer review for research funding?
Katie Meadmore, Senior Research Fellow in NIHR’s Research on Research (RoR) team reflects on the relationship between trust and transparency in peer review for research funding. Her blog marks Peer Review Week 2020.
“Trust in me, Just in Me, Shut your eyes, And trust in me,
You can sleep safe and sound, Knowing I am around”
The hypnotic words sung by Kaa in the Jungle Book to a naive Mowgli who doesn't fully understand the ways of the jungle or Kaa’s true motives. Now apply the words to peer review for research funding: peer review processes are not always fully transparent but require trust in fair and unbiased decisions.
What is peer review and transparency?
Peer review for research funding is the process of experts and members of the public scrutinising a funding application - or parts of it - to provide recommendations on whether the research should be funded. Peer review occurs throughout a research project’s lifecycle, from the initial conception of the research, through the funding decisions to when the research project is complete, published and disseminated.
Transparency in peer review for research funding refers to the level of openness that the peer review processes provide for recommending research to be funded, who makes the decisions, how and when the decisions are made and the extent to which this is made clear to all involved.
Transparency throughout a research project’s lifecycle is important for funders, including the NIHR. This is further supported with the recent launch of the NHS Health Research Authority (HRA) Transparency strategy and the Department for Health and Social Care policy paper on reducing bureaucratic burden, and funders continually strive to improve systems and processes
Is peer review transparent and trusted?
The principle of peer review stems from the Haldane Report of 1918, which suggested science was best evaluated by those in the knowledge rather than politicians. On this basis, peer review has continued to be used, becoming a well-established practice for many funders of research. However, in a recent survey by Publons, researchers indicated that the transparency of peer review in funding could be better.
Peer review is often criticised for being largely behind closed doors. Peer reviewers and their expertise are often not disclosed, the inner discussions and decisions of committees are seen as elusive, and comprehensive feedback is not always provided. There has been extensive work reporting the biases inherent in peer review processes, including gender, cronyism, and conflicts of interest. Indeed, some researchers may relate to peer reviews being likened to Kaa whereby it says one thing but means another. Nevertheless, peer review is seen by researchers as the best method by which to select research to fund. Thus, researchers still trust that peer review will inevitably select the best research.
How can transparency and trust be strengthened?
Many funders are already committed to strengthening transparency throughout a research projects lifecycle. Peer review processes are not a one size fits all and funders are often engaged in activities to assess and evaluate their peer review processes. Whilst this is good practice, continuous tweaks, as well as nuances between and within organisations, can cause confusion to researchers. Better communication and understanding of reasoning behind decision-making may enhance the transparency, and hence the trust, in peer review processes for selecting research to fund.
Suggestions could include:
- Clear and up-to-date flowcharts with guidance documentation to clarify the processes.
- Researchers attendance at committee meetings to answer queries or transcripts of the meetings to dispel the mystery around committee meeting discussion.
- Interactive methods to discuss feedback to support researchers understanding.
- Publishing peer reviewer identities or peer review reports alongside decisions to ensure greater transparency in funding processes.
- Training reviewers more thoroughly on what is expected from them, providing them with feedback on their review and the outcome of the application to aid transparency for reviewers.
Going back to the song lyrics, peer review for research funding relies on trust, and whilst many do trust in peer review, an increase in transparency around the underlying processes might make it easier for all to sleep safe and sound.
Katie Meadmore, Senior Research Fellow, NIHR Research on Research (RoR)
More information on how NIHR is supporting the HRA Transparency strategy is available on the NIHR website.
For early career researchers who are NIHR Academy or Associate members, you may be interested in joining the NIHR Reviewer Development Scheme.
Information on Peer Review Week is available online.
The views and opinions expressed in this blog are those of the authors and do not necessarily reflect those of the NIHR or the Department of Health and Social Care.