As a funder of applied health and care research that is focused on meeting the needs of people, communities and the health, public health and care systems, it is essential that the evidence generated, through the research we fund, reaches those who are able to use or benefit from it. As such, we recognise and value a wide range of research outputs, and support a diversity of approaches to dissemination and knowledge mobilisation; extending the reach of the research we fund to bring about benefit and change.
For NIHR, being responsible in the use of metrics means that we assess the inherent merit of research activity and findings of individual researchers and research organisations, rather than using short-hand proxies for research quality or impact, such as Journal Impact Factor (JIF), H-Index, or judging the quality or importance of research by where it is published or institutional rankings.
Our commitments
We are committed to the responsible use of metrics in our decision-making processes when awarding funding, evaluating and communicating our impact, and appointing advisors.
As a funder we will:
- Be explicit about the assessment criteria we use. The scientific merit and potential for impact is more important than: academic publication metrics; where an output is published; or, the institutional origin
- Be transparent about any metrics we use, and always use these metrics in combination with other sources of evidence
- Actively encourage multi-disciplinary teams from diverse career paths to work with us
- Assess knowledge mobilisation and impact plans as part of the application process.
- Recognise the value of all relevant research outputs and encourage open research practices.
In evaluating and communicating our impact we will:
- Celebrate and amplify diverse research outputs, outcomes and impacts
- Use a broad range of evidence, data and indicators to demonstrate the outcomes and impact we seek to achieve
- Be open and transparent about any evaluation or assessment of NIHR impact we undertake, making use of numerical and narrative sources of information.
When appointing our advisors we will:
- Seek to work with those that can demonstrate relevant content knowledge and expertise - drawing on a variety of evidence sources.
- Ensure that where metrics are used we are clear on the context and limitations of these, and complement them with qualitative sources of evidence.
What do we mean by ‘Responsible Use of Metrics’?
Data about research - the things researchers do and the things researchers produce - are often summarised to create numerical indicators or metrics. These metrics may describe individual researchers’ activities, how other academics have used their work, or the activities of the research organisations they are part of or employed by (such as the university department they are based within, or the institution they work for).
These metrics have become a prominent part of the research ecosystem and are often used as short-hand for denoting high-quality, impactful or important research, and to track trends over time. However, there are issues with using metrics in this way. For example, these metrics may be used inappropriately to judge the performance of people and organisations. Often these metrics capture information that is readily available (e.g. where a paper is published) rather than what is important (e.g. did something change as a result of the research?). And, it may be unclear how the metrics have been calculated. Evidence also shows that inappropriate use of these metrics can negatively affect some people and organisations more than others, as detailed in the report ‘The changing role of funders in responsible research assessment’, published by the Research on Research Institute.
As a funder, we are guided by the principles outlined in The Metric Tide and continue to adapt our practice to better identify and assess the value NIHR research creates, within the context of our mission - to improve the health and wealth of the nation.
As part of this, we worked closely with our stakeholders to develop the NIHR Outcomes Framework; a tool to articulate what the NIHR, as a whole system, is achieving through a set of meaningful, transparent and responsible indicators.