Reporting the outputs, outcomes and impacts of NIHR research 2023 update
The NIHR has a responsibility to demonstrate the value and impact of the research activities it funds and supports. To help us do this we use a tool called Researchfish to collect detailed information, directly from holders of NIHR-funded research and career development awards, about their outputs, outcomes and impact. The information gathered each year is used by the NIHR to demonstrate the need for further pioneering research for the benefit of patients, the public and the economy.
We use your data to respond to information requests and parliamentary questions, to report to our stakeholders and to enable a wide range of evaluative activities about the research funded by the NIHR. It informs the impact stories we showcase on our website and highlight through our social media channels.
It is also used as a data source to enable the NIHR outcomes framework, a recently launched tool that allows a systematic and transparent way of describing the benefits we want to achieve for patients, the public and the economy. It supports in-house evaluations conducted by the NIHR coordinating centres, like the research on the experience of NIHR’s Clinical Lectureship and the determining factors for a clinical academic career post lectureship: a mixed-method evaluation.
This data has significantly contributed to collaborative evaluations, such as our partnership with Marie Curie, assessing the impact of combined investments in palliative and end-of-life care research. It also helped us understand how we are doing in our ambitions to support open access publishing and our progress in building capacity through investment in research. Information on the outputs, outcomes and impacts arising from NIHR awards provides vital data for our programme evaluations, including the 10 year evaluation of the EME Programme, and our current analysis of the Health Technology Assessment programme 30 years impact assessment.
We are actively developing new ways to utilise this data, enhancing information on the NIHR Journals Library and our Funding and Awards pages. Explore more about the diverse applications of the data you provide.
We know it can take time for research outputs, outcomes and impact to arise. Therefore, we maintain contact with you beyond the initial lifecycle of your award(s). This enables us to capture as much information as possible on the work you have done and the benefits achieved.
Our use of Researchfish
NIHR has 15,011 personal and research awards in Researchfish, tracking information on the outputs, outcomes and impact of our funded research
In 2023, 91% of awards we expected to provide a submission did so.
NIHR began using Researchfish in 2012. Since then NIHR awards have reported:
10,761 influences on policy and practice
51,827 engagement activities 
76% awards report involving patients and public 
23,356 collaborations and partnerships
1,639 (88%) of NIHR Academy award holders reported that they had achieved their main motivation for undertaking their award
1,650 new datasets
102,652 publications
Influencing policy
NIHR award holders have been providing evidence to Government reviews, inquiries, committees and forums informing policy and decision-makers, the NHS and life science sector of the latest evidence. Some serve as members of committees, others are participating in working groups and providing advice on best evidence to Government departments, service commissioners and health and care leaders, helping to turn better knowledge into better care.
By actively advising and contributing to clinical governance and influencing key health and care services, organisations and companies, award holders funded by NIHR ensure that future policy directions are informed by high quality research evidence.
Another productive way to maximise the usefulness of research is to participate in National consultations. NIHR award holders report preparing responses and contributing to such consultations to highlight challenges, suggest improvements and provide support to service planning and commissioning.
“The publication of these research findings coincided with a review of services by NHS England in response to the pandemic, with similar conversations in Scotland, Northern Ireland and Wales. It also provided evidence to support the inclusion of frailty virtual wards, also known as Hospital at Home, as part of NHS England’s virtual ward programme. This programme aims to see 40-50 virtual ward ‘beds’ per 100,000 population by December 2023.”
Professor Sasha Shepperd, Professor of Health Services Research, University of Oxford
Health Services and Delivery Research (HSDR) Funding
"An approach developed through this research can help with understanding the causes of multimorbidity and has been used in drug development prioritisation decisions across different disease areas. It has led to considerable savings in halting the development of likely unsuccessful drug compounds at an earlier stage."
Professor Martin Tobin, Professor of Genetic Epidemiology and Public Health, University of Leicester
NIHR Senior Investigator Award
Ways in which NIHR award holders report influencing policy and practice
The numbers of reported influence on policy and practice outputs by type of influence:
Citations in policy documents, systematic reviews, clinical guidelines etc, 2967, 28%
Participation in an advisory committee or national consultation, 2,807, 26%
Influenced training of practitioners or researchers, 2800, 26%
Membership of a guidance or guideline committee, 1293,12%
Implementation circular/rapid advice/letter to e.g. Ministry of Health, 528, 5%
Contribution to new or improved professional practice, 365, 3%
Influencing practice
Our Researchfish data tells us where NIHR research findings have been influencing the development of practice, and our award holders report that they are doing this by:
- presenting or submitting evidence to inform local or national guideline development
- informing and influencing review of clinical practice, social care and wider healthcare
- preparing policy documents and responding to proposals from public bodies and organisations which have subsequent impacts on health and care
- updating systematic reviews of the evidence base to inform policy and decision-making.
It is when research findings change the course of care or treatment that research really makes its impact on health and wellbeing. A crucial step in developing health and care practice is to ensure that findings from research inform guidelines and training information.
“ ...Trial results indicated that effects were unanticipated and in the opposite direction to those sought. The work concluded that caution needs to be exercised in using predictive risk tools at an individual patient level to support clinical decision making. This is a useful piece of research for consideration in decision making and planning. As a result, the PRISM tool was not rolled out more widely in the Welsh health system." Alex Slade, Deputy Director -Primary Care, Welsh Government.
"Our research resulted in the cancellation of a planned roll-out of emergency admission risk stratification tools in Wales. PRISMATIC study findings indicated that use of the tools is associated with increase in hospital service use and costs, with no clear evidence of patient benefit. Modelling of PRISMATIC findings shows that non-implementation in Wales has avoided an estimated 30,000 additional admissions to hospital, 162,000 bed days and £200 million per year.” Professor Helen Snooks, Professor of Health Services Research and Health Data Science, University of Swansea
Health Services and Delivery Research (HSDR) Funding
"The FeverPAIN algorithm reduces antibiotic use and improves symptom control, an app allowing easy calculation of the risk of strep infection is hosted on Oxford Clinical Trials Research Unit site and has been used several hundred thousand times, and has now been incorporated into routinely used GP consultation systems.”
Professor Paul Little, Professor in Primary Care Research, University of Southampton
Health Technology Assessment (HTA) Funding
Keeping patients at the heart of research
Since 2016, NIHR award holders have been asked to describe how patients are involved in their research and to explain the difference it makes. We’ve worked closely with other funders to develop a common Patient and Public Involvement (PPI) question set for use in Researchfish. Seventeen funders (including the NIHR) in the United Kingdom, Europe, North America now use this question set which replaced the NIHR only question set in 2018.
Each year we ask our active awards for information about their PPI. We have analysed the most recently submitted data about PPI from 9,316 awards to provide a snapshot of involvement in 2023. 7,138 (76%) awards reported involving people in their research in different ways. 2,178 (23%) awards either did not involve people or involvement was not relevant for their award. Involving patients and the public in research is a requirement for NIHR, but in some cases patient and public involvement may not be applicable, for example if the award is focused on research method development, or in those awards that do not directly fund research but fund people.
Researchfish data demonstrates how involving patients and the public in research can enhance the reach, quality and impact of research, and that involving people provides wider perspectives and opens new directions for future research.
"Our Patient and Public Involvement activities have kept us grounded in the reality of conditions from a patient's point of view, which is vital for the adoption of novel technologies."
Nicholas Stone, Professor of Biomedical Imaging and Biosensing, University of Exeter
NIHR Invention for Innovation (i4i) Funding
“Working collaboratively with my PPI group who have lived experience of stroke has increased the quality of the research. The group have guided the focus of the research and always ensure it is of relevance to stroke survivors. The group made a brilliant recruitment video which was helpful in explaining the complexities of the research and the benefits of carrying out this research for the people affected.”
Dr Claire Mitchell, Senior Lecturer and Senior Research Fellow, University of Manchester
NIHR Research for Patient Benefit (RfPB) Funding
Members of the public are helping to improve the relevance and quality of research questions. From those who reported involving patients on their research, just over 64% (4,555) NIHR award holders told us that they had used input from individuals and groups to make sure they are asking the right research questions. More than 60% (4292) reported that patients and the public had been involved in the design of their research.
“The PHIRST Programme is all about evaluating public health initiatives led by or commissioned by local authorities across the UK. Patient and Public Involvement in our evaluations has been crucial to getting insight from people directly affected by the services we are evaluating.”
Professor Susie Sykes, Professor of Public Health and Health Promotion, London South Bank University
Public Health Research (PHR) Funding
“The PPI panel has been invaluable in the funding application and development of documents for regulatory approval. The panel have been enormously helpful in developing patient-facing information and are currently helping to develop the trial website, which is now live. The PPI panel has also informed staff training and the implementation group.”
Professor Craig Smith, Professor of Stroke Medicine, University of Manchester
NIHR Research for Patient Benefit (RfPB) Funding
Patient and public involvement tails off over the life cycle of the award, this same trend is seen in comparative data from other funders. In the early stages of research activities (such as identifying research questions, study design and development) patient and public involvement is widespread, but involvement in the later stages of research (such as data analysis, writing up and sharing research findings) is much less common practice.
The number of reported involvements of patients and the public by type of involvement at each stage of the lifetime of the NIHR award are:
Research design: 4,893, 69%
Prioritising research question(s): 4,009, 56%
Developing funding application: 3,785, 53%
Undertaking the research: 2,395, 34%
Management of the research: 2,175, 30%
Dissemination of findings: 2,173, 30%
Analysing and interpreting data: 1,499, 21%
Writing up: 1,176, 16%
Implementing findings: 750, 11%
Other/not specified: 463, 6%
NIHR award holders tell us that the main challenges surrounding PPI are finding the ‘right’ people to involve, and maintaining relationships and continuity over time. We are using this intelligence to continue to reflect and develop the ways we can better support NIHR award holders to involve people throughout their research.
What have been the challenges of involving patients and the public in NIHR research?
The challenges of involving patients and the public in NIHR research by number of responses:
Maintaining relationships and continuity over time, 2,187, 14%
Finding the 'right' people to involve, 2,016, 13%
Concerns about how representative views are, 1,729, 11%
Small number of people or appropriate people are difficult to engage, 1,503, 10%
Nature of condition can affect ability to be involved, 1,459, 9%
Difficulty in finding appropriate ways to communicate and engage, 834, 5%
Lack of resources to support effective involvement, 764, 5%
Other challenges, 422, 3%
No challenges, 1,181, 8%
Global collaboration
Working with others is key to interpreting the results of research, sharing research findings and working out how findings can be applied in practice. Analysis suggests that researchers who collaborate internationally are most likely to be impactful.
The data demonstrates that NIHR award holders are highly collaborative. 70% of reported collaborations with partners based in the UK but there is also strong global collaboration across 100 countries, further extending the potential for greater dissemination and impact with NIHR research. From award holders who reported collaboration, 224 are global collaborations, reaching two or more continents.
Locations of collaborations with NIHR awards:
United Kingdom, 14,115
Europe, 2,853
North America, 1,587
Oceania, 583
Asia, 411
Africa, 262
South America, 122
The benefits we have gained from collaborations include:
- Sharing and disseminating research findings
- Reinforcing existing collaborations and developing new partnerships for future work.
- Sharing information used in business cases for commissioning and developing new services.
- Informing education and training programmes and promoting good practice amongst peers.
“Our international collaboration has provided our research team with new skills in a technology that will add significant value to our programme of research, and value to the future business need for our Immunogenetics Facility. Having in-house skills offers significant advantages compared to outsourcing because it allows greater research flexibility, and will ensure we continually develop deeper intelligence with regard to the complexities, advantages and limitations of proteomic technologies.”
Professor Ann Morgan, Professor of Rheumatology, University of Leeds
NIHR Senior Investigator Funding
“The RESPIRE partner collaborations resulted in successful completion of all its projects, delivering important policy and clinically relevant insights, many of which are already having local health and wellbeing and societal impact. The novel approaches for accurate diagnosis, treatment and management developed in RESPIRE, are vital in low-resource settings where access to highly trained healthcare professionals is limited. This work has so far resulted in 60 peer-reviewed publications, most led by Low and Middle Income Countries investigators, with many more papers in preparation or in press. Our cohort of Early Career Researchers are securing follow-on jobs and applying for independent grant funding and benefiting from our ongoing mentorship and support.”
Professor Aziz Sheikh, Professor of Primary Care Research and Development, University of Edinburgh
NIHR Global Health Research Unit on Respiratory Health
Investing in people
In 2016, NIHR Academy introduced the Researchfish Career Tracker and Your Award question sets to capture the career progression of NIHR Academy Members. Award holders are asked to provide a response to the Your Award question set once upon completion of their award, and to provide a response to the Career Tracker question set for five years after completion of their award. These questions explore issues such as the experience of undertaking an NIHR award, how much time award holders spend on research and future career plans. Analysis of the information allows us to develop its training and support programmes and track outcomes.
For the 2023 submission period, 1856 award holders provided information in response to the Your Award questions. Our analysis shows that 90.8% of respondents who undertook an NIHR Academy award did so because they wished to pursue or further their academic or clinical academic career. In addition, 88.3% reported that they had achieved their objective by undertaking their NIHR Academy award.
2163 award holders provided information in response to our Career Tracker questions; this data shows that 75.5% of those who completed an NIHR Academy award remain in an academic or clinical academic role. Of those who returned to a clinical or other type of role, 65.6% continue to be active in research.
Trends in post-award roles of NIHR Academy Members
The change in percentages of post award roles of NIHR Academy members between 2021 and 2023. Y-axis range 0% -50%.
Clinical Academic: 43% to 44%
Academic: 33% to 31%
Clinical: 20% to 20%
Other: 5% to 5%
"The research professorship has provided a career-defining impact for me. It enabled me to establish a team able to conduct a programme of world-leading research. It provided both the time and resources that allowed flexibility to explore different research possibilities and identify truly game-changing research opportunities. As well as the time and resources, the position of "NIHR Research Professor" also opened up new opportunities and collaborations, both locally and nationally."
Professor Anthony Gordon, Imperial College London
NIHR Research Professorship
"I am enormously grateful to NIHR for this award which has been life changing. There are so many benefits it is hard to pick just one. It has enabled me to come into research at a relatively late point in my career (I had been working as a qualified clinician for 11 years prior to starting the award) and develop the knowledge and skills required to conduct clinically meaningful research in a remarkably neglected area - harmful grandiose delusions. I have learnt a huge amount about conducting research and am truly excited about the potential to continue with a clinical academic role in the future."
Dr Louise Isham, University of Oxford, HEE/NIHR Clinical Doctoral Research Fellowship
"I have developed so much as a person through undertaking the award. Reflecting on my skills and knowledge before the award and now that I have almost completed it, I see that they are worlds apart. Undertaking the award has enabled me to actively navigate my career pathway to where I want to be and to take on new roles where they did not exist before with confidence."
Emily Taylor, University of Exeter Medical School
NIHR Infrastructure
"I very much enjoyed the research time I was able to have as a result of my NIHR award. Despite the Covid pandemic, I was fortunate that my academic time remained fully protected during the entirety of my research project. The award allowed me to work with a fantastic research group at the National Heart and Lung Institute who were incredibly supportive and from whom I learnt a great deal. I was able to learn new research methodologies which I hope to continue using in future research and was able to see the whole trajectory of the research project from inception of an idea, ethical approval, data analysis and interpretation, manuscript drafting and submission and publication, allowing me to develop further as a researcher."
Dr Ardita Koteci, Imperial College London Faculty of Medicine
NIHR Academic Clinical Fellowship
Generating new knowledge
Datasets and collections of data
Making research data available so that it can be effectively re-used by the scientific community benefits numerous research-related activities, such as reproducing analyses, testing secondary hypotheses, developing and evaluating novel research methods, design of future trials, meta-analyses, teaching, and by helping to prevent error, fraud and selective reporting.
In our NIHR position on the sharing of research data we set out that the NIHR strongly supports the sharing of data in the most appropriate way, and are committed to support open access to research data to help deliver research that maximises benefits to patients and the wider public, the health and care system, and which contributes to economic growth in the UK.
Information about data sets generated by NIHR awards and reported to us through Researchfish helps us to understand how they are being used to underpin research collaborations, facilitate further research activities, and to inform policy and guidelines. In 2023, NIHR award holders reported a cumulative total of 1,650 databases or models, of which 46% have been provided to others.
“The PRIMEtime model has been used for numerous policy evaluations in the UK, including evaluations of the Soft Drink Industry Levy, the proposed regulation of advertising of unhealthy food to children, and reformulation of sugar in processed foods.”
Peter Scarborough, Professor of Population Health, University of Oxford
Public Health Research (PHR) Funding
Publications
NIHR award holders share research findings targeted at other academics through journal articles, conference abstracts and proceedings, and writing book chapters. There is a steady growth in the number of publications reported to us, plateauing from 2020 into 2021. During the last ten years a total of 102,652 outputs published from NIHR awards reported in Researchfish.
Number of publications reported by year of publication
Increasing numbers of reported publications by year of publication, from 2013 to 2023, with partial figures for 2023:
Year
2013: 5,951 publications
2014: 6,949
2015: 7,584
2016: 8,582
2017: 8,755
2018: 9,226
2019: 9,776
2020: 10,053
2021: 10,007
2022: 9,899
2023: 1,820
Increasing percentage of reported open access publications by year of publication, from 2013 to 2021, with partial figures for 2023:
2013: 24%
2014: 28%
2015: 31%
2016: 35%
2017: 36%
2018: 38%
2019: 38%
2020: 43%
2021: 45%
2022: 34%
2023: 36%
In November 2021, following an extensive review, NIHR announced its new Open Access policy, which will increase the number of NIHR funded articles published immediately open access. The policy requires all peer-reviewed research articles submitted from 1 June 2022 arising from NIHR-funded studies to be made immediately open access under an open licence. The policy will ensure that NIHR funded research findings are freely accessible, discoverable and reusable to all, maximising the societal, academic, and economic impact of NIHR funded research. It will empower patients and the public, and enhance the integrity and rigour of research through greater openness and transparency. The review of the policy included extensive engagement with stakeholders, including patients and the public, and a range of evidence published on the NIHR website.
The new policy builds on the progress made by NIHR under the previous open access policy, under which analysis of the available open access information provided through Researchfish shows a steady growth in the percentage of all publications that are open access over the past years.
The NIHR Open Access policy is focused on our expectations in relation to publishing academic, peer-reviewed journal articles – but this is just one way in which our researchers may choose to disseminate their research findings. We recognise and encourage a diversity of approaches to disseminating research findings so that people can make use of them. Beyond academia, NIHR award holders publish a wide range of other materials including consultancy reports, policy briefings, technical reports and standards and working papers. Publications of different kinds capture the knowledge from NIHR-funded research that can be shared in a variety of formats to meet stakeholder needs.
Reducing unnecessary bureaucratic burden in research
NIHR has a number of workstreams underway to explore ways in which we can minimise unnecessary research bureaucracy, with the primary aim of freeing up the research community to concentrate on delivering the high-quality research that patients, the wider public and our economy needs.
NIHR works closely with other funders (including UKRI and AMRC charities), HEIs and research organisations to coordinate the activity surrounding the collection of outputs, outcomes and impacts with Researchfish, adopting a shared approach where feasible with a view to reducing the administrative burden of reporting as much as possible. This year, we are working together on a project to review the Researchfish’s question set. Our goal is to ensure our data collection stays up to date with a changing research landscape. The first outcome from this project has been removing the “Other Outputs” section of the reporting. This is a step towards reducing researchers' reporting burden and ensuring we are only asking our award holders for data that is used to inform NIHR’s evaluation needs.
Ongoing developments continue to improve the interoperability between Researchfish and many different sources of information, enabling outputs data to be harvested from other sources such as ORCID, Europe PMC, Crossref, DataCite and GitHub, and greatly reducing the manual input of information required by award holders. Use of the unique NIHR award reference in outputs greatly contributes to our award holders’ ability to take full advantage of the benefits of this harvesting capability, as it aids the automated identification of NIHR outputs most effectively. The majority of publications that are reported by NIHR awards holders in Researchfish are sourced from other systems by the automatic recognition of NIHR references in funding acknowledgments. As the Researchfish system expands its compatibility with a wider range of data sources, outputs—such as datasets, data models, software, web tools, and applications—reported in Researchfish are increasingly originating from diverse data sources beyond the platform itself.
The NIHR remains committed to ensuring that the submission process is as straightforward as possible. Visit the NIHR website and NIHR Researchfish guidance to access our regularly updated guidance and downloadable resources. Contact the NIHR Researchfish team evaluation@nihr.ac.uk for more support and information.
The next NIHR Researchfish submission window opens on 5 February 2024 and closes 4pm 14 March 2024.
This Digital Object Identifier (DOI) https://doi.org/10.3310/rfupdate-2023 provides a permanent link to this report and its metadata.