The NIHR has a responsibility to demonstrate the value and impact of the research it funds. We capture information from researchers and trainees on the outputs, outcomes and impact of our awards through Researchfish. The information gathered is then reported to both the Government and the public to demonstrate the need for further pioneering research for the benefit of patients, the public and the economy.
Researchfish is a vital tool in our toolbox, collecting detailed information directly from NIHR-funded researchers. It allows us to maintain a longer-term relationship with award holders and capture new developments and impacts arising from their research during and after award completion. NIHR personal and research award holders are required to report the outputs, outcomes and impacts from their research via Researchfish in the spring of each year.
We are using our Researchfish data in a variety of ways. We build impact case studies and write reports to share with our stakeholders. The data extracted from Researchfish is currently being used in response to information requests and parliamentary questions, to develop impact case studies, and in impact assessment exercises such as the Efficacy and Mechanism Evaluation (EME) Programme, an MRC and NIHR partnership. We are undertaking collaborative work with Marie Curie using Researchfish data to assess the impact of our combined investment in palliative and end of life care research. Finally, we use Researchfish returns to populate the threaded publication information on the NIHR Journals Library. You can find out more about how we use Researchfish information.
Why we assess impact
Formally assessing the impact of research provides us with data to support several important tasks:
- Improving accountability by improving reporting to government, the public and other stakeholders
- Enhancing the evidence base for the impact of applied health research, to better advocate for funding
- Connecting impacts and new developments with the original research across time
- Helping the Government and NIHR understand how best to allocate resources
- Highlighting potential avenues for continuous improvement and learning
For a full review of the NIHR’s activities, see the NIHR Annual Report.
Our use of Researchfish
- NIHR has over 9,100 personal and research awards in Researchfish, tracking information on the outputs, outcomes and impact of our funded research since 2012
- Data for this update was extracted for analysis September 2019
- In 2019, 96% of awards expected to provide a submission did so
Influencing policy
Since 2012, NIHR award holders reported giving evidence to a government review 239 times Some serve as members of committees, or by participating in working groups and providing advice on best evidence to Government departments, service commissioners and healthcare leaders, and this too helps turn better knowledge into better care.
NIHR award holders also report participating in advisory committees and national consultations. By actively advising and contributing to clinical governance and influencing key health and care services, non-NHS organisations and companies, our award holders ensure that future policy directions are informed by high quality research evidence.
Another productive way to maximise the usefulness of research is to participate in National consultations. NIHR award holders report preparing responses and contributing to such consultations to highlight challenges, suggest improvements and provide support to service planning and commissioning.
Our award holders have also been providing evidence to Government reviews, inquiries, committees and forums informing officials, NHS and life science industries of the latest evidence.
Reported influence of NIHR research
NIHR researchers reported a range of influence channels:
29% influences on training of practitioners or researchers
24% participations in advisory committees or national consultations
22% were citations in guidelines, clinical reviews, systematic reviews and other policy documents
14% were memberships of guidance or guideline committees
4% were evidence given to a government review
4% were implementation circulars, rapid advice or letters
“I chaired the UK Update Guideline on Chronic Kidney Disease associated Mineral Bone Disease. This has changed practice in the UK and altered the way national registry data are collected. This has also been published on the international guidelines website (Kidney Disease Improving Global Outcomes -KDIGO) to help best practice in other countries”.
--James Burton, Professor of Renal Medicine and Honorary Consultant Nephrologist, Department of Cardiovascular Sciences, University of Leicester
“The goal of my NIHR research is to improve the quality and humanity of care that people living with dementia receive during a hospital admission. This is already having impacts in the NHS and on Welsh Government strategy. My work has been identified as 'notable practice' within their strategy 'Together for a Dementia Friendly Wales' 2017-22. In June 2019 I provided expert oral evidence to the Welsh Government Cross Party enquiry on dementia care in hospitals. The Department of Health and Welsh Government have provided statements in response to my NIHR research, which was raised and discussed in questions to the Health and Social Services Secretary Wales, and in the Senedd during 2018”.
--Katie Featherstone, Reader in Sociology and Medicine, Cardiff University
Influencing Practice
It is when research findings change the course of care or treatment that research really makes its impact on health and well-being. A crucial step in developing health and care practice is to ensure that findings from research inform guidelines and training information.
Through Researchfish, NIHR award holders report influencing the development of practice through:
- Local or national clinical guidelines – by presenting or submitting evidence to inform guideline development.
- Clinical reviews - to help inform and influence clinical practice.
- Policy documents - in proposals from public bodies and organisations which have subsequent impacts on health and care.
- Systematic reviews - updates to reviews of the current evidence base to inform policy and decision-making.
Influencing future practice through education and training is also important. The influence up to date research has on education programmes run by Higher Education Institutions and in training programmes for staff is an important area of NIHR impact. We are delighted that we now have nearly 1,700 reported instances of our award holders feeding their findings directly into the education, training and professional development of others.
“Our findings revealed that patients from marginalized communities reported poor management of their medical conditions and significant problems with adherence to prescribed medicines. Their experience of pharmacy services was found to be variable with many experiencing discrimination or disadvantage as a result of their status. The findings contributed to a final co-produced digital educational intervention for community pharmacists and pharmacy support-staff (freely accessible online). This included tips for enabling patients to empower themselves by taking up the offer of a Medicines Use Review."
--Dr Asam Latif, Senior Research Fellow, University of Nottingham, School of Health Sciences
“Our research informed a course run by the Point of Care Foundation on 'Using patient experience data for quality improvement (USPEx)'. This was delivered in Glasgow in February 2019, to 40 NHS Scotland staff. Participants were exposed to new ideas about what kinds of patient experience data they can use for NHS quality improvement and how. They will take this learning back to their organisations and share with colleagues.”
--Louise Locock, Professor of Health Services Research and Fellow of Academy of Social Sciences, University of Aberdeen
Keeping patients at the heart of research
Since 2016 NIHR award holders have been asked to describe how patients are involved in their research and to explain the difference it makes.
Researchfish data demonstrates how involving patients and the public in research enhances the reach, quality and impact of research. Involving people provides wider perspectives and opens new directions for future research.
We have been working closely with other funders to develop a common PPI question set for use in Researchfish. Twenty-two funders (including the NIHR) in the United Kingdom, Europe, North America and New Zealand now use this question set which was first introduced in 2018.
“Our Researchfish data provides valuable insight into the levels of involvement and the methodologies applied, in a variety of contexts. Our data also illustrates the direction of travel around the culture of public involvement in research. There is a wealth of data within Researchfish that can be used as a valuable resource to share best practice, encourage innovation and to develop tailored approaches to involving stakeholders in our research system”.
--Paula Wray, Senior Public Involvement Manager, INVOLVE
In 2019, 78% of NIHR award holders whose project is either active or within the 5 year follow up period said they are involving patients and members of the public in their research.
Patient and public involvement (2010-2019)
Kinds of patient and public involvement
- Prioritising the research question: 1974
- Developing the application: 1873
- Design of the research: 2512
- Management of the research: 1256
- Undertaking the research: 1271
- Analysing and interpreting data: 842
- Writing up the research: 695
- Dissemination of findings: 1286
- Implementing findings: 444
- Other: 159
“It has helped me define my research questions from a patient perspective. The PPI groups have identified practical issues with the protocol and research documentation which was important at this early stage of the study. I also had PPI feedback with my scoping review of the literature. This helped me define my search terms and search strategy.”
--Dr Jemima Dooley, NIHR School for Primary Care Research Launching Fellow, Centre for Academic Primary Care, University of Bristol
30% of NIHR award holders reported involvement in the management and oversight of study delivery and in ensuring that the research methods are easy to understand and acceptable to study participants.
“Our study PPI members are previous ICU patients and their carers. We updated PPI members on study progress and give opportunities for them to comment on the study progress and the way the study is conducted as well as provide their suggestions on how to improve the study further. Some of our PPI representatives are also members of the trial steering committee and are involved in the delivery and management of the study. The involvement of patients and members of the public in our study has provided invaluable perspectives, which has allowed us to conduct the study in a patient friendly and extreme sensitivity manner”.
--Dr Ben Creagh-Brown, Consultant Intensivist, Royal Surrey County Hospital NHS Foundation Trust
NIHR award holders also collaborate with patients and the public to support data analysis and interpretation, and to write up and disseminate research findings.
“We have included patients as authors on three Cochrane reviews so far - something we know is still rare in current Cochrane practice. They have added understanding of impact, supported plain language summaries and helped dissemination. We have met with patient support groups in open sessions and discussed our findings, with great interest. We are supporting charitable organisations in their dissemination activities and have been invited to several of their events over the next 12 months.”
--Professor Morris Gordon, Professor of evidence synthesis and systematic review, Consultant Paediatrician, University of Central Lancashire, Preston
Cross-funder analysis of this data set reflects what we have seen in the NIHR - that involvement in the early stages of research activities - such as identifying research questions, study design and development - is widespread. But involvement in the later stages of research - such as data analysis, writing up and sharing research findings - is much less common practice.
Global collaboration
Working with others is key to interpreting the results of research, sharing research findings and working out how findings can be applied in practice. Analysis suggests that researchers who collaborate internationally are most likely to be impactful (Guthrie, Susan, Anne Kirtley, Bryn Garrod, Alexandra Pollitt, Jonathan Grant and Steven Wooding. A ‘DECISIVE’ approach to research funding: Lessons from three Retrosight studies. Santa Monica, CA: RAND Corporation, 2016. DOI: 10.7249/RR1132)
Our Researchfish data shows that NIHR award holders are highly collaborative. Over 63% of reported collaborations are with partners based in the UK, but there is also strong global collaboration further extending the potential for greater dissemination and impact with NIHR research.
The location of NIHR collaborators by continent:
Asia: 238
UK: 8603
Rest of Europe: 1737
Africa: 103
North America: 983
South America: 43
Oceania: 326
The benefits we have gained from collaborations include:
- Sharing and disseminating research findings
- Reinforcing existing collaborations and developing new partnerships for future work.
- Sharing information used in business cases for commissioning and developing new services.
- Informing education and training programmes and promoting good practice amongst peers.
“In partnership with the Association of Surgeons in Training (ASiT), I designed and delivered a multicentre prospective study on sessional activity in surgical training across their network, with over 70% of representation from current UK Core Surgical Trainee level contributors. The Association engaged with collaborators across the UK and Ireland, and provided feedback to both the UK and Irish surgical training advisory and planning boards to implement change in surgical practice. Our work influenced the design of the Royal College of Surgeons England Improving Surgical Training project.”
--James Glasbey, NIHR Doctoral Research Fellow in General Surgery, University of Birmingham
“My contributions include providing expertise in maternal and early childhood nutrition to an interdisciplinary nutrition, pediatrics and e-Learning partnership in South-East Asia. This partnership has resulted in a new e-Learning platform for health care professionals in South-East Asia www.enea-sea.eu”
--Professor Keith Godfrey, NIHR Senior Investigator, Nutrition Theme Lead, NIHR Southampton Biomedical Research Centre and Professor of Epidemiology and Human Development, University of Southampton
Data sharing
Making research data available so that it can be effectively re-used by the scientific community benefits numerous research-related activities, such as reproducing analyses, testing secondary hypotheses, developing and evaluating novel research methods, design of future trials, meta-analyses, teaching, and by helping to prevent error, fraud and selective reporting.
The NIHR strongly supports the sharing of data in the most appropriate way, to help deliver research that maximises benefits to patients and the wider public, the health and care system, and which contributes to economic growth in the UK.
Our Researchfish data illustrates the reach and pull through of these data resources, and how they are being used to underpin research collaborations, facilitate further research activities, and to inform policy and guidelines.
“The database provides a longitudinal data set for people with dementia living in the community and their family caregivers, and was used in a project funded by Alzheimer Netherlands, who supported a research fellow to work in Bangor for 3 months. The researcher integrated the data with other comparable European datasets to carry out analyses on resilience in caregivers."
--Robert Woods, Emeritus Professor of Clinical Psychology of Older People, Dementia Services Development Centre Wales, Bangor University
“Data from the UK specialist Rehabilitation Outcomes Collaborative (UKROC) database for collating case episodes for inpatient rehabilitation was required by NHS England for analysis and planning purposes and data on rehabilitation costs have been shared with NHS Improvement for updating the specialist rehabilitation tariffs for 2019-20. Data are flowing to NHS Digital for the purpose of commissioning and have been linked with the Trauma Audit and Research Network for the National Clinical Audit for Specialist Rehabilitation following Major Trauma."
--Professor Lynne Turner-Stokes, Northwick Park Professor of Rehabilitation Medicine, King’s College London
Publications
In the last ten years, a total of 54,193 publications have been published from NIHR awards reported in Researchfish, of which 91% are journal articles. There is a steady growth in the number of publications reported to us - reflecting the time lag between study completion and results published. NIHR award holders share research findings with other academics through journal articles, conference abstracts and proceedings and writing book chapters.
Publication count by year
The reported publications count has grown over the years:
Year 2010: 1550 publications
2011: 2385
2012: 3767
2013: 5907
2014: 6699
2015: 7360
2016: 8279
2017: 8124
2018: 8185
and data still coming in for 2019 (due to the time scales of peer reviewed publication): 1937
The NIHR supports the principle of Open Access to the outputs of research, which can offer both social and economic benefits, as well as aid the development of new research and stimulate wider economic growth of the UK economy. A review of our Open Access policy is currently underway. The review is focusing on a number of areas including the development of a common approach to monitoring and compliance tracking.
Publication types
A wide range of types of publications result from NIHR research:
49208 Journal articles
3204 Conference Proceeding Abstracts
548 Book Chapters
209 Books
113 Theses
73 Policy Briefing Report
45 Systematic Reviews
43 Monographs
1549 Other
Beyond academia, NIHR award holders publish a wide range of other materials including consultancy reports, policy briefings, technical reports and standards and working papers. Publications of different kinds capture the knowledge from NIHR funded research that can be shared in different formats to meet different stakeholder needs.
Investing in people
NIHR Academy introduced the Researchfish Career Tracker in 2016 to capture the career progression of NIHR Academy Members. These questions explore issues such as the experience of undertaking an NIHR award, how much time award holders spend on research and future career plans. Analysis of the information allows NIHR to develop our training and support programmes and track outcomes.
For the 2019 submission period, a total of 1,448 award holders were asked to participate in our Researchfish Career Tracker, and 1,124 award holders provided a response. Our analysis shows that 68% of those undertaking an NIHR Academy award are doing so as they wish to pursue or further their academic career, or because they wish to pursue a clinical academic career. In addition, 87% reported that they had achieved what they set out to by undertaking their NIHR Academy award.
Reported post-award employment areas
Post-award, reported areas of employment are:
36% in joint clinical/academic roles
31% in purely academic roles
26% in purely clinical roles
5% in other kinds of role
Our Researchfish data shows that 68% of those who completed an NIHR Academy award remain in an academic role. Of those who return to clinical practice, only 41% are no longer involved in research.
“The doctoral fellowship has been a fantastic opportunity to develop my technical skills as a statistician and health economist. I have been able to attend a number of training courses in these areas that I wouldn't have otherwise had the opportunity to attend. The training and development aspect of the fellowship has also enabled me to spend time at a number of different universities strengthening my networks and collaborations for future research. I have been able to establish myself as a researcher working at the forefront of statistics and health economics with strong technical skills but also the networks required for collaborative and successful research projects”
--Laura Flight, Research Associate in Health Economics and Statistics, School of Health Related Research, The University of Sheffield
“It allowed me the time to undertake my research and put me in control of that time as it was my award. The award also allowed me to meet others who were at higher levels on the academic ladder and therefore able to learn and call upon their experiences when needed. I have also had the pleasure of supervising multiple ACF's, academic F2's who have contributed greatly to my research team. The support I have received via the NIHR has enabled me to build a successful academic career.”
--David Humes, Associate Professor of Surgical Epidemiology, The University of Nottingham
“I have gained numerous benefits from undertaking this award. This has been a unique opportunity for me to experience 'the other side' of running clinical trials, across a wide range of clinical environments, and the insight I have gained has been hugely beneficial. It has also been a fantastic opportunity to work with leading triallists across a range of disciplines, understand the lessons they have learnt from their experience, and for me to gain advice as I prepare to start leading my own trials. I have gained a truly unique set of skills, as someone with an unusual background (MRI physics), and this has positioned me perfectly to become a leader in my field. I have built relationships that will be crucial for the next stage of my career, and the skills and insight I have gained are absolutely invaluable. ”
--Peter Lally, MR Physics Fellow at the NIHR Imperial BRC, Imperial College London
Development
The NIHR is actively engaged in Researchfish governance and development activity. We actively contribute to the Researchfish Steering Board and associated subgroups. Through the Interoperability Steering Group we are working closely with representatives from UK Research and Innovation, research organisations and Researchfish to improve the flow of information between information systems and reduce duplication of effort.
During 2018-19 we have worked closely with the Novo Nordisk Foundation to develop a set of questions to gain better insight into the impacts research has on the education, training and professional development of others. We have also continued to work with research funding charities to further develop cross-funder PPI questions. We are also working with other funders to improve how information can be shared to support comparative analysis.
We continue to support work to improve use of unique identifiers such as ORCID, and encourage researchers to link their Researchfish and ORCID accounts. Giving permission via individual account profiles for the two systems to share publications and award data saves time inputting information and improves data quality.
In 2019, the NIHR continued to implement its sanctions approach. A small number of awards that failed to provide a submission had sanctions applied. We are hugely grateful to award holders who provide detailed information to our annual submission.
Get Involved
NIHR researchers and NIHR-funded research organisations are strongly encouraged to get involved with the Researchfish submission process and shaping future developments through the subgroups, if you’d like to get involved please email support@researchfish.com.
The NIHR remains committed to ensuring that the submission process is as straightforward as possible. Explore our website to access regularly updated guidance and downloadable resources. Contact the NIHR Researchfish team evaluation@nihr.ac.uk for more information. The next Researchfish submission window opens on 3rd February 2020 and closes 4pm 12th March 2020.