David Wright, Dr Kamal R Mahtani
Ensuring primary research is informed by a review of what is already known
Why are systematic reviews so important?
Evidence based healthcare relies on the availability of the best evidence upon which patient centred clinical decisions can be made. Reviews make a significant contribution to this by attempting to identify the totality of evidence available, appraising and then summarising this evidence. There are many examples of how information gained from the systematic gathering of evidence in subsequent reviews has provided patient benefit, or protected them from harm.
Systematic reviews add value and reduce research waste. This is important given that poorly designed or informed studies make a significant contribution to the waste of research funds. By seeking all available evidence for a given topic, the appropriate and timely conduct of a systematic review can help researchers design new primary research (e.g. decisions-making about recruitment strategies, sample sizes and study methods), or indeed whether a new study is actually needed. Unnecessary research wastes resources and can cause harm to patients.
This is powerfully illustrated through cumulative meta-analysis, which can show how early systematic gathering of existing evidence before the conduct of primary research could improve health outcomes and save resources. As Chalmers and Glasziou point out, ‘new research should not be done unless, at the time it is initiated, the questions it proposes to address cannot be answered satisfactorily with existing evidence.’ Despite this message, repeated audits continue to demonstrate that primary research is not often enough informed by a systematic review of what is already known.
Systematic reviews and the NIHR
The NIHR recognises the importance of systematic reviews of existing evidence, funding and supporting reviews through many of its programmes, such as the Systematic Reviews Programme (including support to Cochrane through funding streams and infrastructure funding), and Technology Assessment Reports (TARs) (which provides independent reviews for NICE guidance). The NIHR advocacy of the systematic gathering of existing evidence extends to all primary research applications: researchers applying for funding are expected to ensure that their proposal is informed by a review of existing evidence.
A study conducted by the NIHR Research on Research programme assessed the degree to which trials funded by the NIHR HTA programme used systematic reviews to inform their research design. The study team looked at all trials funded in the period 2006-8 (47 trials) and in 2013 (34 trials) and found that all studies that could reference a systematic review in their application did so. Where the application failed to cite a systematic review, there were appropriate reasons for this.
Novelly, the study went on to explore how researchers use systematic reviews when planning new trials and found that the most common reason was to justify treatment comparisons. Other reasons included obtaining information about adverse events; defining outcomes; and aspects of study design, such as recruitment and consent. The study demonstrated that the NIHR requirement for using systematic reviews in the design of primary research is effective: 94% trials used at least one systematic review in their design and planning, and for the 6% that did not, there were valid reasons for this.
The Programme Grants for Applied Research (PGfAR) scheme, which funds large, interconnected, multi-workstream programmes of applied health research, also analysed their portfolio of 194 funded awards. They found that 91% of funded applications referenced a systematic review in their applications, and a majority (55%) of all funded applications included a workstream in which a systematic review was conducted and directly contributed to the other workstreams within the programme of research. This again demonstrates how the NIHR is encouraging the systematic gathering of evidence to enhance and add value to research being undertaken.
The Programme Development Grants (PDG) scheme was started to increase the rate and number of successful applications for full Programme Grants following an observation that promising applications had been unsuccessful as parts of the proposed programme were deemed too insecure and risky for substantial programme grant funding. Researchers could apply for a PDG to complete the necessary preparatory work to position the team suitably to submit a full Programme Grant application. Analysis of the PDG portfolio found that 53% of applicants who progressed from a PDG to a successful PGfAR grant application had included a systematic review in the PDG, further demonstrating that NIHR is encouraging the full evaluation of evidence before research takes place.
NIHR and the process of adding value in research
Processes have been installed at the NIHR to ensure that applicants utilise systematic reviews to inform new research. These include:
Guidance notes for applicants on systematic review use
All full applications are reviewed by external experts, including clinicians, academics, methodologists, and public contributors. External reviewers are asked to consider whether the proposed research addresses research gaps. Typical questions include:
- ‘What impact are the results likely to have on clinical practice?’
- ‘Does this research ‘overlap’ with other similar research that you are aware of?’
- ‘Does the proposal make a clear case for the research proposed with reference to the current evidence base?’
Concerns raised by reviewers are noted by the programme secretariat and are considered by Programme Board Members when making funding decisions.
Securing external reviews
Guidance notes for applicants state clearly that primary research will only be funded where the proposed research is informed by an appropriate review of the existing evidence. For applicants responding to a commissioning brief, a review of the literature is undertaken by an in-house research team to inform a call, and applicants must respond to this in their application.
For researcher-led primary research proposals, applicants have to reference existing evidence and explain how this has informed the proposed research. Systematic reviews must be referenced and where none exist, applicants must undertake a review of available and relevant evidence and include details of the search strategy in the application.
Programme secretariat support of funding board decision making
The NIHR programme secretariat assess all applications, ensuring guidance has been followed and feeding back concerns or recommendations to Programme Board members. Internal ‘overlap’ checks are performed at outline / expressions of interest stage and more detailed checks are carried out for full applications. This ensures that research applications do not duplicate published or ongoing research.
Summary
Supporting the appropriate conduct of systematic reviews is a vital part of the NIHR commitment. This support extends beyond core streams, such as the Systematic Reviews Programme. Prospective applicants to NIHR funding are offered guidance notes to ensure that all primary research is informed by an appropriate review of the existing literature, and investment in the Programme secretariat help to ensure this requirement is fulfilled.