Internet Explorer is no longer supported by Microsoft. To browse the NIHR site please use a modern, secure browser like Google Chrome, Mozilla Firefox, or Microsoft Edge.

Research for Patient Benefit Programme - Guidance on further evaluations of established interventions


01 July 2017


1.0 - July 2017


Many types of intervention such as cognitive behavioural therapy (CBT), exercise and outreach have been shown to be widely effective. Even so RfPB, together with other NIHR programmes, frequently receives applications for further evaluations of the effectiveness of such approaches or variants of them in different populations and for different indications. In judging such applications, three common questions arise:

Is a new trial in a different target population justified?

A common type of proposal is to evaluate (through a trial or trial feasibility study) the effectiveness of an intervention for a condition for which it has already been tested, but in a new population – for example, an exercise intervention in young people, old people or ethnic minorities. Another common application is for funding to evaluate the intervention in a familiar population that has a physical condition not previously included in previous research – prostate cancer, multiple sclerosis, frequent attendance in primary care and so on. These may be important research questions but a panel will reasonably ask if results suggesting effectiveness in previous research can be extrapolated to the new population or condition so that further research is not needed. After all, the popularity of some interventions such as CBT and exercise encouragement resides in being flexible therapies defined by some general principles, the detailed content of the intervention often being tailored to individual need during therapy.

When submitting applications for evaluating an established intervention in a new target population or condition it is therefore important to identify why and how the new target is different from others that have already been researched. An application justified simply by stating that the intervention has never been tested in the proposed target population is unlikely to be successful if that is the only rationale. A case needs to be made that the new target population or condition has important differences that make extrapolation from previous work inadvisable - for example that the new population has been shown to have a different response to other therapies in other studies, or the new physical condition poses challenges that have not been addressed in previous trials. In short, it is not the absence of evidence that best justifies new studies but the distinctiveness of the target population.

Is a trial of a new variant of an established intervention justified?

The second type of study that is frequently received by the programme is testing of another therapy based upon modifying the form or content of an established one. There are two issues for panels to consider here.

First, proposals may not change the content of therapy but propose different formats for delivery – for example using computerised CBT, smartphone apps or therapists from different disciplines to deliver the intervention. In this situation it is unlikely that the new variant would have considerably greater effectiveness than the conventional therapy and the rationale is usually that cost-effectiveness can be increased by the new format. In many instances further research may not be justified: it might seem a reasonable inference that if, say, chronic obstructive pulmonary disease (COPD) nurses can deliver CBT effectively then cancer nurses or health visitors can too. But if a trial is proposed, the required non-inferiority design will need a large and therefore expensive study and applicants need to bear in mind that the cost of any trial might be judged as outweighing the potential incremental benefit to be achieved. Alternatively, new formats need to be justified by evidence that they are likely to increase coverage or retention in therapy and will therefore be more effective at a population level.

Second, a new variant is sometimes proposed because it is argued that an existing generic intervention does not adequately treat the population or the condition under consideration. Examples might include modification to respond to specific symptoms not otherwise addressed or to specific features of the target population. Since, as noted above, many of these interventions are characterised by their flexibility, a panel will reasonably ask if the proposed variant is really new or simply codifies what a competent therapist would do anyway. Applications for variants of established interventions therefore need to make a strong case either that the new therapy is likely to be considerably more effective (or cheaper) than the existing one if the latter is delivered according to accepted standards.

What facet of the intervention is being evaluated?

The exact active ingredient in many interventions is not well understood. For example, thereare two other components of the response to talking therapies from which the CBT effect needs to be differentiated. One is the non-specific effect of concerned attention, represented for example in the frequency and number of sessions. The other is generic therapeutic effects - the therapist’s skills and experience, the strength of the therapeutic alliance and so on. CBT is a limited and expensive resource in the NHS and applications will need to demonstrate that any effect demonstrated by the proposed intervention can reasonably be attributed to CBT and not to something that could be delivered more cheaply and just as effectively by other means.