Taking Stock – NIHR public involvement and engagement
This report brings together information from a number of recent exercises during 2018/19 to evaluate NIHR’s progress in advancing public involvement and engagement:
- An audit by the Department of Health and Social Care (DHSC) and the Government Internal Audit Agency of progress with implementing the recommendations in NIHR's Going the Extra Mile strategy for patient and public involvement (PPI)
- A survey of patient, carer and public experiences of being involved in NIHR research
- A survey of NIHR’s public involvement ‘workforce.’
- Other information giving some sense of the impact of public involvement, including data from Researchfish and the annual Patient Experience Survey by the NIHR Clinical Research Network.
The information provides helpful context for ongoing developments at the NIHR to strengthen and improve its approach to public involvement in line with its ‘Going the Extra Mile’ strategy. These include the announcement of the new NIHR centre to coordinate patient and public involvement, participation and engagement in research and NIHR research dissemination activity. The centre will be operational from April 2020 with updates on progress towards this milestone published on the NIHR website.
Progress with implementing Going the Extra Mile recommendations
Going the Extra Mile was signed-off as NIHR’s ten-year plan for public involvement and engagement in 2015. At that time, NIHR committed to audit progress with its implementation in three years’ time. This audit was conducted by the Government Internal Audit Agency in the autumn of 2018 with the co-operation of the Department of Health and Social Care, NIHR PPI Senior Leadership Team and patients and the public.
The audit concluded that ‘adequate progress is being made to achieving the Going the Extra Mile recommendations, and progress is being measured within a framework of priority areas and success measures. However, there is limited activity to aggregate this information and assess the overarching position. There is scope to improve the specific tracking of projects and activities, as well as linking to the achievement of the Going the Extra Mile recommendations within a more formalised framework.’
Areas for improvement
The audit identified four areas for improvement:
- Clarifying governance and oversight for public involvement and engagement in the NIHR. This includes: developing the role of the Senior Leadership Team (SLT) as the accountable and responsible body for NIHR public involvement and engagement and; formalising and strengthening current arrangements for prioritising, monitoring and reporting activities.
- Improving collaboration between NIHR Coordinating Centres, ensuring shared ownership for achieving the objectives and recommendations in Going the Extra Mile and realising potential efficiencies.
- Planning for the next phase of implementing Going the Extra Mile, including a formal framework of key accountabilities across NIHR Coordinating Centres and a consistent approach to agreeing and signing off on individual and collective plans.
- Internal and external communications including feedback to partners but, most importantly, to patients, carers and the public.
In addition to recommending publication of a summary of their conclusions, the auditors recommended:
- Recommendation 1: The format of the PPI SLT meetings should be enhanced to have a more formal structure with members reviewing and agreeing mutual expectations of the group.
- Recommendation 2: The NIHR should introduce a Communications Framework that maps out key stakeholders / audiences and a timetable of information requirements.
- Recommendation 3: The National Director should set out the required standards of PPI progress reports to all Coordinating Centres in terms of format, KPIs and frequency.
- Recommendation 4: NIHR should invest time and resources into strengthening the assurance framework and developing more project management methodology into setting objectives, tracking progress and measuring the outcomes of achievement of the Going the Extra Mile recommendations.
- Recommendation 5: Senior Management andtheBoardshouldreviewtheroleofthePPISLT Senior Leadership and consider the following:
- Increased responsibility and involvement in strategic and operational planning across NIHR and Coordinating Centres in terms of PPI;
- Oversight of PPI deliverables within Coordinating Centre contracts with the DHSC;
- Reviewing Coordinating Centre business plans to assess the adequacy of PPI planning.
- Recommendation 6: National Director should produce a progress report after the Internal Audit process (REC 11 of the report) for distribution to key stakeholders within NIHR and wider.
- Recommendation 7: PPI SLT / National Director should review the summary of comments from the survey conducted by Internal Audit, and consider the extent to which there is value in either creating a forum or promoting existing opportunities that enable employees and public contributors to provide a forum for insight and feedback.
NIHR public contributors’ feedback survey
Between December 2018 and January 2019, the NIHR undertook a survey of public contributors. The survey was made available online and proactively distributed through NIHR networks, partner organisations and on social media.
- NIHR’s public involvement community is strong, supportive and highly motivated
- It is highly focused on improving research to produce better outcomes
- There is a lack of diversity in the community in terms of age and background. NIHR’s continued ability to attract new people is, however, a strength.
- Public contributors are fulfilling a variety of roles in keeping with the INVOLVE definition of ‘active involvement’
- The NIHR gets good satisfaction ratings from people but experiences are varied and point to the NIHR needing to develop model approaches.
- The vast majority of public contributors report that they feel they make a difference and their contribution is valued
- The survey identifies clear improvement areas for the NIHR particularly around feedback, communications and acknowledgement
- The survey points to public contributors having strong local relationships with researchers and institutions but that the NIHR is a somewhat distant partner
Background information on survey respondents
There was a good response to the survey - 809 responses with a 72% completion rate. This is the highest response NIHR has had to a national survey of public involvement.
Young people and minority ethnic communities are under-represented in our public involvement.
- Younger age groups (under 25 2%; 26-49 14%)
- Minority ethnic groups (Asian ethnic groups 3%, Black ethnic groups 2%)
However, some of this may well be due to the nature of the survey and its design limitations.
Our community is generally composed of older age groups and predominantly white. - 58% women v 35% men 7% not stated.
74% identify themselves either as patients, service users or members of the public with lived experience. Only 8% describe themselves as carers.
A high proportion are also involved with other organisations, primarily charities, patient groups and the NHS (40-50% for each). Comparatively few are involved with social care organisations ( 7%) or pharma (- 9%).
What do they do with us?
NIHR is attracting new people - 12% of people said they had become involved in NIHR research in the last year.
People also stick with us. 7% of people have been working with us for – more than 10 years. 47% for 1-5 years and 19% for 6-10 years.
People fulfil a range of roles. The top four are:
- Review funding applications - 43%
- Members of a research advisory group - 43%
- Contribute to research funding applications - 33%
- Contribute at a strategic level (nationally and/or locally) - 32%
Experience – What is it like being involved with the NIHR?
67% say it’s generally easy to be involved with us taking into account the initial hurdles of finding out about the NIHR opportunities for involvement and dealing with the various application processes. 27% said it was sometimes easy. 7% said it was not, 4% didn’t know.
The vast majority (65%) say they think their contribution is valued but would like to receive more feedback on how it is valued and what difference it has made. 15% say they sometimes feel it is valued. 19% said they didn’t know.
They receive support from NIHR in a wide variety of ways:
- Help with expenses: 67%
- Reimbursement for their time: 52%
- Training: 36%
- Support for accessibility: 32%
- Mentoring: 19%
Help with payments was the most valued, followed by training. However continuing difficulties were highlighted related to inconsistent arrangements and policies across the NIHR for making payments and long delays in processing claims.
Making a difference
Most (69%) thought they were making a difference or sometimes (15%) thought they were. 3% said no and 13% said they didn’t know.
Over 500 people responded to the question about what difference they would like their involvement to make. Most wanted to focus on improving the quality and relevance of research leading to better health services and provision of care. They were keen to encourage researchers and clinicians to understand the benefits of public involvement, to make participating in research easier and research more accessible and widely disseminated.
People said they would like to improve the quality of research resulting in more effective treatments and delivery of health care.
They wanted their involvement to:
- Result in research that focused on what really matters to patients and patient outcomes.
- Develop and support research that is worthwhile and practical, reducing the delay of putting research into practice and improve treatments and care for specific health conditions – not just result in academic papers.
They also want to help researchers understand the benefits of public involvement and to demonstrate the value of public involvement to researchers and clinicians
People want to improve the research experience for patients participating in studies by, for example, advising on and producing accessible / plain English information for trial documents.
Reach / Research dissemination and accessibility
People want to help improve the way research is presented and reported so that it can be easier for the general public to take part, find and understand research findings and results
Challenges to making a difference
Concerns were raised about the barriers to wanting to make a difference - not being listened to, experience not valued, tokenism and lack of support
Feedback and communication
22% of people said they have not received feedback on their contribution and 19% say it only happened sometimes. 55% say they have had feedback.
Gave feedback: Nearly one third, 32% of people say they have not had an opportunity to provide feedback on their experience. 49% say they have and 13% reporting only sometimes. 6% didn’t know.
64% say they have been kept up-to-date about the NIHR work they are involved in. 20% say sometimes. 13% say they have not, 3% didn’t know.
In terms of being kept up-to-date people’s preferences are:
- Monthly emails
- Face-to-face meetings
- Online forums
Overall satisfaction with experience of involvement in NIHR
Most (79%) said they were very satisfied (41%) or satisfied (38%) with their experience of being involved in NIHR. Generally they said they felt valued and a useful contribution to help improve the quality and focus of research but noted there was still a lack of consistency/ support for PPI across the NIHR, unnecessarily complicated processes, lack of feedback, difficulty of finding opportunities for involvement and the restricted mix of people getting involved. 15% were sometimes satisfied, 7% not satisfied (3%) or didn’t know (4%).
Suggested actions for the NIHR
Towards the end of the survey people were asked to suggest one thing the NIHR could do to improve the experience of involvement.
The two themes highlighted related to feedback and communications.
Suggestions on feedback included requests for:
- feedback on public contributors’ individual contributions and the difference they made, if they should do anything different for example in reviewing and what happened as a result of their review
- feedback on projects that get funded
- feedback on the findings of research they were involved in as well as the impact of PPI in the study
- feedback in general re PPI in NIHR and NIHR research what is happening
Suggestions on communications highlighted the need for:
- More effective and accessible communication about the work of NIHR in general
- NIHR PPI newsletter that includes information on full range of opportunities for involvement, updates on research projects, links to wider NIHR issues
- Acknowledgement of public contributors’ involvement in articles and research
- Keeping in contact – making public contributors feel valued, acknowledged for their involvement as meaningful rather than tokenistic
Other themes highlighted include:
- “Give me more to do!” - Public contributors want to do more, be offered more and know how to find out about other opportunities for involvement
- “Encourage use of plain language” - Plain English/jargon guide - make the NIHR’s work and the public contributors’ role all easier to understand
- “Have a more diverse range of people … to make NIHR’s work much more meaningful and reach out to a wider audience” - Inclusion of different skills and experiences
- “Openness regarding the choices you make and more about who actually makes them.” - More transparency and openness about NIHR processes for public involvement
Concluding insights from the comments provided
Our public contributors are highly positive about NIHR research and want to be involved to improve it and produce better outcomes for patients and the public.
Whilst the survey points to public contributors forging strong relationships locally - whether it be with their local organisation, research team or principal investigator - the NIHR is seen to be somewhat distant.
There is clearly variability in people’s experience pointing to inconsistency in NIHR’s practices and approach.
Feedback, communication, acknowledgement and regular contact are significant areas where we as the NIHR can improve.
We need to pay more attention to some of the fundamental that enable involvement such as use of plain English
People want more to do and know more about the difference their involvement is making to NIHR-funded research.
INVOLVE workforce survey
In August 2018, INVOLVE conducted an online survey to gather information about Patient and Public Involvement and Engagement (PPIE) staff funded by the NIHR, targeting all NIHR centres, units and initiatives. Key data from the survey include:
- 179 PPIE roles were described
- The largest proportion of roles (two in five) described as full time equivalent (FTE), were between 0.1 and 0.4 FTE
- PPIE was the primary job function of 56% (N=100) of the roles described
- 52% of the roles for which the data was provided (n=127) were at manager or team leader level
- 65% of the roles for which the data was provided (n=163) had been in post for one to five years
Key themes to emerge from the survey included:
- The variety of approaches to PPIE roles. In some organisations there was no dedicated PPIE role and PPIE was considered embedded in roles. And for some there were dedicated PPIE posts, while for others there was a strong reliance on volunteers and available expertise for ad hoc or project work.
- Strong commitment to PPIE. A key theme that emerged from the analysis was the strong commitment to PPIE, but this commitment was often undermined by a perceived lack of resources.
- Lack of resource for PPIE workforce. There was a perception that PPIE was not adequately resourced. It was noted that the workforce had already experienced, and felt vulnerable to, budget cuts.
- Valuing the workforce. Respondents indicated that they felt that PPIE work was not always valued within their organisations and the skills that are used in their PPIE work are not always understood, valued or developed.
Other impact data
Since 2016/17 Researchfish has been collecting data from researchers on the extent of involvement in their grants as funded by the NIHR and a number of medical research charities.
To date this has shown that the level of involvement is less in later stages of the research process. Initial analysis by the Research Fish team suggests that the research community is involving people in the early stages of research (such as identifying and prioritising research questions). But that involvement fades as research moves into data analysis, writing up and sharing our learning.
From this year (2019) the following has been added to the common set of questions in the PPI section of the form - “Qu 6. What difference do you think patient and public involvement in your research has made to your research?” This will enable analysis and comparison across different types of research and funder.
Patient experiences of being in research
After a successful pilot in 2015/16, the NIHR Clinical Research Network (CRN) has conducted an annual survey of patients about their experience of being in a research study.
A record-breaking 8,500 responses was collected in 2018/19. Our survey focused on two main questions and we found that of those respondents: 90% reported that they would agree or strongly agree that they had a good experience of taking part in research. 93% reported that they would agree or strongly agree that they were given all the information needed in relation to the study.
The survey – the results of which are available on the NIHR website - also points to key factors that help drive good patient experience. These are:the friendliness and professionalism of research staff; providing the right information about studies at the right time and in the right place and; being able to access research which is local to people. While the instances of people having a bad experience are few, their feedback is just as essential to us too
The NIHR now uses the results of the survey to strengthen and improve its service to patients, carers and healthy volunteers in studies up and down the country.