Public Involvement in Social Care Research

Foreword

The NIHR is one of the principal funders of adult social care research in England. We are committed to boosting social care research and have increased funding and capacity building opportunities across NIHR – including the now annual Research for Social Care call.

The Social Care Institute for Excellence (SCIE) was commissioned by the NIHR to undertake a piece of work to identify good practice in user, carer and public involvement in social care research, explore differences between involvement in health care and in social care research, and highlight opportunities for further work.

Social care research has a long history of user, carer and public involvement and we welcome this report which provides us with helpful guidance on where we can continue to develop and strengthen our partnership with service users, carers and the public in social care.

Some of NIHR’s internal structures have evolved since the writing of this report. Most notably, recommendations refer to NIHR INVOLVE. From the 1st April 2020 the NIHR brought together its activities in patient and public involvement, engagement and participation (previously led by NIHR INVOLVE) with its activities in research dissemination to establish a new NIHR centre.

The work for this report was carried out in 2019 but the recommendations remain highly relevant and have influenced subsequent plans. The report identifies that NIHR needs to work differently – taking research closer to the people in the communities where they live and work, and building sustainable relationships over time.

This need was starkly illustrated by the COVID-19 pandemic. COVID had a significant impact upon people using care and support services. The pandemic exposed inequalities which are yet to be fully explored, but we know that some groups of people using care and support have experienced much greater risk of illness and death (for example, people with learning disabilities), that access to essential services has been reduced or withdrawn and that there has been an increased toll and mental health impact upon carers.  It has never been more important that social care research is prioritised, designed and undertaken in partnership with users, carers and the public, so that future services can better meet the diverse needs of people and communities.

The recommendations made in the report fall within four main themes. Below we lay out how the NIHR has so far been working to address them through its work to meaningfully engage with, and involve, service users, carers and the public.

Our work is by no means complete - and we will continue to challenge ourselves and others to engage with the findings of the report and fully realise the recommendations made within it. We welcome the opportunity to hear your insights on the themes raised in this report. You can share these by contacting ced@nihr.ac.uk.

Jeremy Taylor NIHR National Director for Public Voice and David Abbott, NIHR School for Social Care Research Associate Director

How NIHR is working to address SCIE recommendations

Theme 1: Committing to continue to embed strong public involvement and engagement in NIHR social care research

NIHR has expressed its commitment to social care research, including a new annual Research for Social Care funding call and the appointment of the Applied Research Collaborative (ARC) Kent, Surrey and Sussex as the National Lead for Social Care. There has been investment in the skills and knowledge of researchers through the NIHR Academy and five new large adult social care partnership projects have been established. The NIHR Adult Social Care (ASC) Coordination Group is ensuring a shared approach to raising awareness of, encouraging and supporting social care research across all parts of the NIHR. All social care research funded through NIHR is required to ensure appropriate opportunities for public involvement in the research, with tailored approaches to reach populations using social care services as well as carers.

Public involvement is a key feature of the NIHR School for Social Care Research (SSCR) and is vital to all of its research and research capacity development activities. We will draw on the advice and extensive expertise of the SSCR in embedding effective public involvement and engagement in social care research, and we will work with the NIHR Programme Directors and Chairs, NIHR CED, the NIHR ASC Coordination Group and a broader range of stakeholders to ensure this is truly a cross-NIHR effort.

We will work with the NIHR Programme Directors and Chairs to ensure that they understand what can be done to ensure our public involvement and engagement activities, for both our commissioned and research-led funding calls, promote inclusion and strive to be accessible to social care users, carers and the public.

Theme 2: Championing Equality, Diversity and Inclusion

NIHR is working to ensure that our governance and decision-making groups include a greater presence from those who use and work in adult social care. For example, public involvement is a key feature of the NIHR School for Social Care Research (SSCR) and is integral to their research and research capacity development activities. The SSCR has recently refreshed membership of their User, Carer and Practitioner Reference Group (UCPRG) and is working with them to develop a programme of activity. NIHR Schools for Social Care, Primary Care and Public Health have met to explore options for further developing inclusive public involvement and engagement across all School activities.

The report highlights that universities can sometimes generate significant barriers through their finance and payment arrangements, and by providing  poor incentives for academics that want to work collaboratively with service users, including disabled people. Some of these  barriers relate to HMRC policy - or how it is interpreted - and the processes and procedures of individual universities. The NIHR CED has recently refreshed its payment guidance for researchers and payment guidance for members of the public and we are working with institutions - as a cross UK project - and those affected by payment barriers, both to encourage institutions to adopt more inclusive solutions in line with current HMRC guidance, and to influence the policy environment (in partnership with other organisations like the Health Research Authority).

Alongside the Research for Social Care call and wider NIHR Equality, Diversity and Inclusion activities, we will undertake a strategic project to improve levels of inclusion, diversity, representation and equality in social care research funding – for those developing, conducting research and taking part in it.

Theme 3: Engaging communities and supporting user-led research

We recognise the importance of community engagement and the burgeoning work in this area.  There is learning that can be shared from our Global Health Research portfolio and we will encourage all parts of NIHR to develop effective approaches to stimulate the development of relationships between communities and researchers/NIHR programmes for more inclusive research partnerships. The “Reaching Out” programme has produced a summary of learning on engaging with people from marginalised and seldom heard communities and follow up work is planned to determine next steps for NIHR.

The number of SSCR-funded research studies that are in collaboration with users, carers and/or practitioners is increasing, and is attracting user-led research and career development proposals. To further aid such efforts, the SSCR is currently funding scoping work to build capacity for user-led and co-produced research, with a view to develop a proposal for a pilot scheme and will share learning across NIHR.

In 2021/2 CED and the NIHR Clinical Research network public involvement leads will work with a range of carers to identify how NIHR can best improve and advance how it engages and works with carers in shaping and delivering health and care research.

Theme 4: Providing accessible, practical and relevant resources

The report raises the importance of developing, or adapting, accessible, practical and relevant resources to support social care researchers and users involved in producing research, and disseminating them appropriately. The most used resources which INVOLVE produced have recently been reviewed, updated and republished on Learning for Involvement. CED is now responsible for curating, updating and disseminating NIHR guidance on public involvement, engagement and participation, and has committed to ensuring that new resources are inclusive of social care research and include social care examples. This approach has been taken with the most recently published resource; the final publication of the ‘Co-production in Action’ series in April 2021. This document builds upon the NIHR’s Guidance on coproducing a research project by illustrating co-production in action in different types of NIHR funded research, and has a strong emphasis on social care research examples.  

The SSCR annual conference took place virtually this year, and included live subtitled and British Sign Language interpreters, to improve the accessibility of the event. We will continue to share such learning and practice across NIHR.

Public Involvement in Social Care Research: Final report to NIHR INVOLVE

Alison Faulkner, Kate Pieroudis, Pete Fleischmann - Social Care Institute for Excellence, February 2019

Executive Summary

This report presents the results of a piece of work undertaken by SCIE for NIHR INVOLVE to explore public involvement in social care research. We interviewed 14 key people, held a workshop to discuss early findings and carried out a brief literature review. The aim of the work was to surface good practice in involvement in social care research, explore differences between involvement in health care and social care research, and identify opportunities for further work. 

This piece of work concludes that the practice of public involvement in social care research shares much in common with health care research. Many of the elements that support good practice or act as barriers to involvement are shared across the two fields. Guidelines that address public involvement in health care research can be adapted for use by social care researchers. However, we heard from many people who felt that issues of equal access (for Disabled people, people with different health conditions and people with caring responsibilities) need a greater focus in social care research, in order to ensure that research is accessible and people can be meaningfully involved.

Social care research is the poor relation of health care research, receiving less funding and less status within the research world dominated by a focus on evidence-based practice. Unlike health care, social care is not a universal service, and is largely provided by unqualified care workers employed by a range of different statutory and private social care providers. This results in a fragmented service which does not lend itself to service user and carer involvement. It also results in fewer research-active professionals overall. This means that practitioners also need to be considered as potential stakeholders in research.

Social care researchers less often use the terms 'public involvement' or PPI (patient and public involvement). They more often talk of service user or carer participation or involvement, or use terms such as inclusive or participatory research. This connects with the historical context to social care research rooted in the Disabled People's Movement. This history has created greater awareness among some social care researchers of the power relations inherent in research and has created a context for the use of participatory methods.

Some of the differences lead us to highlight the importance of making available accessible practical examples of involvement in social care research and guidance on enabling equal access for people with different disabilities and needs.

Recommendations

• INVOLVE could develop a resource to direct social care researchers towards useful resources to include, for example, those provided by SCIE; and adapt their current resources to be inclusive of social care research by (for example) reviewing the terminology used.
• INVOLVE could collate practical examples of public involvement and co-production in social care research, would help to enable both researchers and service users & carers to understand what it looks like and what is possible. This project could include the use of film and other formats to ensure accessibility to a wide range of disabled people and social care service users.
• Guidance on making research accessible: The consultation suggests a need to collate and disseminate the experience and expertise enabling equal access for all of the many different groups of disabled people and people who use social care services: how to make research genuinely accessible. For example: the use of different communication formats, managing meetings, engaging people from different communities, the use of technology, using creative methods and so on.
• Co-production: NIHR INVOLVE and SCIE could usefully explore further work together to ensure that guidance on co-production is relevant to the social care research and service user communities.
• The School for Social Care Research: would be a good partner for INVOLVE to explore ways of taking these recommendations forward.
• Community engagement: INVOLVE could explore good practice in relation to community engagement, as part of making research relevant and more widely accessible to people from marginalised and seldom heard communities.
• NIHR could set aside funding to support user-led research and DPULOs: much of the genuinely innovative social care research has been undertaken by Disabled people's and user-led organisations.
• INVOLVE membership: a greater presence of people from the social care world, researchers and service users & carers, would help to build the further presence and priority given to public involvement in social care research.
• NIHR could make a statement committing to do more in terms of social care research, and with an expectation that there must be public involvement.
• Those involved in commissioning NIHR research need training/awareness raising on the issues that are particular to social care research.  This includes the funding implications of ensuring equal access for public involvement in social care research.
• Disabling barriers presented by Universities: Both SCIE and INVOLVE could review together how best to address the disabling barriers presented by Universities - perhaps by producing a discussion paper or organising a seminar. 

This was a small scale piece of work designed for a ‘light touch’ paper: further work would enable a fuller investigation of the issues raised here as well as the development of solutions.

Glossary of acronyms

• BAME: Black, Asian and Minority Ethnic communities
• DPULO: Disabled people's and user-led organisation
• ESRC: Economic and Social Research Council
• IMHA: Independent Mental Health Advocacy
• LGBTQI: Lesbian, gay, bisexual, transgender, queer, intersex
• NHS: National Health Service
• NICE: National Institute for Health and Care Excellence
• PI: Principle Investigator
• PPI: Patient and Public Involvement
• RCT: Randomised Controlled Trial
• SCIE: Social Care Institute for Excellence
• SSRC: School for Social Care Research (part of the NIHR)
• ULO: User-led organisation

Introduction

SCIE was commissioned by National Institute for Health (NIHR) INVOLVE to produce a scoping paper to inform the development of good practice in public, patient and carer involvement in social care research. The aim of the paper was to:

• surface recent examples of good practice in involvement within social care research (ideally across the spectrum of social care provision)
• explore what good involvement looks like and is understood to be in a social care context
• explore any differences in involvement within social care compared with involvement in health research
• identify opportunities for the development of involvement (with a particular emphasis on frameworks, guidance and support to researchers, panels, members of the public, service users and carers) within social care research, by NIHR and its partners
• inform future INVOLVE/cross-NIHR work plans, frameworks and guidance from 2019 onwards

The main audience for the paper is internal to NIHR - senior NIHR managers - as part of a wider examination of what NIHR can do to further stimulate and provide the right frameworks and support for increasing effective social care research. In addition, INVOLVE will consider using some part of the report to share with external audiences in line with its mission to raise aspirations, drive up standards, provide guidance and facilitate partnerships.

Background

Public involvement in research

INVOLVE defines public involvement in research as “research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them.” (INVOLVE, 2012). It posits an active partnership between members of the public and researchers. This could include working with research funders to prioritise research, offering advice as members of a project steering group, commenting on and developing research materials and undertaking interviews with research participants. In using the term 'public involvement', INVOLVE refers to a wide range of potential research partners as members of the public:

• patients and potential patients
• people who use health and social care services
• informal (unpaid) carers
• parents/guardians
• disabled people
• members of the public who are potential recipients of health promotion programmes, public health programmes, and social service interventions
• groups asking for research because they believe they have been exposed to potentially harmful substances or products (for example pesticides or asbestos)
• organisations that represent people who use services.

This describes a broad understanding of the 'public' who might be involved in research and it is not one shared by all organisations or individuals, nor is it relevant to all areas of research. In this piece of work, which focuses on social care research, we reflect the views of those we have consulted in understanding 'public involvement' rather than necessarily retaining the INVOLVE definitions.

In the past, INVOLVE used the terms consultation, collaboration and user controlled to describe different levels of involving people in research. However, it became clear that, in practice, research projects can include a combination of these three and also that the boundaries between them are not clear cut. They now describe these terms  as approaches to involvement as there are many different ways of involving people during the various stages of the research. Other organisations inevitably use different terms and this is explored further in this report.

Social care

In order to understand social care research, it is essential to understand the complex service context in which social care is provided. Social care covers a wide range of services and activities, from child protection through to end of life care, including people with physical disabilities and people with learning disabilities, people with mental health problems, older people and people at risk of abuse. Most of the people who provide social care are care workers working for social care agencies, although this does not account for the estimated 6 million unpaid carers including family and friends. Trained social workers account for a small proportion of the overall workforce.

One of the important differences between social care and health care is that, unlike health care provided by the NHS, social care is not a universal service. There are many small private providers and social care is delivered and managed differently by local councils across the country. As the UK population gets older, the demand for social care is increasing. However, the funding for social care is far less than that for health care. In addition, the recent cuts to funding have resulted in a decline in the number of private agencies and care providers. The funding available for social care research is further limited because there is little incentive for private organisations, such as pharmaceutical companies, to become involved. 

The vast number of small private providers have little incentive to involve service users and carers in the way they work, even if the local council where they operate does so. There is little in the way of infrastructure to support the involvement of service users and carers in social care. In health care, there are networks and forums to support involvement, including Healthwatch, PALS (Patient Advice and Liaison Service) and numerous patient and carer forums in local Trusts.

Nevertheless, there is a history and tradition of involving service users and carers in the training and education of social workers - and this was made mandatory in 2002 (Department of Health, 2002). Most social workers will have been exposed to the perspectives of service users and carers during the course of their professional training. Cossar and Neil (2015) report that it was important to their research team 'that social work research should be undertaken with similar ethical standards to those governing social work practice' (p.230). According to BASW (the British Association of Social Workers), the principles of human rights and social justice are fundamental to the social work profession. In theory, this ethical context could provide a natural environment for public involvement in research to flourish.

Social care research

It should be said that there is no clear dividing line between health and social care research; just as people cross these boundaries with their needs so does the research that involves them. Certain areas in particular, such as mental health and dementia, often cross the boundaries between health and social care services and research.

There is a powerful history of emancipatory disability research dating back to the 1970s and 1980s. This means that the participation of disabled people in research and the existence of disabled researchers, is more widely understood in social care research than it has been traditionally in health care research (Beresford, 2010). There is a strong tradition of exploring the power issues in social care research with strong ethical and political imperatives to challenge the nature of research production, going back to the inception of the disabled people's movement. Peter Beresford has written of this in a number of publications (Beresford and Carr, 2018; Beresford, 2010).

Earlier in this section, we described the different approaches to public involvement as: consultation, collaboration and (service user) control. However, there are other ways in which the approach taken to involvement in research can vary. One of these lies in the underlying ethos or values. Arguably, the ethos of participatory research and of user-controlled research is to make change happen: to make visible the knowledge of marginalised people and to empower people and communities through that knowledge-making process to change their lives (Beresford and Carr, 2018; Turner and Beresford, 2005). As Beresford (2010) points out, the central purpose of emancipatory disability research and user-controlled research, is to empower service users with a focus on what research is for; the process of research and where control of the research lies. Beresford (2009) distinguishes between democratic and managerialist approaches to involvement; the former having explicitly ethical and political aims and the latter having more pragmatic goals that avoid challenging existing power relations. Participatory research is more closely associated with the democratic approach and tends to be concerned with challenging the power relations involved in research. Much of the research taking place across the broad spectrum of public involvement in research, may have more pragmatic aims in mind - such as, bringing in a different perspective to the research or, indeed, gaining funding. The involvement of service users and carers in research may be seen as providing one part of the picture, giving involvement of service users and carers a more pragmatic, if important, presence in research. However, as has been pointed out by Cossar and Neil (2015), the involvement of service users for purely policy or funding purposes is in danger of being tokenistic, as it may not be embedded in the process of research nor expected to influence the outcomes.

In truth, there is a range of political, ideological and pragmatic motivations underlying all of these approaches to research. Terms such as participatory, emancipatory research and inclusive research are used more frequently in social care research, whereas the 'public involvement' terminology has more agency in the health care context. Another term introduced more recently to the research arena is 'co-production', a theme also explored in this piece of work. Whilst the terminology might not necessarily ally itself directly with a particular ethos, it can have implications for the nature and process of involvement.

A relatively recent development in the field is the establishment of the NIHR School for Social Care Research in May 2009, to 'develop the evidence base to inform and improve adult social care practice in England by commissioning and conducting world-class research'. The NIHR SSCR is a partnership between five leading academic centres in adult social care research in England. The SSCR has a service user, carer and practitioner reference group to advise their work.

Many of the publications and guidelines that address 'public involvement in health and social care research' make little specific reference to social care research (Brett et al, 2012). This means that more work is needed if we are to understand some of the differences and similarities, which is one of the aims of this piece of work.

Methods

SCIE has extensive experience of service user and carer involvement and co-production in research, through, for example, our involvement in the NICE Collaborating Centre and the new What Works Centre for Children’s Social Care (https://www.scie.org.uk/children/what-works-centre). SCIE has good knowledge of existing practice and access to relevant networks, and is a known expert on co-production.
The SCIE team adopted the following approach to this piece of work:

1. A brief literature review
2. Interviews with key people engaged and involved in social care research
3. A workshop to review the interim findings

A total of 14 people were interviewed: eight were employed as academics (two retired/emeritus professors); six identified as service users, survivors or disabled people; three were employed by voluntary organisations in a managerial capacity. The subject areas covered by the interviewees were: mental health, social care, disability, dementia, race and racialisation, LGBTQI issues, children and families, learning disabilities, alcohol, housing. In addition, interviewees had expertise in co-production, social work practice and teaching, law, participatory research methods, funding research and involvement in research.

The topic guide used for interviews is given in Appendix A. All interviews were recorded and service user participants were paid £50 if they were not otherwise employed. The literature review has been used to inform this report.

The workshop was attended by 15 people: disabled people, people who use services and carers associated with SCIE, and a couple of academics and people from voluntary sector organisations and DPULOs. 

The programme for the workshop is attached in Appendix B. During the day, some participants were filmed talking to camera and an edited film is available as a resource from the day.

The interviews and the workshop revealed considerable passion among the academic, service users and disabled people involved in social care research. Some of the passion was for carrying out genuine participatory research; equally some of the passion originated from bad experiences of involvement or inauthentic co-production. In this report, we are assuming some prior knowledge of public involvement in research, as it is in the first instance intended for INVOLVE.

Please note: Projects included in this report as good examples of public involvement or co-production were nominated by interviewees; however, they have not been checked with other members of the relevant research teams for their views.

Findings

The findings are divided into the following sections:

1. Identifying the differences
2. Good practice in public involvement in social care research
3. Barriers to public involvement in social care research
4. Co-production
5. Recommendations

1. Identifying the differences

This piece of work confirmed that many of the practical elements of public involvement in research are common across social care and health care research - the section on good practice (page 15) reflects this. Issues such as: involving people from the start (or before the start) of a research project, ensuring good support and shared training, building in adequate time and resources, fostering relationships, are familiar themes from across the literature (INVOLVE, 2012; Rice et al, 2015; NIHR Mental Health Research Network, 2013). A few differences did emerge, however, and these are explored below. Particular thanks are due to the workshop participants who helped to draw these out.

The social, policy and funding context

As discussed in the Background section to this report, the policy and funding context for social care services is very different to the context for health care. This has implications for research practice and public involvement. There is less funding available for social care and for social care research than there is for health care. The involvement of a multiplicity of private providers in social care gives a fragmented and atomised context in which to develop service user and carer involvement.

There is a difference between health care and social care expertise in relation to care practice. Social care takes place overwhelmingly in people's own homes or residential care where practitioners are not 'experts'; whereas, health care tends to take place in clinics, surgeries and hospitals where practitioners are assumed to have some specialist expertise - or can refer the person to an expert. This potentially places the social service user in a different role in relation to the practitioners involved: at times, for example, more knowledgeable (about their own needs) but at times more at risk from people coming into their home or dependent on care home staff for their basic daily needs. Power imbalances are ever-present.

Social attitudes may affect social care service users more than (many) users of health care services. Social care covers a wide range of different people, many of whom face discrimination, stigma within society, poverty and social disadvantage. In recent times, social attitudes towards people depending on welfare benefits appear to have worsened, with people sometimes regarded as 'scroungers' and doubt cast on people's lived experience of disability. In addition, the tendency to perceive people receiving social care as vulnerable can militate against them being involved. (However, some groups within health care services and research will be equally affected by social attitudes; for example, people who use mental health services, people living with HIV and AIDS.)

Language and terminology

The language used to refer to people in social care research can be quite different to that used in health care research. Researchers less often use the terms 'public involvement' or PPI (patient and public involvement) in social care research. They more often talk of service user or carer participation or involvement, or use terms such as inclusive or participatory research. Some social care researchers have also used the term 'emancipatory research' and, more recently, co-production (see section page 19). For some people, the impression they have of 'PPI' is of a tick box exercise to meet funders' requirements: it can often represent a very low bar. (Conversely, co-production entails a more comprehensive exploration of the relationships, power and the expertise brought by all of the partners).
A key problem with public involvement terminology identified by the consultation surfaces a direct challenge to the nature of public involvement. The term 'public involvement' implies inclusion of members of the public who may be 'lay' people - i.e. they do not necessarily have direct experience as recipients of social care (or, indeed, of health care). This was felt to be particularly inappropriate in a social care context. The involvement of 'lay' people (or generic PPI representatives) can open up involvement to 'well-meaning retired professionals' or representatives from voluntary organisations, who have no direct experience of what it means to use social care services. People felt that it was essential to bring the lived experience of using social care services to the research - that involvement (and co-production) is about having a particular expertise grounded in lived experience, not about having no research expertise. [See the good practice example from SCIE]
Some interviewees felt that language in social care is generally less medical, not driven by or based in the medical model, as it is in health research. The social model and recognition of disabling social barriers is more prevalent and helpful. However, conversely, some disabled people/service users suggested that social care retains a significant reliance on the medical model, which tends to result in disabled people being pathologised.

Equal access and accessibility

The importance of addressing access requirements and making research accessible to people with a range of different impairments and needs, is not solely an issue for social care research. However, it is an issue of great significance to social care research, due to the wide range of people who use social care services and for whom the research might be relevant. Ensuring equal and easy access has to be a primary concern for researchers undertaking research in social care. The organisation Shaping Our Lives produced the report 'Beyond the Usual Suspects' (Beresford, 2013) which gives practical guidance on developing more inclusive involvement. They usefully distinguish between the following sources of exclusion:

• Equality issues: gender, ethnicity, culture, belief, sexuality, age, disability and class.
• Where people live: homelessness, living in residential services, in prison or secure services, travellers and gypsies.
• Communication issues: Deaf people, blind people and people with visual impairments, Deaf and blind people, people who do not communicate verbally, people for whom English is not their first language.
• The nature of impairments: people with complex and multiple impairments, people with dementia, people who are neuro-diverse.
• Unwanted voices: e.g. activists, people who are more experienced and confident in challenging the status quo.

Below, we address some of the issues that emerged from the interviews.

• Interviewees mentioned: flat access to meetings and buildings, easy read formats for written information, using different forms of communication - and the role of attitudes in enabling people to feel equally important and welcome in a research team. It is essential to take away the stress from being involved. 
  • The example of involving people with dementia was given, with many examples of ways in which it is possible to engage people with dementia (e.g. using cards to communicate in groups, pictures, turn-taking in groups, ways of remembering and reminding...)
• There can be a significant cost element to access and this needs proper planning and budgeting at the start of a project.
• Social class is a major issue in relation to involvement - particularly, but not exclusively, in social care research. Middle class educated people 'fit in' more easily to the academic enterprise. The use of 'lay' terminology complicates this, as it potentially opens up involvement to members of the public who do not have direct experience of social care services. Equally, if people are expected to take part without immediate (or any) payment of expenses, this creates a perverse incentive for less disadvantaged people to take part.
• Diversity has to be purposefully addressed - it cannot be left to chance. Exclusion of certain groups - particularly people from BAME (Black, Asian and Minority Ethnic) communities and people in residential facilities - remains a challenge to good involvement practice. Participants also mentioned the white middle class environment of academia as excluding people from BAME communities and working class backgrounds.
• There was some concern that some academics and academic departments have been using the same model of participatory research for a long time, involving the same people over and over again. Although this builds long term relationships, it can also become exclusionary over time and does not lead to involving new, younger or different people.
• The technology often used to aid access (e.g. Skype and its equivalents) can fail. It is vital that these aids work if people are not to feel excluded.
• Involvement needs creative methods, art, videos, not assuming everyone can talk in groups, and learn from others who have developed expertise in communicating with different groups of people.
• It is important to bear in mind those people who are not in the room and to be aware of gatekeepers in social care: people in care homes, people deemed to be too vulnerable or without capacity.
• It is also important to bear in mind that BAME community leaders may not represent their communities authentically - it is vital to reach the people with direct experience of using social care services.

Research methodology and the role of evidence

There are differences in methodology between social care and health care research. Participatory methods are more prevalent and have greater credibility in social care than in health care research. In health care, quantitative methods are more prevalent; the randomised controlled trial (RCT) is commonplace as it occupies a significant role in the formation of evidence-based health care. To obtain large grants in health care research, a clinical trial is more or less obligatory. The tendency for social care research to employ more mixed, qualitative and participatory methods, potentially opens it up more easily to public involvement. 

In conjunction with the above, the role of NICE (the National Institute for Health and Care Excellence) and its assessment of research evidence is greater in health care than in social care.

In social care research, the principle researchers are often academics but not clinicians - for example, sociologists. In health care, the researchers are more often clinical academics - leading research from a medical model and clinical perspective. In theory, this could lend social care research a broader theoretical and methodological approach. However, it also means that research is, or can be, separated from practice which has implications for the involvement of social care practitioners in research.

2. Good Practice Involvement in Social Care Research

This section does not look unlike good practice in any research context. A quick Google search reveals a plethora of guidance on public involvement in research, produced by individual NGOs in different health and social care conditions, DPULOs and Universities, as well as specialist organisations like INVOLVE. The following list reflects the responses received from the consultation; issues in bold were those most frequently mentioned.  

• Involvement from the START was identified by almost all participants as essential to good practice. Unless service users are involved from the start, much that follows can be tokenistic and people do not feel true ownership of the research. However, resources are needed to achieve this: funding that provides for engagement prior to getting funding.
• TIME is a major issue, both a need and a barrier. Good involvement takes time and research deadlines often mitigate against this.
• It is important for researchers to build up relationships of trust with people who user services, carers and disabled people.
• Recognising, acknowledging the different skills, experience and contributions people bring to research - including life experience and including the contributions of all members of the research team.
• Paying people appropriately for their time and expertise - and finding other ways of creating positive incentives for people to participate.
• Clear job or role descriptions help to clarify roles and tasks and reduce uncertainty.
• Transparency, honesty about the limits of influence: start by agreeing the terms of involvement/co-production but be honest about what is and is not in the scope of the involvement influence. Involvement is likely to fare better if the project begins with a well thought-out and negotiated strategy from the start: with clarity about the purpose of involvement, the limits of influence and where decision-making takes place.
• It is important to give space and support for people to attend to identity issues:  not just how people identify themselves but how they become identified as a result of involvement. Methods suggested included mentoring, peer support or external supervision; some people suggested finding creative ways to enable people to discuss this.
• Useful to have someone to talk to outside the project, a mentor, who is not the line manager, about difficult issues
• Endings emerged as a major issue: how to manage endings well in the context of  research contracts and funders' deadlines. It is frustrating for everyone when people who are 'involved' or on short contracts cannot be involved in writing up and disseminating. Academics can move on to another project but continue to write up when others have left.
• Funders also need awareness of the cost implications of good involvement and co-production: assessment of funding applications should include checking the budget for appropriate elements for inclusion and accessibility.
• Disabled people's organisations and service users/disabled people are often seeking practical outcomes from the research; it is important to build this in from the beginning.
• A plea was made for funders to put in place some kind of external audit after the funding has been awarded, to monitor the public involvement aspect of the research - and for researchers to monitor and write up the public involvement aspects of their research.

Example of involvement in research

'Tackling Disabling Practices: co-production and change'. University of Bristol School for Policy Studies, funded by ESRC. Nominated by PI Val Williams

The idea and proposal came from two sources: Disability Rights UK and Norah Fry colleagues; some colleagues were also disabled people. Instead of simply describing the barriers faced by disabled people, the project focused on mechanisms of change, aiming to find out what theoretical ideas have practical use. The research aimed to develop and sustain practices that are on the terms of disabled people themselves. Involved disabled people from the start. Disability Rights UK was in control of part of the research and on the management group. It was a large project with time: 'we had the luxury of being able to think.' They had regular meetings, the group was enthusiastic, met frequently to thrash out the theories and ideas that would be useful. The project took opportunities to bring in and work with other groups, e.g. - the dementia strand. Co-researchers included three disabled people who were also involved in analysing data and making the product at the end. Outputs included an accessible booklet; it was important that disabled people were involved in taking the messages and using them, disseminating the findings. Easy read outputs from the project can be found here.

Example of supporting user-led research

Dementia Enquirers: Innovation in Dementia cic, nominated by Rachael Litherland [text edited from website]

The aim of this project is to develop a new approach to research or 'enquiry', that is led and controlled by people with dementia themselves. The BIG lottery Fund has awarded over £700,000. The project runs from October 2018 - March 2022. Innovations in Dementia will support people with dementia to explore how knowledge is acquired and applied in a way that feels relevant to their own lives. Dementia Enquirers will support up to 20 DEEP groups to conduct their own small-scale research projects. Many groups have been involved in research studies carried out by universities. However, this project will give people with dementia a chance to lead, and be in control, of their own research - on issues that are important to them.

Training, guidance, support and ideas will be available to help groups to build their own research skills and expertise. University researchers may be involved as advisers or have roles in the projects. Small grants will be available for groups to get on with their own research projects. These groups will be encouraged to share their learning with the rest of the DEEP network.

The project will also explore with people with dementia the lessons that emerge from their work, and share this learning widely. We are confident that the project can influence the prevailing approaches in research - while bringing new respect for the skills, expertise and resilience of those who are living with dementia.

3. Barriers to public involvement in social care research

As with good practice, the barriers to public involvement in social care research are not dissimilar to those encountered in other areas of research. Beresford (2013), for example, identifies the following categories of barriers to involvement in social care: physical access, organisational, attitudinal and structural. Snape et al (2014) identified perceived tokenism as a major barrier to public involvement in research, as well as time and appropriate resources, scientific jargon, attitudes of researchers, power and control. In a systematic review on the impact of older people’s involvement in health and social care research, Baldwin et al. (2018) suggest that restrictive time limitations in research projects appear to be associated with tokenistic involvement and do not allow for the dynamic nature of public involvement. The authors call for funding bodies to have less restrictive contracts and timeframes to accommodate public involvement and provision of additional funding for evaluation within projects. 

Below we examine the barriers raised during the course of this piece of work, as there are perhaps some different emphases to be found. 

• Time and resources: funders rarely allow for the additional requirements of public involvement - and researchers may not take them into account either. This can result in projects ending badly or with only those on permanent contracts left to complete the writing up and dissemination.
• A perennial barrier is paying people for involvement and the difficulties of paying people who receive welfare benefits. It is essential to make payments easy to claim and to pay people's expenses promptly, otherwise some people can be excluded or left out of pocket - leading to inequalities of access. Benefits rules make paying people difficult and potentially open to sanctions - it is important to give people up to date information and support them in their claims.
• Employment status for people employed to be involved in research tends to be variable, precarious and problematic. Some people do not have formal employment status or pay and may be on zero hours contracts. This results in significant power imbalances and uncertainty, as well as unpredictable work and pay.  
• The University as a place of disabling barriers: this was a significant theme to emerge from this work.
  • Finance and arranging payments is a huge barrier - causing real upset on occasions, examples of projects and individuals waiting long periods for payment or a contract to be settled. One academic said that the barrier is so great that he no longer tries to involve service users in his teaching.
  • The incentives that count for career progression within academia (publications, research grants) are inherently disabling: they are individualising and competitive and do not support those academics who want to work in a collaborative way with service users and disabled people. Equally, these same issues represent significant barriers to the employment of disabled people and service users.
  • Precarious employment status for people involved or employed in a University setting has implications for the (lack of) power associated with this status. This also leads to difficulties in completing projects ethically, as those who remain to write up and complete when contracts and funding have finished will be the permanent employees.
• Power emerged in a number of ways:
  • (As above) the power imbalances represented by different employment status and pay in relation to academic partners, plus the knowledge imbalance between academic researchers and people involved for their experiential knowledge.
  • Power imbalance inherent in identity (as disabled person, service user, carer) can be exacerbated if it is not discussed within the team and the value of experiential knowledge made clear.
  • Service users/disabled people becoming aware of their own power and status as employed in relation to service users on an advisory group getting £20 for attending.
  • Power of academia and those in positions of power can limit how critical you can be particularly if you are seeking further work in the same setting.
• The advancement of certain progressive and life-limiting conditions, e.g. dementia, is a barrier in itself.
• Identity was discussed at length by some disabled people and service users. This emerged as a significant issue, particularly for people employed within an academic setting. One or two people had not fully identified themselves as 'disabled' or 'service user' until becoming involved in research. Another spoke of the challenge of being unable to escape the label of mental health service user, and its implications for how they were treated by other academics, once they were identified as a survivor researcher within an academic setting. Generally, people felt that some professionals did not understand the significance of identity within the research team and how personal it felt to bring your lived experience, your life even, into this formal workplace: "It's about disabled people generally but it's also about me." They described a significant emotional investment to involvement.

Example of co-production

Co-production of the National Institute for Health and Care Excellence (NICE) guidelines, nominated by Pete Fleischmann, SCIE Head of Co-production

The NICE Collaborating Centre for Social Care (NCCSC) hosted by SCIE developed 11 social care guidelines using NICE's methods and processes. SCIE was able to negotiate several adjustments to NICE's processes to improve user and carer involvement. The most important of these was to agree that up to four people who use services or carers could be part of the guideline committee which oversaw the production of the guideline and owned the recommendations. NICE guideline committees usually include two lay members, Further adjustments were that users and carers were involved in the selection of the Guideline committee chair and also in the scoping phase.

NCCSC conducted two guidelines relating to people with learning difficulties. Co-producing these guidelines with people with learning required NCCSC staff and the Co-production Team to develop several new approaches to support everyone involved:

A workshop as part of recruitment of service user and carer representatives to the committee to give candidates a full briefing about the role

Training to support all staff working on the guideline on sharing complex information with people with learning difficulties, from SCIE and People First Lambeth.

A facilitator to support people with learning difficulties on the Guideline Committees. Committee members worked before each meeting with the facilitator to understand the evidence-base. They worked out what they think is important in relation to the evidence and what they think the committee should focus on when making recommendations. They then presented their ideas at each meeting and this has also helped to inspire the rest of the committee.

As the work progressed some further adjustments to committee meetings were made. These have included producing meeting papers two weeks in advance, all papers and presentations being in an easy read format and having more frequent and longer breaks during meetings. The resulting guidelines can be found here: Learning disabilities and behaviour that challenges: service design and delivery and Care and support of people growing older with learning disabilities.

4. Co-production

We were interested to understand people's views about co-production, whether it is a term used in social care research and whether they would describe any of their research experience in these terms, rather than using the term 'public involvement'. First we explore some of the context for co-production in research.

A recent literature review (Tracey & Paylor, 2018) discussed how public involvement relates to co-production and identified the differences and similarities they share. The key differences they identified were that co-production: 

• underpins the way research is designed and conducted and is not something that can be added into the research methodology
• demands a shift in power relations and establishing an equal partnership between citizens and professionals
• focuses not on the outcomes derived from the research findings, but from the process itself and the benefits for those involved. This could include expanded social networks or the development of new skills
• involves different stakeholders in analysing the research findings, whereas, in public involvement, this can often be something that researchers control and dominate
• places an emphasis on the importance of reflecting on the knowledge production process, and the relations between participants and the research itself
• opens up possibilities of involving a wide range a stakeholders, including policymakers and practitioners.

For SCIE, there is no simple definition of co-production. They emphasise the core of co-production as being an equal partnership: 'people who use services and carers working with professionals in equal partnerships towards shared goals' (SCIE, 2015). They describe four key principles for putting co-production into action: equality, diversity, accessibility and reciprocity (SCIE Guide 51 Co-production).

SCIE believes that, for co-production to be transformative, people who use services need to be recognised as experts in their own right. There needs to be respect for the assets that everyone brings to the process and an emphasis on all the outcomes that people value, rather than just those, such as clinical (or in this case, research) outcomes that the organisation values. Real change is accompanied by a movement of resources to people who use services and to frontline staff.

INVOLVE also proposes a number of key principles for co-producing research:

• Sharing of power – the research is jointly owned and people work together to achieve a joint understanding
• Including all perspectives and skills – make sure the research team includes all those who can make a contribution
• Respecting and valuing the knowledge of all those working together on the research – everyone is of equal importance
• Reciprocity – everybody benefits from working together
• Building and maintaining relationships – an emphasis on relationships is key to sharing power. There needs to be joint understanding and consensus and clarity over roles and responsibilities. It is also important to value people and unlock their potential.

In guidance on co-producing a research project, (Hickey et al., 2018) notes that co-producing research will “usually build on existing public involvement frameworks in organisations...sometimes these involvement frameworks will provide firm foundations for building co-produced research while at other times these frameworks will require modification”. The guidance recognises that people may only want to co-produce parts of a research project.

The interviews for this piece of work revealed a wide range of views and experiences of co-production. There was considerable scepticism about the authenticity of co-production in a research context and some bad experiences were related. Equally, a few good examples of co-produced research were given. For several people it was seen is an aspiration rather than a reality. The power differentials frequently remained despite some degree of power-sharing as is required by co-production. For some interviewees, it was seen as similar to 'involvement' in that there could be different levels of co-production. However, many workshop participants believed that we should be careful to define co-production and emphasise its distinct identity in our final report. The following views were given during the course of the interviews:

• Co-production is people being able to work equally with people - all round the table, not just a couple of service users/disabled people among a group of academics and practitioners:
  • Service users have more thorough ownership of the research
  • You cannot ignore what people are saying in co-production as you might do with a consultation: you are working with people, taking them seriously.
  • There is a higher bar than for involvement or participation
• Co-production values the skills, experience, contributions of all partners - professional skills too - drawing on the skills and assets of everyone.
• The term can be used to mean partnership with any variety of partners, and certainly includes practitioners in social care research. However, this can mean that it does not always include service users or carers.
• Co-production is more common within a service context than in research.
• It is difficult to do co-production well, it is rare, it has been colonised, researchers can be bit fraudulent in their claims to be co-producing research; for example, agendas set by professional researchers, meetings spaced wide apart when most of the real work and decision making goes on in between. One participant described seeing papers written about a project they had been involved in and not recognising the project from the write-up; she felt very disillusioned.
• Co-production is inevitably compromised by the limits and constraints imposed by organisations and research funders, - e.g. employment and payment limitations in Universities.
• People have learnt to use the word to get funding; it is not as meaningful as it could be.

Examples of co-produced research

LGBTQI+ disabled people using self-directed support, nominated by PI David Abbott

This project was a partnership between the University of Bristol, SCIE, Stonewall and Regard (a national organisation of LGBTQ people who self-identify as disabled). The research was funded by NIHR SSCR. David Abbot was the PI for the research, but it was Regard and therefore LGBTQI+ disabled people who determined the need for the project and invited him to become involved as a researcher. This meant that the researchers held less power within the partnership than in a conventional involvement model - although the PI was accountable to funders.

The project was an exploration of LGBTQI+ disabled people's needs around personal assistance. DA described it as a good example of co-production, led by disabled people and with authentic relationship struggles and dilemmas. The research findings are published here.

It helped that the project partnered with SCIE, who could pay the service users and disabled people involved in the project and circumvent the problems presented by trying to get payments administered by the University. The outputs from the project were managed and produced by SCIE in collaboration with Regard and Stonewall; they are practical tools aimed at LGBTQI disabled people and their support workers.

The Right to Be Heard: Review of independent mental health advocacy services, UCLAN, funded by NIHR, moninated by Julie Ridley. Text edited from the report

this review of the quality of independent mental health advocate (IMHA) provision across England was commissioned by the Department of Health. A team of independent researches including four academics and nine co-researchers with experience of mental health services undertook the study, which lasted eighteen months from 2010. Using different methods, the experiences and viewpoints of those with experience of IMHA, advocacy services, and health and social care professionals were explored in-depth.

One of the hallmarks of this research was the partnership between academics and mental health service users. The nine co-researchers were recruited for their interest in advocacy and research, and experience of using mental health services, including detention under the MH Act, and experience of secure services. Co-researchers were given training and support and the team worked hard to be reflexive, to consider how prior viewpoints might have influenced the research process.

"It is our view that the research has been considerably enriched by service user involvement as a consequence of deeper insight into the experience of distress and detention. Furthermore, it has been strengthened by the direct accountability to service users, as most of the research team come from this background, and have consistently provided a challenge to assumptions, methods, process and analysis." p.49

An implementation project was designed with SCIE to produce practical outputs. The team wrote a book from the resaerch, also co-produced with the service user researchers. "Independent Mental Health Advocacy - the right to be heard: context, values and good practice", Jessica Kingsley, 2015.

Glasgow Centre for Independent Living: Evaluation, nominated by Gerry Zarb

Glasgow Centre for Independent Living (CIL) commissioned GZ to develop a participatory method for them to evaluate their own services and to plan new services.The approach he took was to enable them to take a lead role in the evaluation in order that, when it was finished, they would be in a position to do it themselves in the future. The project had tow guiding principles:

• Social model of disability: GZ's role was to put his research expertise at their disposal ' do I keep my research expertise to myself or do I share it around?'
• The evaluation had to contribute to the empowerment of disabled people - it would be of real material use to them

GZ set up an Evaluation Action Group, including staff and service users. They became an extended research team - designed the evaluation together as a team and carried out the data collection themselves. GZ trained people to run focus groups. They came up with some basic questions with which to interrogate the service, and then designed a focus group topic guide. The evaluation was truly collaborative at every stage; the CIL knew that they anted to do it in this way, they wanted a co-production approach.

5. Guidelines/guidance

There were divergent views about the role and value of guidelines for public involvement in social care research. Many people felt there were enough guidelines in existence already; or that the principles are the same for health and social care, it is the context and language that are different - so perhaps guidelines need to be pitched differently. For some people, 'guidance is an alternative to action' and there is no need for any new guidance. Some doubt was expressed about the potential for guidelines to make any difference if they have no status or clout. Few people interviewed were actively using guidelines, other than the INVOLVE guidance on payment. Some people said that they suggest or recommend guidelines to others, often INVOLVE (INVOLVE, 2012). These are some more of the views expressed:

• Any new guidelines must themselves be co-produced
• Guidelines are often too shiny and optimistic - what is needed are need real accounts of the journey, authentic accounts of what it is like to involve/be involved
• Relationships are key - guidelines don't solve this - need to establish and maintain relationships of trust whilst also not only using the same people every time
• Several people said they had been doing it for so long, they rarely look at guidelines. A couple of people mentioned the book 'This is Survivor Research' - edited by Sweeney, Beresford, Faulkner, Nettle and Rose, PCCS Books, 2009. Mentions were also made of Peter Beresford's work and Shaping Our Lives as sources of guidance.
• There is definitely a need for more guidelines that enable researchers to think about how to involve particular groups of people, e.g. people with dementia, people with learning disabilities: understanding how to make research accessible for people with different needs.

Conclusions

This piece of work concludes that the practice of public involvement in social care research shares much in common with health care research. Many of the elements that support good practice or act as barriers to involvement are shared across the two fields. Guidelines that address public involvement in health care research can easily be used by social care researchers. However, we heard from many people who felt that issues of equal access need a greater focus in social care research, in order to ensure that research is accessible and people can be meaningfully involved.

Social care research is the poor relation of health care research, receiving less funding and less status within the research world dominated by a focus on evidence-based practice. Few social workers become active researchers, meaning that practitioners also need to be considered as potential stakeholders in research. We heard far more about service user involvement than carer involvement.
Some of the differences lead us to highlight the importance of making available accessible practical examples of involvement in social care research and guidance on enabling equal access for people with different disabilities and needs.

Recommendations

• INVOLVE could develop a resource to direct social care researchers towards useful resources to include, for example, those provided by SCIE; and adapt their current resources to be inclusive of social care research by (for example) reviewing the terminology used.
  INVOLVE could collate practical examples of public involvement and co-production in social care research, would help to enable both researchers and service users & carers to understand what it looks like and what is possible. This project could include the use of film and other formats to ensure accessibility to a wide range of disabled people and social care service users.
• Guidance on making research accessible: The consultation suggests a need to collate and disseminate the experience and expertise enabling equal access for all of the many different groups of disabled people and people who use social care services: how to make research genuinely accessible. For example: the use of different communication formats, managing meetings, engaging people from different communities, the use of technology, using creative methods and so on.
• Co-production: NIHR INVOLVE and SCIE could usefully explore further work together to ensure that guidance on co-production is relevant to the social care research and service user communities.
• The School for Social Care Research: would be a good partner for INVOLVE to explore ways of taking these recommendations forward.
  Community engagement: INVOLVE could explore good practice in relation to community engagement, as part of making research relevant and more widely accessible to people from marginalised and seldom heard communities.
• NIHR could set aside funding to support user-led research and DPULOs: much of the genuinely innovative social care research has been undertaken by Disabled people's and user-led organisations.
• INVOLVE membership: a greater presence of people from the social care world, researchers and service users & carers, would help to build the further presence and priority given to public involvement in social care research.
• NIHR could make a statement committing to do more in terms of social care research, and with an expectation that there must be public involvement.
• Those involved in commissioning NIHR research need training/awareness raising on the issues that are particular to social care research.  This includes the funding implications of ensuring equal access for public involvement in social care research.
• Disabling barriers presented by Universities: Both SCIE and INVOLVE could review together how best to address the disabling barriers presented by Universities - perhaps by producing a discussion paper or organising a seminar.  

References

Baldwin, J. N., Napier, S., Neville, S., & Wright-St Clair, V. A. (2018). Impacts of older people’s patient and public involvement in health and social care research: a systematic review. Age and Ageing, afy092-afy092. doi: 10.1093/ageing/afy092
Beresford, P. and Carr, S., Eds. (2018) Social Policy First Hand. Bristol: Policy Press.

Beresford, P. (2013) Beyond the Usual Suspects: towards inclusive user involvement. London: Shaping Our Lives. www.shapingourlives.org.uk
Blackburn H., Hanley B. and Staley K. (2010) Turning the pyramid upside down: examples of public involvement in social care research. Eastleigh: INVOLVE.

Brett, J., Staniszewska, S., Mockford, C., Herron-Marx, S., Hughes, J., Tysall, C. and Suleman, R. (2012) Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expectations, 17,  pp.637-650.

Cossar, J. and Neil, E. (2015) Service User Involvement in Social Work Research: Learning from an Adoption Research Project. British Journal of Social Work 45, 225–240.

Hickey, G., Brearley, S., Coldham, T., Denegri, S., Green, G., Staniszewska, S., Tembo, D., Torok, K., Turner, K. (2018). Guidance on co-producing a research project. Southampton: INVOLVE.

INVOLVE. (2012). Briefing notes for researchers: public involvement in NHS, public health and social care research. Eastleigh: INVOLVE.
INVOLVE (2013) Budgeting for Involvement: Practical advice on budgeting for actively involving the public in research studies.

McLaughlin, H. (2010) ‘Keeping service user involvement in research honest’, British Journal of Social Work, 40(5), pp.1591 – 608.
NIHR Mental Health Research Network (2013) Good Practice Guidance for Involving People with Experience of Mental Health Problems in Research. London: NIHR.

Rice, C., Moule, P., Lima, M., Donskoy, A.-L., Pollard, K., & Donald, C. (2015). Developing and evaluating guidelines for patient and public involvement (PPI) in research. International journal of health care quality assurance, 28(2), 141-155. doi: 10.1108/IJHCQA-01-2014-0001
SCIE. Co-production in social care: what it is and how to do it – At a glance. SCIE At a glance 64, October 2015.

Snape, D., Kirkham, J., Preston, J., Popay, J., Britten, N., Collins, M., Froggatt K., Gibson A., Lobban F., Wyatt, K., Jacoby, A. (2014). Exploring areas of consensus and conflict around values underpinning public involvement in health and social care research: a modified Delphi study. BMJ open, 4(1), e004217.

Tracey, J., & Paylor, J. (2018). Co-producing research: what does it mean?

Turner, M. & Beresford, P. (2005) User Controlled Research: its meaning and potential Eastleigh: INVOLVE.