Policy Research Programme - (36-01-02) Measuring the experience of maternity service users - development of a maternity Patient Reported Experience Measure (PREM)

Update 28 June 2023: Supplementary clarifications on the research specification have been provided. Please see end of this Research Specification

Timetable and Budget

Description	Deadline/Limit
Deadline for Stage 1 Applications	11 July 2023, 1 PM
Notification of outcome of Stage 1 Application	October 2023
Deadline for Stage 2 application	28 November 2023, 1 PM
Notification of outcome of Stage 2 Application	April-May 2024
Project Start	July 2024
Project Duration	18-24 Months
Budget	£350,000

Introduction 

The National Institute for Health and Care Research (NIHR) Policy Research Programme (PRP) invites applications for the development of a short-item Patient Reported Experience Measure (PREM), and to test the feasibility of utilising this PREM to assess the quality and experiences of care provided for women using maternity services. 

Background 

The Maternity Transformation Programme (MTP) is supporting a national maternity safety strategy to halve stillbirth rates, maternal mortality, and preterm birth by 2025, to address inequities in maternity care, and to facilitate personalised care within maternity and neonatal systems. This extensive programme has many elements, each focused on improving both clinical outcomes and the experiences of those that use maternity and neonatal services in England.

Many reviews of maternity care have highlighted problems with assessing the service user experience (Ockenden First Report, Ockenden Final Report, East Kent Review). The experience of those women and families whose infants are admitted to neonatal services was also recognised in the NHS England Implementing the Recommendations of the Neonatal Critical Care Transformation Review (NCCR). The evidence of better outcomes for babies whose parents are able to play a role in their infants’ care was central to the review, and enhancing the family experience is one of the three key themes within the NCCR.

Women’s experiences of maternity and neonatal care may have a lifelong impact for women and their families, and in addition inequalities exist in the access, outcomes and experiences of women and their families during pregnancy, postnatal and neonatal care for reasons including multiple deprivation, asylum seeker status, and ethnicity.

Improvements to clinical outcomes are measured and reported through existing national reporting systems, including the Maternity Services Data Set and the Neonatal Audit programme. However, experience of pregnancy, childbirth and the postpartum period is multifactorial and subjective, and in addition many factors affect a woman’s agency in accessing care, and/or communicating her needs, and getting a response. 

Currently, if policy and service leads want to get an overall picture of how women experience their care they have three main sources of information: 

1. assessing their formal complaints (which limits knowledge to those who (i) have a very negative experience and (ii) have capacity and language skills to complain)
2. use of the friends and family test which is a crude assessment with limited value
3. the Care Quality Commission (CQC) Survey, which is a point in time snapshot. It has a delay between service provision and feedback, leaving service leads unclear which problems have been addressed already and which require focus to inform improvements within maternity and neonatal services.
   

These questionnaires allow performance to be compared across trusts, and over time the organisations can also use the data to support local quality improvement, however these measures do not include everyone accessing maternity and neonatal care, or women who have suffered a perinatal loss, and there is a need to better capture experiences of neonatal care.

The Delivery Plan for Maternity and Neonatal Services aims to listen to women and families with compassion, which includes an ambition to reduce inequalities in experience and outcomes, and that women: experience care that is always kind and compassionate, are listened and responded to; that families who experience harm or raise concerns about their care are responded to with a kind and compassionate process; have clear choices, supported by unbiased information and evidence-based guidelines; experience personalised, joined-up, high-quality care throughout their care pathway; are provided with practical support and information; are partners in their baby’s care in the neonatal unit, and that bereaved families receive compassionate and high-quality care. There is an intent in the Delivery Plan to create a patient reported experience measure (PREM) by 2025 to help trusts and ICBs monitor and improve personalised care.

Current options to assess user experience of care include qualitative methodologies, which are useful for depth but costly, time intensive and create difficulties in generalisability or specifically designed surveys for a particular programme or intervention of study, which require repetitive survey designs each requiring methodological and statistical expertise. However, to date there are no tools to measure this experience, and therefore it is difficult to assess the impact of programmes[1].

There is a policy and clinical need for a generic tool to provide timely, robust, feedback from users of maternity and neonatal services following exposure to either a novel type of care (research and policy utility) or during standard care when leads wish to understand how users experience their service. A nationally (or internationally) accepted measure of experience would facilitate comparisons with different models of care, provide baselines to identify if an intervention made a difference, and highlight where focus should be given to improve services.

Patient centred healthcare is the cornerstone of many healthcare strategies. Patient specific health needs should be at the forefront of healthcare improvements and quality measurements. Patient reported outcome measures (PROM) that support real world clinical effectiveness assessments are increasingly used to highlight domains where there is the greatest scope for change [2] . A Patient Reported Outcome Measure (PROM) can be used to measure quality of care and provide new information on the impact treatment or interventions have on a patient's self-assessed health and health-related quality of life. Importantly, a Patient Reported Experience Measure (PREM) measures the experience of care.

To date, there is no PREM agreed which would be suitable as a patient reported outcome measure for the assessment of the quality of care women receive during pregnancy, birth, the postpartum period, or if an infant is admitted to a neonatal unit. However, there are a variety of available assessment tools which could potentially be helpful in developing new and existing PREMs for perinatal care [3].

The planned system transformation targeting personalisation of care, and each of the other maternity transformation programmes each require an assessment of the experience of service users, and each programme has expressed a desire for a PREM which could be used at a service level or aggregate level to inform service improvement at a local level and success of policy and programme delivery. There is currently no way that policy makers or service managers can uniformly assess maternity and neonatal user experience, thereby creating issues with comparing programmes, or measuring improvement when there is no consistent way of measuring the baseline.

There is therefore a need for robust research to design and validate a PREM for use with women who experience maternity and neonatal services in the UK, to inform our policy objective of personalised and safe care across perinatal services. 

Research priorities

It is important to develop a tool that can assess dimensions of the experience of perinatal service users that is important to them, is relevant in a variety of conditions, including when assessing an intervention or policy directive (eg a new model of care, a new technology) researching maternity or neonatal issues (eg a new drug or treatment) and for generic clinical care (eg to measure improvements over time or to identify problem areas of care).

The proposed research would need to draw on a systematic review that synthesises evidence on the use of PREMs and validate their use in a variety of populations. Other examples in both primary and secondary care can be used as a base, but maternity and neonatal specific questions may need to be developed because of the unique situation that pregnancy presents. This research would build upon some in-depth qualitative work commissioned by the programme to understand what is important to women and families from the widest range of demographics, experiences and pathways during pregnancy, birth, neonatal care, and the postnatal period. An important component of this work will be to ensure that there is an understanding of what is important to women from a wide range of ethnic and cultural backgrounds, and those who find services hard to reach. Maternity and Neonatal Voices Partnerships and Local Maternity and Neonatal Systems across England hold a wealth of qualitative and quantitative data which should be considered during this research. 

Research questions may include: 

Can a short item PREM be developed for maternity and neonatal service users that explores key areas identified by service users as important to them. Possible areas include:

1. Personalised care (does the person feel the care was adapted to their/their babies needs?)
2. Respectful care (does the person feel the care respected their choices?)
3. Feeling safe (does the person feel physically and psychologically safe and secure/ that their infant is safe and secure?)
4. Informed decision making (was the individual made aware of the advantages and disadvantages of care options in a format/language they can understand before making choices?)
5. Informed consent (does the person feel they could decline treatment?)
6. Responsiveness (does the person feel their views and concerns have been listened to and responded to?)
7. Coordinated care (does the person feel their care is planned across organisations and professionals?)
8. Access (does the person find services easy to reach?)
9. Can this tool have the appropriate comprehension level for the vast majority of maternity and neonatal service users, including easily translatable language for non-English speakers and available in formats appropriate for non-literate service users?
10. Is the tool both ‘valid’ (measures what it is supposed to) and ‘acceptable’ (service users are happy to use, and policy makers/researchers/clinical leads are happy with the data it provides)?
11. Which is the preferred way to complete the designed survey (electronic versus postal versus in person?
12. How will service details be captured to contextualise the PREM. How do differing service offers impact upon experience?
13. Can the tool be used at multiple time points, and significant time points?
14. Clarify who would administer the tool and use the data? Who would own the data from the tool in use? Who would get to see/analyse it? How will we ensure that MNVPs see it/use it?
15. How can the tool be responsive to changes over time?  Can questions be changed? Where and how will the data be shared- so that it is publicly available?
16. Developing a tool user guide to accompany the tool.
17. Testing the feasibility of delivery methods (ie digital modes) to ensure women from all groups are able to access and respond.

Further Information following Applicant Queries:

1. The in-depth qualitative work project referred to above has already been commissioned and is underway. It is anticipated that it will be complete by March 2024 when findings will be shared with the successful research team.
2. The Policy team would like to further add that there are two key populations of women we wished to make sure are included. The first is women who have a baby in neonatal care, and the second group is women who have lost a baby. It is implicit in the specification that no group of women should be excluded, but they wanted to emphasise these two group should be included.
   
   
3. Question: Are you more concerned with achieving a single, simple set of questions that can be used across all settings (antenatal, delivery, post-natal and neonatal) with a possible loss of sensitivity to change (because to be applicable across a wide variety of settings questions have to be more generic). Or do you want to prioritise sensitivity (meaning that different questions might need to be asked in different settings and therefore the scores are not comparable)?
   Answer: We would prefer to have questions that can be asked and compared across different parts of the maternity pathway (antenatal, intrapartum, postnatal, neonatal).
   
   
4. Question: The team fully understands the requirement for an instrument that is methodologically robust in many languages and for a diverse range of cultural groups. However, as the issues of ensuring linguistic, conceptual and psychometric equivalence are non-trivial we wondered if you could advise on priority setting here?
   Answer: The PREM needs to be accessible for as many maternity service users as possible. We would like the research to explore how best to achieve this.

New Guidance on Health Inequalities data collection within NIHR PRP Research

Inequities and inequalities in the provision, access to and experience of health and care services is a high priority area within the Department of Health and Social Care and the NIHR and is present in the majority of funded projects. We are now assessing all NIHR research proposals in relation to health and care inequalities. NIHR asks researchers to explore issues of inequality and inequity in the research being proposed. 

Our goal is also to facilitate more widespread and consistent reporting of data on health and care inequalities relating to the primary outcomes of NIHR funded research. Applicants should, if feasible, collect or analyse data related to health and/or care inequalities, we request that this stated in the proposal. This is a new request from the NIHR PRP and we will be continuing to monitor queries and make any necessary clarifications, as needed.

We understand that research projects may employ different methodologies, and focus on different populations. We ask that you please clearly identify in the research plan section of the application whether your application has a health and care inequalities component. If the project uses quantitative methods, please include detail of the core set of data that will be collected or analysed. Submission of the data collection will be a condition of final reporting that will need to be submitted to NIHR PRP when the grant has finished. If a health and/or care inequalities component is not included, please explain clearly why this does not fit within your research.

Specifically for this project, it may be useful to consider whether any particular groups are particularly impacted by discharge delays, and whether interventions have any differential impact for different groups. This may include consideration of local / regional variation, deprivation or characteristics covered by the Equality Act.

A recording of the Health Inequalities in NIHR PRP Research Q&A Event which was held on 19 September 2022 is available online, this may be useful to refer to as it provides additional information.

Further details about this new request can be found in Appendix A.

Eligibility

Eligibility for the NIHR PRP is laid out in our Standard Information for Applicants and applies to all calls unless otherwise stated in the individual research specification.  

Expertise required 

• Methodological experience of the development and assessment of the implementation of  PREMS in health care. This would include qualitative and quantitative methodologies.
• Evidence synthesis.
• Maternity and neonatal services research.
• Service user experience.
• Experience in co-production.
• Experience in research with a range of populations who find it hard to engage with health services and with a range of cultural and non-English language backgrounds. 

Outputs 

• An assessment of what is important to women from a wide range of ethnic and cultural backgrounds and those who find services hard to reach, and who have different experiences of maternity and neonatal services.
  
• A review of what is already known regarding the availability of PREMS and their use in maternity and neonatal services.
  
• A short item PREM that has been tested and ready to be implemented across NHS England maternity programmes.
• Applicants are asked to consider the timing and nature of deliverables in their proposals. Policymakers will need research evidence to meet key policy decisions and timescales, so resource needs to be flexible to meet these needs. A meeting to discuss policy needs with DHSC/NHSE officials will be convened as a matter of priority following contracting. 

Budget and duration 

• Applications will be assessed on value for money, we would not expect them to cost more than £350K.
  
• The duration of the project should be as short as is consistent with delivering a high-quality study.
  
• Costings can include up to 100% full economic costing (FEC) but should exclude output VAT. Applicants are advised that value for money is one of the key criteria that peer reviewers and commissioning panel members will assess applications against.

Management arrangements

A research advisory group including, but not limited to, representatives of DHSC, other stakeholders (Royal College of Obstetricians and Gynaeologists. Royal College of Midwives, Royal College of Paediatrics and Child Health, British Association of Perinatal Medicine, maternal mental health alliance, Birth Companions, National Childbirth Trust, Birthrights, SANDS, Tommys, specific organisations representing ethnic minority group and excluded groups such as travellers) and the successful applicants for the research should be established. The advisory group will provide guidance, meeting regularly over the lifetime of the research. The successful applicants should be prepared to review research objectives with the advisory group, and to share emerging findings on an ongoing basis. 

They will be expected to:

• Provide regular feedback on progress
• Produce timely reports to the advisory group
• Produce a final report for sign off
  

Research contractors will be expected to work with nominated officials in DHSC, its partners and the NIHR. Key documents including, for example, research protocols, research instruments, reports and publications must be provided to DHSC in draft form allowing sufficient time for review. 

References and key documents

1. Mahmud, A., Morris, E., Johnson, S., & Ismail, K. M. (2014). Developing core patient-reported outcomes in maternity: PRO-maternity. BJOG: An International Journal of Obstetrics & Gynaecology, 121, 15-19.
2. O'Byrne, Laura J., et al. "Patient reported outcome measures evaluating postpartum maternal health and wellbeing: a systematic review and evaluation of measurement properties." American Journal of Obstetrics & Gynecology MFM (2022): 100743.
3. Dickinson, F., McCauley, M., Smith, H., & van den Broek, N. (2019). Patient reported outcome measures for use in pregnancy and childbirth: a systematic review. BMC pregnancy and childbirth, 19(1), 1- .
4. Wolff, A. C., Dresselhuis, A., Hejazi, S., Dixon, D., Gibson, D., Howard, A. F., ... & Edwards, L. (2021). Healthcare Providers’ Implementation of Patient-Centred Outcome Measures (PROMs) and Patient-Reported Experience Measures (PREMs) in Practice A Protocol for a Mixed Method Systematic Review.
5. Greenhalgh, J., Gooding, K., Gibbons, E., Dalkin, S., Wright, J., Valderas, J., & Black, N. (2018). How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. Journal of patient-reported outcomes, 2(1), 1-28.

Appendix A: Further Detail on the New Guidance on Health Inequalities data collection within NIHR PRP Research

Health Inequalities is a high priority area within the Department of Health and Social Care and the NIHR and is often present in a majority of funded projects. We are now assessing all NIHR research proposals in relation to health inequalities. We are asking applicants to identify in their application whether or not there is a health inequalities component or theme and how this research hopes to impact health inequalities. We are also asking researchers to collect relevant data related to health inequalities, if appropriate for the research. Collecting specific information about health inequalities in research submitted to the programme will allow for categorisation of health inequalities research, curation of data to aid future health inequalities research and enable policymakers to better understand the implications of health inequalities within their policy areas. This is a new request from the NIHR PRP and we will be continuing to monitor queries and adapt the process as needed. If you have any feedback on this new request, please contact us at prp@nihr.ac.uk. 

Our goal is to facilitate more widespread and consistent reporting of health inequality breakdown data relating to the primary outcomes of NIHR funded research. We would ideally like researchers to focus on the following equity-relevant variables: age, sex, gender, disability, region*, 5 ONS Ethnic groups**, and the 5 IMD quintile groups. These variables are considered an ideal, but we understand that these are subject to change depending on the sample population and specific research question.   

For qualitative research projects, this can be purely baseline characteristics of the participants, for example, the number of participants in each ethnic group.  

For quantitative research projects, if there are multiple outcomes/effects with your stakeholders, select a small number of main outcomes as appropriate to report equity breakdowns. We will not be prescriptive about the number of the outcomes, as it will depend on the number of study design types and the nature of the project aims. We are asking for one way cross tabulations of each primary outcome by these equity-relevant variables, if appropriate for your research, together with the number of observations in each cell. If more detailed cross tabulations are appropriate for your proposed research, please include these as well. This request applies to both primary data collection studies and secondary analysis of routine data, and to causal inference studies as well as descriptive studies; however, if this is not possible due to data limitations then please explain. Due to sample size and other data limitations there may be difficult scientific and/or data security*** judgement calls to make about which breakdowns to report and whether to merge categories to increase counts in particular cells; we ask you to make these judgments yourself, bearing in mind our data curation aim of enabling future evidence synthesis work in pooling results from different studies. We also ask that researchers report breakdowns for the unadjusted as well as adjusted outcomes/effects, as appropriate.

We understand that research projects may employ different methodologies, and focus on different populations. Please explain how the variables and data collection methods chosen are appropriate to the methodologies used. 

We ask that you please clearly identify in the research plan section of the application whether your application has a health inequalities component or not and detail the core set of health inequality breakdown data that will be collected, if applicable. Submission of the data collection will be a condition of the final report for all research with relevant methodologies regardless of whether the research has a health inequalities component that will need to be submitted to NIHR PRP when the grant has finished. This should only take a few sentences within the research plan section.  

* Table below uses the nine regions in England, further regions can be used if using the UK as the study population. Please report region breakdown for large samples in nationally representative descriptive studies. There is no need to report this for small sample studies, for sub-national studies, or for quasi-experimental studies where it would require time-consuming re-estimation.

** White, Mixed/ Multiple ethnic groups, Asian/ Asian British, Black/ African/ Caribbean/ Black British, Other ethnic group. If the sample size is small then it is fine to report only some of the requested equity breakdowns and to merge some of the sub-groups as appropriate.

*** For guidance on how to handle data security concerns in reporting of sensitive data please see ONS guidance.

Example data table for submission at the end of the funded research project 

(N.B. If there is more than one main outcome then you will require more tables and if you adjust your outcome then you will need two tables for the adjusted outcome and unadjusted outcome. For other methodologies, variable vs number of observations may be more appropriate to record participant data). This table is for an example only. It does not contain sub variables and does not illustrate any preference for certain variables, as these will be dependent on the proposed research.

Variable	Outcome (an appropriate average for this subgroup, usually the mean)	Number of observations	Additional information about variation if appropriate, e.g. range, standard deviation
Age	0	0	Additional information
Sex	0	0	Additional information
Gender	0	0	Additional information
Disability	0	0	Additional information
Ethnic Group	0	0	Additional information
IMD Group	0	0	Additional information
Region	0	0	Additional information