This guidance for is public reviewers of research funding applications to: NIHR Invention for Innovation; NIHR Policy Research Programme; NIHR Programme Grants for Applied Research (PGfAR); and NIHR Research for Patient Benefit.
This guidance document aims to help you:
- understand the task of reviewing
- successfully complete a review.
What are external peer and public reviews and why are they important?
After a research team has submitted their application for funding, the application is reviewed by a variety of people, for example by scientific experts and professionals in the relevant field. This is called a peer review. Many funding organisations invite patients and members of the public to review and comment on the application before they decide whether to award funding. This is called a public review.
External peer and public reviews play a crucial role, and are a part of the process where applications are sent to independent relevant individuals for assessment. Individuals have no connection to the funding body or the applicant.
What we need from your review
Your review is written from your point of view and experience as a member of the public, patient or carer. When we ask you to think about patient and public involvement and engagement (PPIE), we are very interested in your thoughts on the following:
- Is the proposed research important and relevant to patients, service users, carers and/or the public?
- Does the research have the potential to make a sustained difference, and lead to benefits in terms of people’s wellbeing and/or the way health and care is delivered and can be reached/accessed?
- Will any of the research plans stop some people from taking part?
- Is the research plan inclusive of a diverse population, both as research participants and as patient, service user, carer and/or public contributors?
- What do you think about the plans for patient, service user, carer and/or public involvement and engagement in the research application? What difference will they make? Have they already made any difference? How could they be improved?
- Do the project management plans include patient, service user, carers and/or public representatives as appropriate?
- How good are the dissemination plans? Do they involve patient, service user, carer and/or public representatives as appropriate, and include ways to ensure the research outputs reach the target population and the general population?
- Is the plain English summary of the funding application easy to understand for a lay person? Could it be improved?
- Is the funding for patient, service user, carer, involvement and engagement sufficient to cover PPIE related activities, and PPIE representatives’ time? Does it adhere to relevant NIHR payment guidelines?
- Is this research inclusive in its design and does it include strategies to engage with under-served groups to ensure representation and equality? Whenever possible, research should take account of age, disability, sex, gender, sexual orientation, ethnicity, culture, religion and the other protected characteristics in its design, undertaking and reporting. Reviewers are advised to review whether the applicants have considered the relevance of diversity in their proposed research, if appropriate.
- How could the researchers improve their plans for PPI in their research?
- How will the researchers make sure the findings of the research reach the general population?
Remember, we are getting a range of reviewers to comment on each application, including those with professional and scientific expertise. We do not expect public reviewers to be able to comment on the scientific or technical aspects of the proposal.
Getting started
A research funding application can be under 50 to well over 100 pages in length. The application may include supporting documents, CVs, charts and letters of support.
Before you complete the online review form please read through the application.. Here are a few tips from experienced public reviewers:
- Start by reading the plain English summary; it can give you a useful overview of the application and ideas about what to look out for in the application. Then, at the end of your review, go back to the plain English summary to assess whether it is a good, clear representation of the proposed study, and could be used on its own to describe the research.
- Applications can seem large and hard to understand, but remember that we don’t expect you to be able to follow or comment on all of it. It can be helpful to break down the review task into smaller parts over a few days. This gives you thinking time too. You will need to read some or all of the application several times whilst you are completing the review. Another tip is to print or use the highlighting function in Word, or to use post-its.
- It’s impossible to say how long it takes to complete a review. Like most things, the more reviews you do the easier and quicker it gets to look at the documents and understand the research. However, some applications will just be more complex than others.
- Don’t spend hours looking for information and reading about the things that you don’t know about. We have asked you to do the review because of your personal knowledge and experience as a patient, service user or carer, but not as a professional in the field.. Take your time, but trust your instincts – you are an expert too.
- It may help you to think of yourself as a ‘critical friend’. Someone who is encouraging and supportive and who comments honestly and helpfully about problems and strengths and successes. Style and tone are important. Comments can be missed or dismissed because of an apologetic or a hostile sounding remark.
- Single word answers are not helpful. Committee members who will read and use your review don’t just want to know what you think. They also want to know the reason or reasons why you think it. For example, ‘On the basis of my experience as a carer of a person with motor neurone disease, I would suggest that ….’, or ‘In the absence of any budget for PPIE in this application it is difficult to imagine how ….’.
Completing the review form
The online form is the same for peer and public reviewers. It is separated into sections with prompts to help you complete your review. These suggestions are for guidance only. You do not have to give your views on all of them. These might help you complete the review from your perspective as a patient, service user or a carer. A template of the public review form is available to download as a Word document.
Section: Relevance of the research:
- Is the research important to, and address the needs of patients, service users, carers and the public? If so, why do you think so?
Section: Potential of the research to make a difference:
- Does the research have the potential to produce findings that can lead to a significant change, and lead to benefits for patients/service users, carers and the public. If so, in what way? This change could be on individuals, communities, recipients or providers of health and care services, decision makers as well as treatments and services themselves.
- Is it clear how many people could potentially benefit?
Section: Team:
- Does the team have the right mix of skills and experience to deliver the research?
- Are patient, service user, carer and/or public representatives included i n the team? If so, is their role (for example, as a co-applicant), and the added value they bring, clear? You may wish to refer to the ‘Public Co-applicants in Research’ guidance.
- Is there a named patient, service user, carer and/or public member lead to help support and deliver PPIE related activities?
Section: Quality of the research plan:
- Are the plans for patient, service user, carer and/or public involvement and engagement appropriate, clear, meaningful and inclusive?
- Are the resources set aside for PPIE activities sufficient? You may wish to refer to the payment guidance
- Is the research inclusive in its design, and includes strategies to engage with under-served groups to ensure representation and equality? You may wish to refer to INCLUDE
- Do the governance and management arrangements seem satisfactory, and do they include patient, service user, carer and/or public representatives as appropriate?
- Are the dissemination plans satisfactory? In particular, do they involve patient, service user, carer and/or the public representatives, and opportunities to enable people who have taken part in the study to participate in activities, and ensure the public learn about research results?
- Is the plain English summary easy to understand, clear, free from jargon, and could it be used on its own to describe the research? You may wish to refer to the plain English summary guidance for more information.
Section: What else could the applicants do to improve the research?
Here you can add additional comments about the application and how it can be improved. This section is for comments which are not covered by the sections above.
You can type your comments directly into the online form. Other people prefer to work offline and then copy and paste their comments into the form.
Once you have written your comments into each of the sections, you’ll then be asked to score the application. Please look at the scoring guide for more information. When you have completed all the sections, you will be able to submit your review.
Please note that you do not have to complete the form in one session. You can save your review and return to it later. For step-by-step guidance to completing the online form, please refer to our System Help (RMS User Guidance).
What happens next
After you have submitted your review, you will receive an email from us acknowledging receipt of it and sending you a form to complete to claim a fee. You can access a copy of your review, saved as a PDF file, in the ‘Submitted Reviews’ section of ‘My Reviews’ in your Research Management System account.
The review that you have provided, along with all other reviews, will be sent to the committee that then makes a funding recommendation about that application. We will contact you to let you know the outcome of that funding decision. Please be aware that for successful applications this could be some months after you have completed the review, as there are often many checks and conditions applied before an application becomes part of a binding legal contract.
Support available to you
If you have a problem with accessing the application or the review form online or you need any more guidance on the task, please don’t hesitate to contact us.
Fee and expenses
We offer a fee for completing a review. You will be sent a claim form after we receive your review. For more information on fees and expenses please see the NIHR’s public payment guide.
Confidentiality and conflicts of interest
Please be aware that research applications are confidential. This is because research is a competitive area and researchers want to protect their original ideas. It’s very important that you don’t discuss an application with anyone else, unless you check with us first. If you receive or print a paper copy please shred the documents when your review is completed or send them back to us. We will reimburse postage through the expense form.
Please check with us if you think you may have a conflict of interest. For example, do you know anyone on the research team personally? Have you been involved in developing this application? For more information read: Confidentiality and disclosure: A guide for applicants, reviewers and commissioning committees.
Thank you for getting involved. We very much appreciate and value the time, skills and effort that members of the public contribute to our work.