Dissemination Centre Rater Guidance
The role of raters
Raters play a crucial role in the work of the NIHR Dissemination Centre by helping us to decide which research abstracts are important enough to turn into NIHR Signals. A wide range of health and social care professionals, patients and carers make up our pool of raters. We have a substantial number of raters, so please don’t worry if you are invited to rate very infrequently.
When an abstract is ready to be rated, our system automatically assigns the task to five raters, of whom one is a patient or member of the public. The system finds raters based on their roles, experience and areas of interest, according to the information that people give us when they register.
When you are assigned a rating task, we will ask you to read a selected research abstract and assess its likely importance to professionals or patients with similar interests. You do not need to have a detailed technical knowledge of a clinical or research area but you do need to be familiar with, and have an interest in, the area or issue covered by the abstract.
Receiving an abstract for rating
An e-mail will be sent to each rater with a link to the abstract and a deadline date. At this point, you can decide whether or not you are willing to rate the abstract. If you are not able to complete a rating for any reason, simply click the ‘Decline’ button. You will still be taken to the rating form as above, but at the bottom of that page you can click the ‘Sorry I’m unable to rate this’ button.
If you are willing to provide a rating for the abstract that you have received, click the ‘View Abstract’ button. You will then be taken to the full abstract to review and rate. We expect your rating task, based on a quick judgement of likely importance and interest, to take around 10-20 minutes. You do not need to link to the full article; your rating is based on how you judge the abstract we have sent you.
When reading the abstract, you may find it useful to keep the following questions in mind. Please note, these are just a guide and you do not need to answer them specifically:
- Is the information in this abstract new?
- Is the evidence in this abstract important?
- Is the information in the abstract relevant to current NHS practice?
- Would a Signal based on this abstract be useful for people like you, who have an interest in this issue?
- Would you share it with colleagues?
- Would a Signal based on this abstract be useful for the people living with the condition/issue or those who care for them?
- Does the abstract show findings that are likely to lead to a change in practice?
After considering the above, please score the abstract between 1 and 6:
6 - Definitely worth turning into a Signal
5 - Worth turning into a Signal
4 - Probably worth turning into a Signal
3 - Probably not worth turning into a Signal
2 - Not worth turning into a Signal
1 - Definitely not worth turning into a Signal
We also need your comments as they are really useful in helping us to decide which abstracts to take forward as Signals. Once you have entered a score and your comments, press the ‘Submit your rating’ button. Your feedback will then be saved. You will receive an automated e-mail thanking you for your assistance.
Your scores and comments will be taken into account by the Editorial Board who will decide which abstracts to take forward. You can see the Signals as they are published each week on the Discover portal.
Do's and don'ts of commenting
- Say if you think the evidence in the abstract is credible
- Tell us if the evidence is topical
- Tell us if the technology or intervention described in the abstract is used currently in your area of practice
- Tell us if the evidence is likely to be useful to specific patient or professional groups
- Tell us if and how the evidence is likely to change practice
- Tell us if you think the evidence is already widely known or understood among people like you
- Restate the findings of the research that are set out in the abstract. What we need to know is whether the evidence is useful to people like you
- Tell us that the abstract doesn’t read well – we will solve that by writing a more readable account of the evidence in the Signal if the evidence is sound and relevant
- Confuse the abstract with any other research document! We will only send you abstracts, never research briefs or applications
What happens next?
The abstracts that you have rated are discussed at the Dissemination Centre’s next editorial team meeting. Rating scores and comments are valuable parts of the decision whether or not to develop a Signal from any given abstract. Abstracts that are selected are then developed as Signals by the writing staff and edited by the Centre’s senior team. Finally, the Signals are published on the Dissemination Centre's Discover portal.