Artificial Intelligence in Health and Care Award - Guidance for Competition 3 All Phases

Published: 30 October 2020

Version: 2.0 - June 2021

Please be aware that guidance may change for future competitions

In August 2019 the Secretary of State for Health and Social Care announced the formation of an NHS Artificial Intelligence (AI) Lab to develop and adopt the technologies that are most promising for health and social care.

AI has the potential to make a significant difference to health and care. The NHS AI Lab has been established to ensure the NHS is harnessing these benefits in a safe and ethical fashion that is supported by patients, the public and clinicians. Given the ethical and safety concerns associated with the use of AI in health and care, the AI Lab has aligned to the principles of the NHS Constitution, addressing transparency, safety and privacy by building on the foundations already laid out, for example in the NHS’ Code of Conduct for Data-Driven Health and Care Technologies.

The AI Lab addresses barriers to adoption and development of AI, including an AI SWAT team, Skunkworks, Regulation Incubator, the Accelerating of Diseases programme, the Disease Clusters AI programme and an AI in Health and Care Award (AI Award).

The AI Award will accelerate the testing and evaluation of the most promising AI technologies that meet the strategic aims set out in NHS Long Term Plan. The AI Award will support technologies across the spectrum of development, from initial feasibility to evaluation within clinical pathways in the NHS and social care settings, to the point that they could be nationally commissioned.

The AI Award is delivered by Accelerated Access Collaborative (AAC), NIHR and NHS alongside relevant partners. The AI Award is run through an open competitive process.

Applications of AI in health and care

There is no single, universally agreed definition of AI, nor indeed of ‘intelligence’. NIHR has published a description of how the AI Award defines AI.

AI technologies funded through the award can have a variety of applications in health and social care (see applications funded in Competition 1 and Competition 2), and may come under the broad categories of ‘Health promotion and prevention’, ‘Diagnosis & treatment’ and ‘System efficiency’.

Category	Examples of areas in which AI solutions could be used
Health promotion and prevention	Digital epidemiology and disease surveillance National screening programmes Preventative advice Self-management
Diagnosis and treatment	Symptoms checkers and decision support for differential diagnosis Risk stratification Prediction of deterioration Personalised treatments
System efficiency	Optimisation of care pathways Identification of resource requirements Electronic roster system Natural Language Processing for administrative tasks

Phases of the AI Award

The AI Award supports innovators and technologies across the spectrum of development, from concept through to initial NHS adoption and testing of the AI technology within clinical pathways.

Following the success of the first two rounds of the AI in Health and Care Award, we will now be focusing on later stage applications.

This is in line with the AI Award’s aim of supporting a greater number of early-stage technologies in the first rounds of the Award and then to progress to scaling up more technologies into the NHS and care systems in later rounds.

Through Competition 1 and 2, we have already supported 29 Phase 1 technologies (subject to contract). We hope to fund some of the successful Phase 1 winners at more advanced phases in Competitions 3 and 4.

Competition 3 will be open to applications to Phase 2, 3 and 4. You may only apply to one phase per product.

Phases 1 and 2 of the AI Award aim to create a pipeline of products, interventions and services which are ready for real world testing. Phase 2 Awards will support product or technology development, first-in-man and clinical feasibility studies, and pivotal clinical studies to evaluate the safety and efficacy for the intended use. This early stage pipeline is managed by the NIHR Innovations Programme Management Office in close collaboration with the AAC Delivery Team.

Phases 3 and 4 of the AI Award aim to get products or services to a position where they can enter and be used within the NHS or social care setting. These awards will support hybrid clinical trials to generate real-world evidence, health economics evaluations, and initial system adoption and implementation studies to discover and address barriers. The AAC will provide additional support to Phase 3 and 4 award teams to facilitate the delivery of these activities within clinical or operational pathways.

Phase	Phase 1	Phase 2	Phase 3	Phase 4	Phase 5
Partner	NIHR	NIHR	NHSX/AAC	NHSX/AAC	-
Development stage	Feasibility	Development & clinical evaluation	Real World Testing	Initial Health System Adoption	National Scale-up
Funding objective	To show product and clinical feasibility of the proposed concept, product or service	To develop prototypes and generate early clinical safety/efficacy data towards CE/UKCA marking	First real-world testing in health and social care settings to develop evidence of efficacy and preliminary proof of effectiveness, including evidence for routes to implementation to enable more rapid adoption	To facilitate initial systems adoption of the AI technologies with market authorisation into the NHS, evaluating the AI technology within clinical or operational pathways to determine efficacy or accuracy, and clinical and economic impact	To address barriers to adoption into routine care for NICE-approved products with proven health system benefits, in order to facilitate rapid uptake nationally
-	Not eligible for Competition 3	12-36 months Funding uncapped but typically range £500k-£1.5m	12-36 months Funding uncapped but typically range £500k-£1.5m	12-36 months Funding uncapped but typically range £1m -£7m	Not eligible for research and development funding under this programme

The three phases are intended to fund the following development stages:

Phase 2 is intended to develop and evaluate prototypes of demonstration units and generate early clinical safety and efficacy data. Award amounts are uncapped (any amount can be requested but must be reasonably justified; indicative range ~£500k-1.5m) and are for 12-36 months.
Phase 3 is intended to support first real-world testing in health and social care settings to develop further evidence of efficacy and preliminary proof of effectiveness, including evidence for routes to implementation to enable more rapid adoption. Awards are uncapped (any amount can be requested but must be reasonably justified; indicative range ~£500k-1.5m) and are for 12-36 months.
Phase 4 is intended to identify medium stage AI technologies that have market authorisation but insufficient evidence to merit large-scale commissioning or deployment. Award amounts are uncapped (any amount can be requested but must be reasonably justified; indicative range ~£1-7m depending on number of sites) and are for 12-36 months.

Application and assessment process

Guidance for completing the application form is available on the NIHR website, along with specific finance guidance.

AI Award applications should clearly describe the AI technology proposed, and explain how it meets an evidenced need. A strong case should be made which explains how the AI technology will address the strategic priorities of the health and social care systems in England (e.g. NHS Long Term Plan, NHSX strategic priorities and/or wider government priorities including the Industrial Strategy grand challenges or resource efficiency). If the AI technology is in current use, it is advisable to get support or advice from the NHS or social care organisations using your AI technology when completing the application form.

Applications that do not meet the entry or exit criteria for the selected phase will be rejected. Applications which are ready for national spread and adoption are not eligible for funding through the AI Award. Applications to Phase 1 are not eligible for Competition 3.

The following section of this guidance document describes the application and assessment process. All applications must be made through the online application portal.

Overview of the application process

The application process is run in two stages:

Stage 1 applications must meet all award specifications, funding prerequisites and phase appropriate entry/exit criteria to be considered for review. Applications that do not meet these essential requirements will be rejected at this stage. Eligible applications will be reviewed against the AI Award assessment criteria. Selected applications will be shortlisted and invited to submit a Stage 2 application. Phase 3 and 4 applicants are particularly welcomed if submitting proposals within the following focus areas:

Self-management of long-term conditions

Preventative healthcare and long-term conditions
Reduced hospitalisation, accident and emergency attendances

Diagnostic support

Reducing elective care backlog
Symptoms checkers and decision support for differential diagnosis
Prediction of deterioration

Improving operational/system efficiency

COVID workforce issues
Optimisation of care pathways
Identification of resource requirements
Electronic roster system
Natural Language Processing for administrative tasks

Supporting elective recovery

Prioritisation of patient waiting list and critical procedures
Risk stratification of waiting lists
Elective care pathways

All applicants will be notified of the outcome but due to the volume of applications, we may not be able to provide feedback to unsuccessful applicants at this stage.

A Stage 2 application form must be submitted which will be subject to external peer and public review as well as expert and public panel review. Applicants will be invited to present their project and respond to questions at the Stage 2 panel meeting.

Award specifications

Specification	Phase 2	Phase 3	Phase 4
Development stage	Development and clinical evaluation	Real World Testing	Initial Health System Adoption
Location of lead organisation	Within the UK	Worldwide (provided you have a UK registered office or a UK health or social care organisation as a co-lead)
Location of partners	Can be outside the UK provided there is reasonable justification and a clear trajectory to NHS benefit
Eligible organisation types	Higher Education Institutions (HEIs) Small and Medium-sized Enterprises (SMEs) NHS and social care organisations Charities Local Authorities	Higher Education Institutions (HEIs) Small and Medium-sized Enterprises (SMEs) NHS and social care organisations Charities Local Authorities Large enterprises
Collaborators	Minimum of two different organisations types	Minimum of two different organisation types, one must be NHS or social care	3 or more NHS or social care adoption sites
Funding limit	Uncapped but typically range £500k-£1.5m	Uncapped but typically range £500k-1.5m	Uncapped but typically range £1-7m
Project Duration	12-36 months	12-36 months	12-36 months

Funding prerequisites

The following funding prerequisites apply to all applications and will be considered by the funding panel:

The AI technology utilises artificial intelligence to address a need or problem facing the NHS in a priority area, which may include those identified in the NHS Long-Term Plan.
The AI technology has the potential for routine use in health and/or social care in England as demonstrated by a clear route to market and ability to scale-up.
Sufficient evidence that the AI solution can meet at least one of the following criteria at a level appropriate to the stage of development:
- Improvement in patient and/or service user outcomes
- Improvement in patient and/or service user experience
- Improvement in operational efficiency
A commitment to involving members of the public and patients in the design and management of the research, evaluation or study.
Ability to demonstrate rights to access and use the data as required to deliver the proposed project, including but not limited to any clinical/patient data needed to train or validate any models or algorithms.
Commitment to relevant standards: Where appropriate, these will include the AI Code of Conduct for data-driven health and care technology (for artificial intelligence systems used by the NHS), the NICE Evidence Standards Framework for digital health technologies, the NHS Digital Standards for commissioning or developing Personal Health Records
Ability to demonstrate interoperability with existing NHS systems or a commitment to work towards and fund any relevant product development required to achieve interoperability. This includes ensuring the AI solution is vendor neutral.
Relevant approvals in place or working towards relevant approvals:
- Regulatory, intellectual property protection, ethical framework or any other relevant approvals
- Conformité Européene (CE) or UK Conformity Assessed (UKCA) marking and/or market approvals
- Demonstrate Information Governance (IG) compliance in line with General Data Protection Regulation (GDPR)
- Not subject to any Medicines and Healthcare products Regulatory Agency (MHRA) safety alerts

(Further information on UK product marking from 1 January 2021 can be found on the gov.uk website)

Phases entry and exit points

Point	Phase 2	Phase 3	Phase 4
Development stage	Development & clinical evaluation	Real World Testing	Initial Health System Adoption
Entry	Prototype /MVP Pre-clinical efficacy data (internally / lab validated)	Externally / clinically validated Safety demonstrated CE/UKCA marked or close	RW efficacy demonstrated, with prospective evidence generated from at least two implementation sites CE/UKCA marked Regulatory approvals in place Market authorised Robust health economics
Exit	Clinical efficacy data Safety data Regulatory data CE/UKCA marking Health economics	RW implementation feasibility Prospective clinical efficacy data from more than one site Regulatory approvals Market authorisation Full health economics assessment	Feasibility for large scale deployment RW efficacy demonstrated in multiple sites RW health economic evidence

Fundable activities for each phase

Phase 2 - Development & clinical evaluation

The objective of Phase 2 projects is to develop and evaluate prototypes and generate early clinical safety and efficacy data towards CE/UKCA marking.

The entry point (or the minimum requirement on eligibility) for Phase 2 projects is usually a technology validated in a lab (TRL4)*. As an example, projects may focus on AI innovations that require a second wave of product development or have not been clinically tested before and now require clinical validation including a number of training and testing iterations in order to achieve the finished product.

Fundable activities include, but are not limited to:

Research and development to enable clinical use
Acquisition of big data through technology development or by acquiring existing clinical datasets for the purpose of training the technology
Studies to provide data relating to the safety and efficacy of the technology including small-scale randomised controlled trials
Interventional studies that may demonstrate non-inferiority/equivalence in comparison to a standard of care or an existing benchmark
Large scale randomised controlled trials for marketing approval and to facilitate identification of any adverse effects due to population-based generalisations
Usability trials to determine wider clinical acceptability and tolerance and where applicable patient-reported experiences
CE/UKCA marking and other regulatory requirements, including any associated preparation for a future clinical trial application
Activities in relation to intellectual property protection, freedom to operate and market analysis or business case development, including plans for commercialisation and NHS adoption
Health economic analyses including cost-effectiveness or budget impact analyses to ascertain value for money in accordance with the NICE Evidence Framework (Section B)
Health economic modelling to forecast the long-term health benefits as a result of technology use
Activities associated with the dissemination of outputs
Research to support patient and public engagement or involvement
Identification of relevant stakeholders and/or partners to support the next stages in the development

Please see What we do not fund for activities we do not fund.

*Technology Readiness Levels as defined by the European Commission for public sector innovation

Phase 3 - Real world testing

The objective of Phase 3 is to carry out the first real world testing of an innovation in health or social care settings to develop evidence of effectiveness in practice, including evidence for routes to operational implementation to enable more rapid uptake.

The entry point (or the minimum requirement on eligibility) for Phase 3 is usually a technology that has been validated in a relevant environment (TRL5). As an example, this may be an AI product that has been tested in a clinical or social care setting, has generated a significant amount of safety and efficacy data and is close to CE/UKCA marking or newly CE/UKCA-marked and requires further evidence to demonstrate effective implementation of the technology in clinical practice.

Fundable activities include, but are not limited to:

Activities associated with the design and delivery of evaluations of AI solutions in health and social care settings, including any trial methodology aimed at demonstrating the clinical utility of the product with respect to its real-life implementation and use:
- Prospective studies that evaluate the ability to estimate outcomes where there is typically no suspected risk
- Qualitative research on setting-specific aspects including readiness to cultural change, clinical adaptation (IT, information governance and data infrastructure) and long-term adherence to the technology
- Technology pilots to determine the impact on service delivery and management and/or to support re-design of the clinical pathway
Collection of efficacy data if this is part of a clinical utility study
Small changes to the technology that might be needed for its optimisation during the lifetime of the project, e.g. any changes deemed required for end user acceptance as part of this evaluation but not requiring any further regulatory approvals
Activities associated with the data analysis, management and governance of the real-world evaluations
Development of rich and in-depth case studies conducted in single or multiple NHS sites that may serve as adoption exemplars and reference sites
Costs associated with implementation research, including the design of the implementation strategy, clinical pathway analysis and sustainability evaluation
Training associated with the implementation of new technology, including the development of training resources and materials
Health economic analyses including cost-effectiveness, cost-utility or cost-benefit analyses to determine any cash savings or cash releases
Activities in relation to business development, market analysis and development of a case for adoption
Activities associated with the dissemination of outputs
Research to support patient and public engagement or involvement

Please see What we do not fund for activities we do not fund.

Phase 4 - Health system adoption

The objective of Phase 4 is to demonstrate clinical and economic impact of promising products in the NHS and/or social care setting to inform reimbursement and procurement decisions and facilitate systems adoption.

The entry point (or the minimum requirement on eligibility) for Phase 4 is CE/UKCA-marked AI products with clearly identified NHS and/or social care benefits derived from a robust health economic assessment or NICE appraisal (TRL6-7).

Phase 4 is intended to identify medium stage AI technologies that have market authorisation but insufficient evidence to merit large-scale commissioning or deployment. Grants are uncapped and funding awards are per technology. The AAC Delivery team will work with NHS sites to support their adoption of these technologies, to stress test and evaluate the AI technology within routine clinical or operational pathways to determine efficacy or accuracy, and clinical and economic impact.

Fundable activities include, but are not limited to:

Activities associated with the design and delivery of evaluations of AI solutions across multiple health and social care settings, including any trial methodology aimed at demonstrating real world evidence of clinical or economic utility of the product with respect to its real-life implementation and use:
- Prospective studies that evaluate the ability to estimate outcomes where there is typically no suspected risk
- Qualitative research on setting-specific aspects regarding scaling the technology including readiness to cultural change, clinical adaptation (IT, information governance and data infrastructure) and long-term adherence to the technology
Small changes to the technology that might be needed for its optimisation during the lifetime of the project, e.g. any changes deemed required for end user acceptance as part of this evaluation but not requiring any further regulatory approvals
Activities associated with the data analysis, management and governance of the real-world evaluations
Development of research protocol and ethics
Development of rich and in-depth case studies conducted in multiple NHS sites that may serve as adoption exemplars and reference sites
Costs associated with implementation research, including the design of the implementation strategy, clinical pathway analysis and sustainability evaluation
Training associated with the implementation of new technology, including the development of training resources and materials
Health economic analyses including cost-effectiveness, cost-utility or cost-benefit analyses to determine any cash savings or cash releases
Activities in relation to business development, market analysis and development of a case for adoption
Activities associated with the dissemination of outputs
Research to support patient and public engagement or involvement

What we do not fund

The following activities will not be considered for funding under this competition:

Basic science (TRL1 space)
R&D around drug discovery and development

Assessment process

Step	Process	What's involved
1	Screening for prerequisites	Stage 1 applications will be screened against award specifications.
2	Assessment of Stage 1 applications	Proposals that meet the award specifications will be assessed against the Assessment criteria by internal and expert reviewers. A selection of applications will be shortlisted and applicants will be invited to submit a Stage 2 application. All applicants will be notified of the outcome. Due to the volume of applications we may not be able to provide feedback to unsuccessful applicants at this stage.
3	Peer review and panel assessment of Stage 2 applications	Stage 2 applications will be subject to independent peer review as well as panel review. Applicants will be invited to present their project at the panel meeting, followed by a Q&A session.
4	Ratification	Panel outcomes will be ratified by the Department of Health and Social Care, NHS England and NHSX.

Assessment criteria

Stage 1 applications

Stage 1 applications will be assessed against the following criteria:

NHS unmet need and market pain
How well does the AI solution support health and care priorities and align with wider government strategies?
Benefit to patients, the NHS, social care and the wider population
What is the expected improvement in health and care outcomes, health and care inequalities, operational efficiency, patient/service user experience and/or safety and quality of care?
The proposed technology and level of innovation
How innovative is the proposed AI solution and how significant is the competitive advantage that this technology affords?
Quality of the work plan
How appropriate is the work plan, are the risks and mitigation strategy clearly articulated?
Intellectual Property (IP), commercialisation and NHS adoption strategy
Based on the phase of development, how appropriate and sustainable are the plans described?
Patient and public involvement
Is involvement of patients, the public and end users appropriate and relevant?
Strength of the project team
To what extent does the team have the right skills and experience to deliver the project, and if applicable, have they demonstrated sufficient engagement in deployment sites?
Value for money
Is the overall budget realistic and justified in terms of the aims and methods proposed? Does the funding amount present value for money with regard to potential impact?

Key dates

The table below highlights the key application dates. Please note that we reserve the right to alter these dates if deemed required.

Activity	Date
Suite of webinars for applicants to address queries regarding the AI Award application process (recorded webinars will be available on the NHS AI Virtual hub)	17 June 2021: launch webinar 2 August: Q&A 1 1 September: Q&A 2
Open for Stage 1 applications	29 June 2021
Stage 1 application deadline	7 September 2021, 1:00pm
Invitations to submit a Stage 2 application	w/c 19 October 2021
Stage 2 application deadline	w/c 23 November 2021
Panel meetings Phase 2, 3 and 4	w/c 31 January 2022
Outcomes communicated to applicants	February 2022
Projects start	From April 2022 earliest, subject to due diligence

How to apply

Please see the following guidance for more information on how to apply for funding:

Contents