EU AI Act (Regulation (EU) 2024/1689)

Determine and record, for each AI system, whether it is high-risk. A system is high-risk where it is intended to be used as a safety component of, or is itself, a product covered by the Union harmonisation legislation listed in Annex I and that product must undergo third-party conformity assessment, or where it falls within an Annex III use case. Where the provider concludes that an Annex III system is not high-risk because it performs only a narrow procedural task, improves the result of a previously completed human activity, detects decision patterns without replacing or influencing human assessment, or performs a preparatory task, that assessment must be documented before the system is placed on the market or put into service and produced to authorities on request. A system that performs profiling of natural persons is always high-risk and the derogation is not available to it.

Deployers shall use high-risk AI systems in accordance with the IFU; assign human oversight to appropriately competent natural persons; ensure input data is relevant and sufficiently representative; monitor operation and inform the provider of risks/incidents; retain automatically generated logs for at least 6 months (longer where required); inform workers/representatives where used in the workplace; carry out a DPIA where required under GDPR; and where a deployer is a public authority, register the system in the EU database.

Providers and deployers of AI systems must take measures to ensure, to their best extent, a sufficient level of AI literacy among their own staff and any other persons who deal with the operation and use of AI systems on their behalf. The measures must be calibrated to those persons' technical knowledge, experience, education and training, to the context in which the AI systems are to be used, and to the persons or groups of persons on whom the systems are to be used. The duty attaches to every AI system regardless of its risk class.

High-risk AI systems shall be designed and developed in such a way that they can be effectively overseen by natural persons during the period in which they are in use. Oversight measures shall enable persons to understand the relevant capacities and limitations and monitor operation, remain aware of automation bias, correctly interpret the output, decide not to use the output or override or reverse it, intervene in operation, and stop the system.

High-risk AI systems shall be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness, and cybersecurity, and shall perform consistently in those respects throughout their lifecycle. Resilience to errors, faults and inconsistencies; protection against attempts by unauthorised third parties to alter use, output or performance (incl data poisoning, model poisoning, adversarial examples and confidentiality attacks).

High-risk AI systems shall be designed and developed in such a way as to ensure that their operation is sufficiently transparent to enable deployers to interpret the output and use it appropriately. Providers shall provide instructions for use including the system's intended purpose, level of accuracy/robustness/cybersecurity, foreseeable misuse, performance characteristics, human oversight measures, hardware/software requirements, lifetime and maintenance/care.

High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system, ensuring a level of traceability appropriate to the intended purpose; logging capabilities for biometric remote-identification AI systems include the period of each use, the reference database against which input data has been checked, the input data for which the search led to a match, and the natural persons involved in the verification.

Distributors/importers/deployers/other third parties become providers when they place on the market or put into service under their own name or trademark, substantially modify the system, or modify the intended purpose making it high-risk. The original provider shall cooperate with the new provider, providing access to information, technical access and other assistance reasonably needed.