With the proposed Artificial Intelligence Act, the European Commission has taken the first steps towards uniform regulation on artificial intelligence in the EU. The act aims to take a balanced approach to regulate artificial intelligence, which ensures effective protection of fundamental rights without limiting artificial intelligence’s socioeconomic benefits. However, as Van Kolfschooten argues, the proposal fails to address the health specific challenges that artificial intelligence presents. 

Given the unfathomable “black box” nature of artificial intelligence, it may also affect fundamental rights, such as the rights to non-discrimination, privacy, and access to justice. The EU’s artificial intelligence proposal takes a risk-based approach to the regulation of artificial intelligence: the higher the risk, the stricter the rule. While the proposal stipulates that all devices falling under the Medical Devices Regulation (MDR) qualify as “high risk,” healthcare is nonetheless conspicuous in its absence from the list of high risk areas. Van Kolfschooten argues that this omission is foremost caused by the lack of a human-centric approach: the proposal focusses on companies rather than people. The proposed Act ignores the perspective of the “end users” or those affected by artificial intelligence powered decisions.This is especially harmful in the health and clinical context, where people are particularly susceptible to the risks of artificial intelligence because of the inherent dependency and information asymmetries in the patient-doctor relationship.

In order to adequately protect people’s rights in the context of health-related artificial intelligence, Van Kolfschooten states that the EU must empower those affected by artificial intelligence systems with effective and enforceable rights. In addition, she stresses that health and healthcare must be included in the list of “high risk” areas. This way, Europe can reap the benefits of artificial intelligence in health and medical science as a whole.

The blog has been posted on BMJ Opinion.