To strengthen the regulatory framework for AI and human rights, ECNL submitted three sets of proposals to the current European Commission proposal for an EU-wide regulatory framework on Artificial Intelligence (Artificial Intelligence Act – AIA).
Click here to read our first proposal on how to ensure stakeholder engagement in AI governance structure and supervisory tasks.
In this article we present the second set of our proposals on exemptions and exclusions of AI used for military and national security purposes from the AIA, also endorsed by European Digital Rights (EDRi), Access Now, AlgorithmWatch, ARTICLE 19, Electronic Frontier Finland (EFFI), Electronic Privacy Information Center (EPIC) and Panoptykon Foundation.
AI-enabled national security and law enforcement measures and their impact on civic space
Recent years have seen a proliferation of AI in the field of national security and law enforcement. This trend has been reinforcing adverse effects on our civic freedoms and democratic processes. It has also been negatively impacting the critical work of civil society, online and offline. The use of “security” technology has been known to target protestors (e.g., via biometric recognition) or limit activism (e.g., through removal of “terrorist” content on the internet). There is also a continuous demand for adapting the technology from the security and military arena to other fields (e.g., use of surveillance tools to detect terrorism is repurposed to monitor activism on the streets or adherence to pandemic rules). All this warrants a close watch and clear safeguards for civic freedoms while using AI for security related purposes.
Exemptions in the proposed AIA
However, the proposed AIA excludes AI systems developed or used exclusively for military purposes from its scope of application. Furthermore, the compromise text on the AIA approved by the Slovenian Presidency of the Council of the EU calls for an additional exemption regarding AI systems developed or used exclusively for national security purposes.
An (in) famous example of technology originally developed for national security only
The notorious case of NSO Group’s Pegasus spyware is the perfect example of a technology originally developed exclusively for national security purposes. In practice, however, this technology has also been used for law enforcement purposes – i.e., to help government agencies detect and prevent terrorism and crime – and even abused in those circumstances for spying ministers, journalists and dissidents/protesters among others. This resulted in civic space restrictions and human rights violations on a massive scale around the world.
What would be the consequences of excluding AI for “national security purposes” from the AIA?
Without proper regulatory safeguards, intrusive AI-based technologies – including with mass surveillance outcomes – could be used in the public sector with no special limitations or safeguards whenever “national security” grounds are invoked by a Member State. Even those AI systems presenting “unacceptable” levels of risks and therefore prohibited by the AIA could be easily “resuscitated” or “recycled” for the exclusive purpose of national security, affecting our freedoms of movement, assembly, expression, participation and privacy, among others.
In the context of developing technology, there is no clear line between law enforcement and national security. Therefore, AI developers will not be able to distinguish between an AI-based system to be used in an investigation related to an organised drug crime or a bomb threat.
ECNL’s legal briefer proposing amendments to Article 2 of the AIA (Scope)
The ultimate purpose of the AI Act is to ensure a harmonised horizontal legal framework with minimum common rules and safeguards for the development, putting into service and use of AI systems. Thus, the rules and safeguards should also be applicable to AI systems that can potentially be used for military as well as national security purposes.
Therefore, in our legal briefer to the EU policymakers jointly endorsed by the above civil society groups, we propose to eliminate the reference to “national security purposes” from the overall exemptions to the scope of the AI Act and to clarify the exact scope and implications of “military purposes”.
Read more in our proposal: