Algorithmic Warfare Is Coming. Humans Must Retain Control

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.  By Peter Maurer

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using.

Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

First Published on WEForum.org

Peter Maurer is President, International Committee of the Red Cross @ICRC

You Might Also Read: 

Killer Robots Are A Revolution in Warfare:

Robots Could Join The Ukrainian Conflict:

 

« Routers With Weak Passwords Will Soon Be Illegal In California
Cyberwars Heat Up In AsiaPac To Battle Chinese Aggression »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

CSI Consulting Services

CSI Consulting Services

Get Advice From The Experts: * Training * Penetration Testing * Data Governance * GDPR Compliance. Connecting you to the best in the business.

LockLizard

LockLizard

Locklizard provides PDF DRM software that protects PDF documents from unauthorized access and misuse. Share and sell documents securely - prevent document leakage, sharing and piracy.

Syxsense

Syxsense

Syxsense brings together endpoint management and security for greater efficiency and collaboration between IT management and security teams.

NordLayer

NordLayer

NordLayer is an adaptive network access security solution for modern businesses — from the world’s most trusted cybersecurity brand, Nord Security. 

Perimeter 81 / How to Select the Right ZTNA Solution

Perimeter 81 / How to Select the Right ZTNA Solution

Gartner insights into How to Select the Right ZTNA offering. Download this FREE report for a limited time only.

MIIS Cyber Initiative

MIIS Cyber Initiative

The Cyber Initiative's mission is to assess the impact of the information age on security, peace and communications.

SolarWinds

SolarWinds

SolarWinds as a worldwide leader in solutions for network and IT service management, application performance, and managed services.

Centrify

Centrify

Centrify’s Next-Gen Access is an identity & access management solution that uniquely converges Identity-as-a-Service, enterprise mobility management and privileged access management.

AA Certification (AAC)

AA Certification (AAC)

AAC provide ISO Quality Management System certification services including ISO 27001.

Cybersecurity Philippines CERT (CSP-CERT)

Cybersecurity Philippines CERT (CSP-CERT)

Cybersecurity Philippines CERT is the national Computer Emergency Response Team for the Philippines.

International Telecommunication Union (ITU)

International Telecommunication Union (ITU)

ITU is the United Nations specialized agency for information and communication technologies – ICTs. Areas of activity include cybersecurity.

SGCyberSecurity

SGCyberSecurity

SGCyberSecurity is Singapore's No.1 Cyber Security portal. From this platform, you will be able to find useful articles, resources and connect with the security companies for your business needs.

Get Safe Online

Get Safe Online

Get Safe Online is a leading source of unbiased, factual and easy-to-understand information on online safety.

Document Security Systems (DSS)

Document Security Systems (DSS)

DSS anti-counterfeit, authentication, and brand protection solutions are deployed to prevent attacks which threaten products, digital presence, financial instruments, and identification.

Tokio Marine HCC

Tokio Marine HCC

Tokio Marine HCC is a leading specialty insurance group with a Financial and Professional product line including Tech and Cyber.

Enso Security

Enso Security

Enso is the first Application Security Posture Management (ASPM) solution, helping security teams everywhere eliminate their AppSec chaos with application discovery, classification and management.

Defscope

Defscope

Defscope is an Azerbaijani company entirely focused on cybersecurity offering training, security consulting, and other professional services.

Qohash

Qohash

With a focus on data security, Qohash supports security, compliance and optimization use cases enhancing your risk management process.

Xscale Accelerator

Xscale Accelerator

Xscale's vision is to create world-class startups out of India by transforming sales and providing access to global markets.

Techsolidity

Techsolidity

Techsolidity is an emerging e-learning platform that offers a wide range of upskilling programs worldwide in areas including cybersecurity.

Brightside AI

Brightside AI

Brightside AI is a Swiss cybersecurity SaaS that helps teams combat AI-enabled phishing threats. Protect your team today.