Killer Robots: The Case for Human Control

Countries should retain meaningful human control over weapons systems and ban fully autonomous weapons, also known as "killer robots," Human Rights Watch and the Harvard Law School International Human Rights Clinic said in a report issued today. The concept of meaningful human control will be a centerpiece of deliberations at a week-long multilateral meeting on the weapons, opening April 11, 2016, at the United Nations in Geneva.

The 16-page report, "Killer Robots and the Concept of Meaningful Human Control," discusses the moral and legal importance of control and shows countries' growing recognition of the need for humans to remain in charge of the critical functions of selecting and firing on targets.

"Machines have long served as instruments of war, but historically humans have directed how they are used," said Bonnie Docherty, senior arms division researcher at Human Rights Watch and the report's lead author. "Now, there is a real threat that humans would relinquish their control and delegate life-and-death decisions to machines."

Fully autonomous weapons would go a step beyond existing remote-controlled drones as they would be able to select and engage targets without human intervention. Although these weapons do not exist yet, the rapid movement of technology from human "in-the-loop" weapons systems toward "out-of-the-loop" systems is attracting international attention and concern.

Human Rights Watch and the Harvard program also examined the rules requiring control in various areas of international law, including disarmament, and how they could provide insight into the use of the term in the context of autonomous weapons. Bans on mines, biological weapons, and chemical weapons show the value disarmament law has placed on control of weapons. A requirement for meaningful human control over lethal force would in effect prohibit the use of fully autonomous weapons and thus achieve a preemptive ban on fully autonomous weapons, the organizations said.

The report will be distributed at the third international meeting on lethal autonomous weapons systems at the UN in Geneva from April 11 to 15. Many of the 122 countries that have joined the Convention on Conventional Weapons are expected to attend this meeting of experts on the subject, which Germany is chairing. The meeting continues deliberations on the matter held in April 2015 and May 2014.

Human Rights Watch coordinates the Campaign to Stop Killer Robots, which believes that work in the Convention on Conventional Weapons forum should lead to new international law prohibiting fully autonomous weapons. The Convention on Conventional Weapons preemptively banned blinding lasers in 1995.

"Humans should retain control of weapons systems and individual attacks, not only of overall operations," said Docherty, who is also a lecturer at the Harvard human rights clinic. "Mandating human control would help avoid threats to the fundamental moral principles over the decision to use force."

Fully autonomous weapons would lack the human capacity to feel empathy, which can act as a key check on killing, the organizations said. Ceding human control over decisions about who lives and who dies would also deprive people of their inherent dignity, as inanimate machines can neither truly comprehend the value of human life nor the significance of its loss.

Mandating meaningful human control would close the accountability gap that would be created by the use of fully autonomous weapons. It would ensure that someone could be punished for an unlawful act caused by the use of the weapon. With a legal requirement for human control, a commander could be held criminally liable for using any weapon without such control.

Meaningful human control over the use of weapons is also consistent with and promotes compliance with the principles of international humanitarian law, notably distinction and proportionality. Distinction requires the ability to understand an individual's behavior and proportionality requires human judgment to weigh civilian harm and military advantage. Human control is also crucial to upholding human rights law. Two UN special rapporteurs said in February that: "Where advanced technology is employed, law enforcement officials must remain personally in control of the actual delivery of use of force."

Since the international debate over full autonomy in weapons systems began at the Convention on Conventional Weapons meeting in 2013, almost 30 countries have specifically addressed the concept of human control in their statements, largely characterizing it as meaningful, appropriate, or effective. Many of these countries explicitly support requiring human control and most have called for more in-depth discussions of the topic.

The concept of meaningful human control is also gaining currency outside of the Convention on Conventional Weapons. For example, a 2015 commentary on the right to life issued by the treaty body of the African Charter of Human and People's Rights found that: "Any machine autonomy in the selection of human targets or the use of force should be subject to meaningful human control."

Countries participating in the Geneva meeting will not make any formal decisions. The meeting aims to build a common base of knowledge about technical, ethical, legal, operational, security, and other concerns relating to the weapons. However, they "may agree by consensus on recommendations for further work for consideration" by the treaty at its Fifth Review Conference, a multilateral meeting held every five years, in December 2016.

Human Rights Watch is a co-founder of the Campaign to Stop Killer Robots. Mary Wareham, advocacy director for the Human Rights Watch arms division, serves as the campaign's global coordinator. The international coalition of more than 60 nongovernmental organizations is working to preemptively ban on the development, production, and use of fully autonomous weapons.

"Countries should not only dedicate more time to considering concerns about killer robots but also commit to pursue a timely and tangible outcome," Docherty said. "Substantive work will only be possible if countries adopt policy and legislative measures to retain human control of weapons before the technology advances too far."