SUBSCRIBE TO OUR FREE NEWSLETTER
Daily news & progressive opinion—funded by the people, not the corporations—delivered straight to your inbox.
5
#000000
#FFFFFF
5
#000000
#FFFFFF
autonomous weapons
Anthropic Sues Trump Pentagon Over 'Supply Chain Risk' Designation
“These actions are unprecedented and unlawful,” the lawsuit said after the Pentagon punished the AI company for refusing to lift restrictions on using their products for autonomous killer robots or mass surveillance.
Mar 09, 2026
Anthropic is suing the Trump administration over its unprecedented attempt to coerce the company into allowing the military to use its artificial intelligence technology without ethical restrictions.
After the company refused to bend to Defense Secretary Pete Hegseth's demands that it drop limits on the use of its product for specific purposes—including to create autonomous weapons and for the mass surveillance of Americans—the Pentagon formally designated Anthropic as a “supply chain risk" on Thursday.
The designation under the Federal Acquisition Supply Chain Security Act (FASCSA) imposes a sweeping prohibition on contractors using the company's technology, including its highly advanced language model Claude.
Hegseth said that effective immediately, "no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic."
The "supply chain risk" designation has typically only been used against foreign companies with ties to adversaries of the United States. According to the Associated Press, Anthropic is the first American firm to be slapped with the label.
On Monday, the San Francisco-based company filed two lawsuits—one in California federal court and another in the federal appeals court in Washington, DC—each challenging different aspects of the designation.
“These actions are unprecedented and unlawful,” Anthropic’s lawsuit says. “The Constitution does not allow the government to wield its enormous power to punish a company for its protected speech. No federal statute authorizes the actions taken here. Anthropic turns to the judiciary as a last resort to vindicate its rights and halt the executive’s unlawful campaign of retaliation.”
Anthropic CEO Dario Amodei has warned about the dangers of "AI-enabled autocracies" that use their technology to more efficiently invade and dominate less powerful countries and stamp out anti-government sentimet.
"Anthropic’s Usage Policy has always conveyed its view that Claude should not be used for two specific applications: (1) lethal autonomous warfare and (2) surveillance of Americans en masse. Anthropic has never tested Claude for those uses. Anthropic currently does not have confidence, for example, that Claude would function reliably or safely if used to support lethal autonomous warfare," the lawsuit continued.
"These usage restrictions," it said, "are therefore rooted in Anthropic’s unique understanding of Claude’s risks and limitations—including Claude’s capacity to make mistakes and its unprecedented ability to accelerate and automate the analysis of massive amounts of data, including data about American citizens."
The Trump administration issued its ultimatum to Anthropic just days before the US and Israel launched a massive war with Iran, which has involved the targeting of thousands of civilian sites, according to the Iranian Red Crescent Society, including schools, hospitals, oil and water facilities, and residential areas.
The war has resulted in the deaths of at least 1,255 Iranians so far as of Monday, according to the country's deputy health minister, Ali Jafarian. Most of those killed have been civilians, Jafarian said, and have included about 200 children and 11 healthcare workers.
Hegseth, who has said the US would follow "no stupid rules of engagement" and boasted that the military was raining down “death and destruction from the sky all day long" upon Iran, has described adopting artificial intelligence as something necessary to make America's military "more lethal."
Last week, the Washington Post reported that in Iran, the US has “leveraged the most advanced artificial intelligence it’s ever used in warfare, a tool that could be difficult for the Pentagon to give up even as it severs ties with the company that created it.”
During the war's first 24 hours, Palantir’s Maven Smart System, which contains Claude, reportedly helped US commanders select 1,000 Iranian targets, according to the Post, which credited the program with "speeding the pace of the campaign."
This is despite the fact that, as SkyNews tech correspondent Rowland Manthorpe recently demonstrated, when presented with "tricky images," AI programs from Claude to ChatGPT to Google's Gemini still "struggle to recognize what is really going on."
"Now," he said, "this very same system is being used for war."
That first day of the war, February 28 saw a massacre in which a Tomahawk missile likely directed by the US obliterated a girls' school in Minab, resulting in at least 175 people killed—mostly children aged 7 to 12—in what was reportedly a "double-tap" strike. Despite video evidence suggesting otherwise, the Trump administration has claimed that Iran was responsible for the massacre.
It is unclear what, if any, role artificial intelligence systems played in the bombing of the Minab school, which was adjacent to an Islamic Revolutionary Guard Corps facility. One investigation by Al Jazeera concluded that the bombing of the school was likely "deliberate."
Beyond putting the lives of innocent people at risk through indiscriminate attacks that lack human intervention, media analyst and journalist Adam Johnson has warned that the adoption of AI in warfare will also allow the US, Israel, and other countries to avoid responsibility for atrocities their militaries commit while using the technology.
"One reason these systems are attractive to militaries is that they double as moral laundromats. Offsetting responsibility to AI is a feature, not a bug," Johnson said. "If the decision about what to bomb can be pawned off on some over-eager or sloppy 'AI', then no person, or system even, is responsible. That's a primary selling point of off-setting 'target-choosing' responsibilities to a machine. It's not just speed, it's blanket indemnification."
Keep ReadingShow Less
Anthropic CEO 'Cannot in Good Conscience Accede' to Pentagon's AI Demand
"Anthropic and Dario deserve credit for standing up for two very basic and obvious principles: no mass surveillance and no autonomous killer robots," said one progressive commentator.
Feb 26, 2026
Defense Secretary Pete Hegseth gave Anthropic until Friday evening to agree to let the Pentagon use the company's artificial intelligence technology however it wants, or else. Roughly 24 hours ahead of the deadline, CEO Dario Amodei announced that "we cannot in good conscience accede to their request," and reiterated opposition to enabling autonomous weapons or surveillance of US citizens.
Anthropic's Claude was the first AI model allowed to handle classified US military data. While the Department of Defense (DOD) has now signed an agreement with Elon Musk's xAI and "is getting close to making a deal with Google," as the New York Times reported Monday, Hegseth demanded "unfettered" access to Claude during a Tuesday meeting with Amodei.
Hegseth threatened to declare the Anthropic a "supply chain risk," effectively blacklisting it for military use and ending its current contract, or invoke the Defense Production Act, which would force Anthropic to tailor the product to the DOD’s needs, if Amodei refused to drop the company's guardrails.
The CEO responded publicly with a Thursday blog post. Using President Donald Trump's preferred name for the Pentagon, he wrote that "Anthropic understands that the Department of War, not private companies, makes military decisions. We have never raised objections to particular military operations nor attempted to limit use of our technology in an ad hoc manner."
"However, in a narrow set of cases, we believe AI can undermine, rather than defend, democratic values. Some uses are also simply outside the bounds of what today's technology can safely and reliably do," Amodei continued. He explained the company's position that "using these systems for mass domestic surveillance is incompatible with democratic values."
"AI-driven mass surveillance presents serious, novel risks to our fundamental liberties. To the extent that such surveillance is currently legal, this is only because the law has not yet caught up with the rapidly growing capabilities of AI," he wrote. "For example, under current law, the government can purchase detailed records of Americans' movements, web browsing, and associations from public sources without obtaining a warrant, a practice the Intelligence Community has acknowledged raises privacy concerns, and that has generated bipartisan opposition in Congress. Powerful AI makes it possible to assemble this scattered, individually innocuous data into a comprehensive picture of any person's life—automatically and at massive scale."
The CEO also argued that "frontier AI systems are simply not reliable enough to power fully autonomous weapons. We will not knowingly provide a product that puts America's warfighters and civilians at risk." He noted that Anthropic offered to work directly with the department on research and development to "improve the reliability of these systems, but they have not accepted this offer."
Amodei concluded by expressing hope that the Pentagon revises its position, writing that "our strong preference is to continue to serve the department and our warfighters—with our two requested safeguards in place. Should the department choose to offboard Anthropic, we will work to enable a smooth transition to another provider, avoiding any disruption to ongoing military planning, operations, or other critical missions."
Amodei's blog post followed CBS News reporting earlier Thursday that "Pentagon officials on Wednesday night sent Anthropic their best and final offer in negotiations for use of the company's artificial intelligence technology."
It also came just hours after Pentagon spokesperson Sean Parnell responded to a related post from a Google scientist on Musk's social media platform X. The DOD official claimed that "the Department of War has no interest in using AI to conduct mass surveillance of Americans (which is illegal) nor do we want to use AI to develop autonomous weapons that operate without human involvement. This narrative is fake and being peddled by leftists in the media."
"Here's what we're asking: Allow the Pentagon to use Anthropic's model for all lawful purposes. This is a simple, commonsense request that will prevent Anthropic from jeopardizing critical military operations and potentially putting our warfighters at risk. We will not let ANY company dictate the terms regarding how we make operational decisions," Parnell added, noting the Friday deadline and the threat to "terminate our partnership with Anthropic and deem them a supply chain risk."
While Amodei and observers await the Pentagon's next move, several Anthropic employees, other tech experts, and critics of the Trump administration praised the CEO for "standing on principle" and choosing "war with the Department of War."
"Anthropic and Dario deserve credit for standing up for two very basic and obvious principles: no mass surveillance and no autonomous killer robots," said progressive commentator Krystal Ball. "Perhaps this is a low bar but it isn’t clear any of the other leading AI companies would put principle above profits in ANY scenario. The Pentagon is sure to make Anthropic pay for daring to defy them."
Keep ReadingShow Less
Congress Must Review an Autonomous Weapons Deal Between UAE and US Companies
There is a serious concern that these autonomous weapons could end up in the arms of the UAE-backed militia and regimes in the Middle East, which would fuel ongoing wars and cause great harm.
Dec 23, 2025
In December 2025, a joint venture was announced between EDGE, the leading Emirati advanced technology and defense conglomerate comprising 25 companies involved in military and civilian products and technologies, and Anduril Industries, an American defense company specializing in the development of advanced autonomous systems. The first product envisioned under this joint venture is Omen, a newly developed hover-to-cruise Autonomous Air Vehicle, or AAV. According to the terms of the agreement, the UAE will acquire the first 50 units of Omen.
There is, however, a serious concern that these autonomous weapons could end up in the arms of the UAE-backed militia and regimes in the Middle East, which would fuel ongoing wars and cause great harm.
For years, the United Arab Emirates has presented itself as a stabilizing force in the Middle East and the Horn of Africa. Yet its actions on the ground tell a far more troubling story. From Libya to Ethiopia, the UAE has repeatedly backed armed groups and proxy forces, deepening conflicts rather than resolving them. In Somalia and Yemen, it has been bolstering the separatists' voice. During the ongoing War in Sudan, the UAE has been backing the RSF militia with financial and military support to the RSF militia, including a recent supply of foreign combatants. The impact of UAE funding to the RSF militia has been catastrophic; it enabled the militia to commit numerous massacres and genocides in the Darfur region. According to United Nations experts, it is estimated that the militia killed 15,000 members of the Massalit tribe based on their ethnicity. In other parts of Darfur, women were raped and abducted, and children were piled up and shot to death. For months, El-Fashir city, the main refugee area in Darfur, has been besieged by the militia.
Here, we are not speculating but building our analysis on previous solid violations cases. EDGE has consistently supported the UAE's allied militias in different parts of the Middle East. In November 2024, an investigative report by Amnesty International exposed that armored personnel carriers (APCs) were found in Sudan. These arms are manufactured by EDGE and are used by the RSF militia in stark violations of the UN arms embargo in Darfur. UN experts said that vehicles built by EDGE were also found in Libya and Somalia. These revelations show how EDGE could go far to collude with the UAE regime in its devastating wars and reckless interventions.
Ideally, an oversight mechanism should be established to ensure that these autonomous weapons and drones will not reach outlaw military militias and rogue regimes.
Recently, the UAE started to use Wagner and other Russian militia in its operations in Africa, for instance, in September 2024, it used Wagner to funnel arms to its RSF ally. The UAE also supported the Central Republic of Africa to pay the cost of hiring the Russian Atlas Corps to defend the government. And in November 2020, a report by the Pentagon mentioned that he UAE funded Wagner in Libya. Given these growing ties, it is legitimate to consider a scenario where these advanced autonomous systems could be leaked to Russian mercenary groups at any point in the future, which indeed represents a serious threat to US security and its interests in Africa.
Related to this are the recent reports that show that the UAE has increased its arms to the region in 2025. This escalation reflects its plans and goals, and that these autonomous weapons will be on its list for the next shipments, providing its militia and allied regimes with a competitive advantage over its foes, i.e stable governments and nations
When thinking about this deal, we have to take into consideration that the UAE isn’t a democratic country. It's ruled by a single family, with no parliament to review decisions made concerning wars. Hence, it can’t be trusted by any measure to act responsibly if it owns this advanced technology.
Congress must take this issue seriously and review this deal. Ideally, an oversight mechanism should be established to ensure that these autonomous weapons and drones will not reach outlaw military militias and rogue regimes and be part of subverting countries and jeopardizing the US long-term interests.
Keep ReadingShow Less
Experts Warn Window for International Regulation on Killer Robots Is 'Rapidly Shrinking'
"Time is really running out to put in some guardrails so that the nightmare scenarios that some of the most noted experts are warning of don't come to pass," said the disarmament director at the Austrian Foreign Ministry.
May 13, 2025
Countries met at the United Nations on Monday as part of an effort toward establishing global rules around the use of so-called "killer robots"—autonomous weapons systems that select targets based on inputs from sensors rather than from humans. Arms control and humanitarian officials who spoke at the meeting and to press expressed that time is running out to prohibit and regulate these weapons.
Mirjana Spoljaric, president of the International Committee of the Red Cross, said at the U.N. informal consultations on autonomous weapons that the "technology is moving at lightning speed, and the implications grow more worrying. The window to apply effective international regulations and controls on autonomous weapon systems before they are in widespread use is rapidly shrinking."
"The reality of autonomous weapons systems on the battlefield is increasing. Crucially, the need for robust international law is becoming all the more pressing and more consequential," said Verity Cole, a senior adviser and campaigner for the human rights group Amnesty International, at the U.N. meeting on Monday.
"We need a legally binding instrument... the time has come to negotiate and adopt a treaty that prohibits and regulates autonomous weapons systems," said Nicole van Rooijen, executive director of the Stop Killer Robots coalition, on Monday.
Autonomous weapons are already in use, such as in the conflict in Ukraine. Last month, the group Human Rights Watch released a report warning about the possible human rights implications of the potential unchecked proliferation of autonomous weapons.
Since 2014, countries that are party to the Convention on Conventional Weapons (CCW) "have been meeting in Geneva to discuss a potential ban fully autonomous systems that operate without meaningful human control and regulate others," according to Reuters.
In 2023, with the support of 164 states, the U.N. General Assembly adopted the first ever resolution on autonomous weapons, calling on the international community to address the risks posed by these weapons.
The Monday gathering of the U.N. General Assembly was the first time the body has met for a discussion dedicated to autonomous weapons, Reuters reported.
U.N. Secretary-General António Guterres has called for states to come up with a "legally binding instrument" to bar certain lethal autonomous weapons and regulate all other types of autonomous weapons by 2026. He reiterated this call on Monday.
According to Reuters, human rights groups are concerned there's a lack of consensus among countries for this sort of instrument.
Alexander Kmentt, director of disarmament and arms control at the Austrian Foreign Ministry, told Reuters that that needs to change.
"Time is really running out to put in some guardrails so that the nightmare scenarios that some of the most noted experts are warning of don't come to pass," he said.
Keep ReadingShow Less
Not Just for the Battlefield: Rights Group Warns of Dystopian World Where Killer Robots Reign
"To avoid a future of automated killing, governments should seize every opportunity to work toward the goal of adopting a global treaty on autonomous weapons systems," according to the author of the report.
Apr 28, 2025
In a report published Monday, a leading human rights group calls for international political action to prohibit and regulate so-called "killer robots"—autonomous weapons systems that select targets based on inputs from sensors rather than from humans—and examines them in the context of six core principles in international human rights law.
In some cases, the report argues, an autonomous weapons system may simply be incompatible with a given human rights principle or obligation.
The report, co-published by Human Rights Watch and Harvard Law School's International Human Rights Clinic, comes just ahead of the first United Nations General Assembly meeting on autonomous weapons systems next month. Back in 2017, dozens of artificial intelligence and robotics experts published a letter urging the U.N. to ban the development and use of killer robots. As drone warfare has grown, those calls have continued.
"To avoid a future of automated killing, governments should seize every opportunity to work toward the goal of adopting a global treaty on autonomous weapons systems," said the author behind the report, Bonnie Docherty, a senior arms adviser at Human Rights Watch and a lecturer on law at Harvard Law School's International Human Rights Clinic, in a statement on Monday.
According to the report, which includes recommendations on a potential international treaty, the call for negotiations to adopt "a legally binding instrument to prohibit and regulate autonomous weapons systems" is supported by at least 129 countries.
Drones relying on an autonomous targeting system have been used by Ukraine to hit Russian targets during the war between the two countries, The New York Times reported last year.
In 2023, the Pentagon announced a program, known as the Replicator initiative, which involves a push to build thousands of autonomous drones. The program is part of the U.S. Defense Department's plan to counter China. In November, the watchdog group Public Citizen alleged that Pentagon officials have not been clear about whether the drones in the Replicator project would be used to kill.
A senior Navy admiral recently told Bloomberg that the program is "alive and well" under the Department of Defense's new leadership following U.S. President Donald Trump's return to the White House.
Docherty warned that the impact of killer robots will stretch beyond the traditional battlefield. "The use of autonomous weapons systems will not be limited to war, but will extend to law enforcement operations, border control, and other circumstances, raising serious concerns under international human rights law," she said in the statement
When it comes to the right to peaceful assembly under human rights law, which is important in the context of law enforcement exercising use force, "autonomous weapons systems would be incompatible with this right," according to the report.
Killer robots pose a threat to peaceful assembly because they "would lack human judgment and could not be pre-programmed or trained to address every situation," meaning they "would find it challenging to draw the line between peaceful and violent protesters."
Also, "the use or threat of use of autonomous weapons systems, especially in the hands of abusive governments, could strike fear among protesters and thus cause a chilling effect on free expression and peaceful assembly," per the report.
Killer robots would also contravene the principle of human dignity, according to the report, which establishes that all humans have inherent worth that is "universal and inviolable."
"The dignity critique is not focused on the systems generating the wrong outcomes," the report states. "Even if autonomous weapons systems could feasibly make no errors in outcomes—something that is extremely unlikely—the human dignity concerns remain, necessitating prohibitions and regulations of such systems."
"Autonomous weapon systems cannot be programmed to give value to human life, do not possess emotions like compassion that can generate restraint to violence, and would rely on processes that dehumanize individuals by making life-and-death decisions based on software and data points," Docherty added.
In total, the report considers the right to life; the right to peaceful assembly; the principle of human dignity; the principle of nondiscrimination; the right to privacy; and the right to remedy.
The report also lists cases where it's more ambiguous whether autonomous weapons systems would violate a certain right.
The right to privacy, for example, protects individuals from "arbitrary or unlawful" interferences in their personal life. According to the report, "The development and use of autonomous weapons systems could violate the right because, if they or any of their component systems are based on AI technology, their development, testing, training, and use would likely require mass surveillance."
Keep ReadingShow Less
Report on US Killer Robots Warns of 'Reckless, Dangerous Arms Race'
"All signs point to the Pentagon developing 'killer robots' via Replicator, despite deflections from Pentagon representatives themselves," according to Public Citizen.
Nov 22, 2024
A report from the government watchdog Public Citizen released Friday gives the who, what, when, where, and why of the Pentagon's flagship Replicator initiative—a program to increase the number of weapons, particularly drones, in the hands of the U.S. military.
In the report, Public Citizen re-ups concerns about one particular aspect of the program. According to the report's author, Savannah Wooten, the Defense Department has remained ambiguous on the question of whether it is developing artificial intelligence weapons that can "deploy lethal force autonomously—without a human authorizing the specific use of force in a specific context." These types of weapons are also known as "killer robots."
"It is not yet clear whether or not these technologies are designed, tested, or intended for killing," according to the report.
"All signs point to the Pentagon developing 'killer robots' via Replicator, despite deflections from Pentagon representatives themselves," wrote Wooten in the summary of the report.
The program, which was announced last year, is part of the Department of Defense's plan to deter China.
"Replicator is meant to help us overcome [China's] biggest advantage, which is mass. More ships. More missiles. More people," said Deputy Secretary of Defense Kathleen Hicks in a speech announcing the project last year. That mission will be achieved specifically by "mastering the technology of tomorrow," Hicks said.
There will soon be a "Replicator 2.0" that will focus on counter-drone technologies—per a memo from the defense secretary released in September—according to Public Citizen's report.
In a letter sent in March, Public Citizen and 13 other civil society groups highlighted remarks Hicks made in 2023 as an example of the ambiguity the Pentagon has created around the issue.
"Autonomous weapons are inherently dehumanizing and unethical, no matter whether a human is 'ultimately' responsible for the use of force or not. Deploying lethal artificial intelligence weapons in battlefield conditions necessarily means inserting them into novel conditions for which they have not been programmed, an invitation for disastrous outcomes," the organizations wrote to Hicks and Secretary of Defense Lloyd Austin.
Wooten's report reiterates that same call: "The Pentagon owes Americans clarity about its own role in advancing the autonomous weapons arms race via Replicator, as well as a detailed plan for ensuring it does not open a Pandora’s Box of new, lethal weapons on the world by refusing to hold its own operations accountable."
Additionally, "'Artificial intelligence' should not be used as a catchall justification to summon billions more in Pentagon spending, especially when the existing annual budget for the U.S. military already dwarfs every other U.S. agency and is careening towards the $1 trillion mark," Wooten wrote.
The fear that these types of weapons would open a Pandora's Box—and set off a "reckless, dangerous arms race," as Public Citizen warned of Friday—is not new. Back in 2017, dozens of artificial intelligence and robotics experts published a letter urging the United Nations to ban the development and use of so-called killer robots. As drone warfare has grown, those calls have continued.
The report also highlights the public statements of the head of one defense contractor that has been selected to produce for the Replicator initiative as a hint that the program is aimed at creating weapons that are capable of autonomous lethal force.
In early October, CEO of Anduril Palmer Luckey said that, "societies have always needed a warrior class that is enthused and excited about enacting violence on others in pursuit of good aims."
"You need people like me who are sick in that way and who don't lose any sleep making tools of violence in order to preserve freedom," he said.
Keep ReadingShow Less
Pentagon Urged to Just Say No to AI-Powered Killer Robots
"The Department of Defense should declare its opposition to the development and deployment of autonomous weapons."
Mar 26, 2024
The watchdog group Public Citizen on Tuesday led a letter urging Pentagon leaders "to clarify that the Replicator Initiative will not involve the development and deployment of autonomous weapons systems," also known as "killer robots."
Last September, U.S. Deputy Secretary of Defense Kathleen Hicks "asserted that the development of all-domain, attributable autonomy systems (ADA2) is an essential way for the Pentagon to maintain its comparative cutting-edge and keep up with the technological advancements of other states," notes the letter, which was addressed to her and Defense Secretary Lloyd Austin.
"However, those comments failed to specify whether or not supporting autonomous weapons systems is one of the key focuses of this initiative," the letter stresses. "When addressing whether or not 'ADA2 means weapons systems,' Secretary Hicks stated: 'That's a serious question to be sure. They are not synonymous. There are many applications for ADA2 systems beyond delivering weapons effects.'"
"Autonomous weapons are inherently dehumanizing and unethical, no matter whether a human is 'ultimately' responsible for the use of force or not."
Public Citizen and the 13 other organizations argued that "this is no place for strategic ambiguity. Autonomous weapons are inherently dehumanizing and unethical, no matter whether a human is 'ultimately' responsible for the use of force or not."
Deploying lethal weapons that rely on artificial intelligence (AI) "in battlefield conditions necessarily means inserting them into novel conditions for which they have not been programmed, an invitation for disastrous outcomes," the groups warned. "'Swarms' of the sort envisioned by Replicator pose even heightened risks, because of the unpredictability of how autonomous systems will function in a network. And the mere ambiguity of the U.S. position on autonomous weapons risks spurring a catastrophic arms race."
"We believe the Department of Defense should declare its opposition to the development and deployment of autonomous weapons," the coalition concluded. "However, even if you are not prepared to make that declaration, we strongly urge you to clarify that the Replicator Initiative will not employ autonomous weapons."
In addition to Public Citizen, the coalition included the American Friends Service Committee, Autistic Women & Nonbinary Network, Backbone Campaign, Demand Progress Education Fund, Fight for the Future, Future of Life, National Priorities Project at the Institute for Policy Studies, RootsAction.org, United Church of Christ, the Value Alliance, Women's International League for Peace and Freedom U.S., Win Without War, and World Beyond War.
The letter comes on the heels of Public Citizen releasing a report about the rise of killer robots, AI Joe: The Dangers of Artificial Intelligence and the Military.
The February report addresses the Pentagon's AI policy, the dangers of killer robots, the need to ensure decisions about nuclear weapons aren't made by automated systems, how artificial intelligence can increase not diminish the use of violence, risks of using deepfakes on the battlefield, and how AI startups are seeking government contracts.
The publication concludes with recommendations that Public Citizen president Robert Weissman echoed in a statement Tuesday.
"The United States should state plainly that it will not create or deploy killer robots and should work to advance global treaty negotiations to ban such weapons," Weissman said. "At minimum, the United States should commit that the Replicator Initiative will not involve the use of autonomous weapons."
"Ambiguity about the Replicator program essentially ensures a catastrophic arms race over autonomous weapons," he added. "That's a race in which all of humanity is the loser."
Keep ReadingShow Less
New Report Warns of the Rise of Killer Robots
Public Citizen says autonomous weapons "inherently dehumanize the people targeted and make it easier to tolerate widespread killing."
Feb 29, 2024
A Public Citizen report released on Thursday outlines the risks of the Pentagon adopting the use of autonomous weapons on the battlefield and how this could occur in the not-too-distant future.
The report focuses on the Pentagon's efforts to utilize artificial intelligence in various ways and the American military contractors that are developing weaponry that utilizes AI, including autonomous weapons.
"The single greatest concern involving AI and the Pentagon is the integration of AI into weapons systems such that they can function autonomously, delivering lethal force without intervention or meaningful human control," the report state. "The most serious worry involving autonomous weapons is that they inherently dehumanize the people targeted and make it easier to tolerate widespread killing, including in violation of international human rights law."
"Autonomous weapons are already in development around the world and racing forward."
Though the Pentagon has instituted some policies to ensure AI will be used ethically by the military, the report states, these policies don't go as far as to ban the use of autonomous weapons—often referred to as "killer robots."
Furthermore, the report notes that military contractors like General Dynamics, Vigor Industrial, Anduril Industries, and others are currently developing unmanned tanks, submarines, and drones.
"As a discrete weapons technology, autonomous weapons deployment is nearly certain to create an AI weapons arms race. That is the logic of international military strategy. In the United States, a geopolitical rivalry-driven autonomous weapons arms race will be spurred further by the military-industrial complex and corporate contractors," the report states. "Autonomous weapons are already in development around the world and racing forward."
Israel has purchased and at times deployed self-piloting, lethal drones. Russia has utilized autonomous drones in Ukraine. The age of the killer robot is here, and Public Citizen warns that the U.S. needs to help prevent the deployment of killer robots from being normalized.
"The United States should pledge not to develop or deploy autonomous weapons, and should support a global treaty banning such weapons," the report states.
Public Citizen also recommends the U.S. adopt policies that prevent using AI to launch nuclear weapons or using deepfakes on the battlefield, which could be used for "influence operations."
Keep ReadingShow Less
When the AI-Powered Killer Drones Start Talking to Each Other
It should be obvious that humanity could face a significant risk to its existence, should killing machines acquire the ability to think on their own.
Feb 20, 2024
Yes, it’s already time to be worried — very worried. As the wars in Ukraine and Gaza have shown, the earliest drone equivalents of “killer robots” have made it onto the battlefield and proved to be devastating weapons. But at least they remain largely under human control. Imagine, for a moment, a world of war in which those aerial drones (or their ground and sea equivalents) controlled us, rather than vice-versa. Then we would be on a destructively different planet in a fashion that might seem almost unimaginable today. Sadly, though, it’s anything but unimaginable, given the work on artificial intelligence (AI) and robot weaponry that the major powers have already begun. Now, let me take you into that arcane world and try to envision what the future of warfare might mean for the rest of us.
By combining AI with advanced robotics, the U.S. military and those of other advanced powers are already hard at work creating an array of self-guided “autonomous” weapons systems — combat drones that can employ lethal force independently of any human officers meant to command them. Called “killer robots” by critics, such devices include a variety of uncrewed or “unmanned” planes, tanks, ships, and submarines capable of autonomous operation. The U.S. Air Force, for example, is developing its “collaborative combat aircraft,” an unmanned aerial vehicle (UAV) intended to join piloted aircraft on high-risk missions. The Army is similarly testing a variety of autonomous unmanned ground vehicles (UGVs), while the Navy is experimenting with both unmanned surface vessels (USVs) and unmanned undersea vessels (UUVs, or drone submarines). China, Russia, Australia, and Israel are also working on such weaponry for the battlefields of the future.
The imminent appearance of those killing machines has generated concern and controversy globally, with some countries already seeking a total ban on them and others, including the U.S., planning to authorize their use only under human-supervised conditions. In Geneva, a group of states has even sought to prohibit the deployment and use of fully autonomous weapons, citing a 1980 U.N. treaty, the Convention on Certain Conventional Weapons, that aims to curb or outlaw non-nuclear munitions believed to be especially harmful to civilians. Meanwhile, in New York, the U.N. General Assembly held its first discussion of autonomous weapons last October and is planning a full-scale review of the topic this coming fall.
Some American strategists have championed an alternative approach to the use of autonomous weapons on future battlefields in which they would serve not as junior colleagues in human-led teams but as coequal members of self-directed robot swarms.
For the most part, debate over the battlefield use of such devices hinges on whether they will be empowered to take human lives without human oversight. Many religious and civil society organizations argue that such systems will be unable to distinguish between combatants and civilians on the battlefield and so should be banned in order to protect noncombatants from death or injury, as is required by international humanitarian law. American officials, on the other hand, contend that such weaponry can be designed to operate perfectly well within legal constraints.
However, neither side in this debate has addressed the most potentially unnerving aspect of using them in battle: the likelihood that, sooner or later, they’ll be able to communicate with each other without human intervention and, being “intelligent,” will be able to come up with their own unscripted tactics for defeating an enemy — or something else entirely. Such computer-driven groupthink, labeled “emergent behavior” by computer scientists, opens up a host of dangers not yet being considered by officials in Geneva, Washington, or at the U.N.
For the time being, most of the autonomous weaponry being developed by the American military will be unmanned (or, as they sometimes say, “uninhabited”) versions of existing combat platforms and will be designed to operate in conjunction with their crewed counterparts. While they might also have some capacity to communicate with each other, they’ll be part of a “networked” combat team whose mission will be dictated and overseen by human commanders. The Collaborative Combat Aircraft, for instance, is expected to serve as a “loyal wingman” for the manned F-35 stealth fighter, while conducting high-risk missions in contested airspace. The Army and Navy have largely followed a similar trajectory in their approach to the development of autonomous weaponry.
The Appeal of Robot “Swarms”
However, some American strategists have championed an alternative approach to the use of autonomous weapons on future battlefields in which they would serve not as junior colleagues in human-led teams but as coequal members of self-directed robot swarms. Such formations would consist of scores or even hundreds of AI-enabled UAVs, USVs, or UGVs — all able to communicate with one another, share data on changing battlefield conditions, and collectively alter their combat tactics as the group-mind deems necessary.
“Emerging robotic technologies will allow tomorrow’s forces to fight as a swarm, with greater mass, coordination, intelligence and speed than today’s networked forces,” predicted Paul Scharre, an early enthusiast of the concept, in a 2014 report for the Center for a New American Security (CNAS). “Networked, cooperative autonomous systems,” he wrote then, “will be capable of true swarming — cooperative behavior among distributed elements that gives rise to a coherent, intelligent whole.”
As Scharre made clear in his prophetic report, any full realization of the swarm concept would require the development of advanced algorithms that would enable autonomous combat systems to communicate with each other and “vote” on preferred modes of attack. This, he noted, would involve creating software capable of mimicking ants, bees, wolves, and other creatures that exhibit “swarm” behavior in nature. As Scharre put it, “Just like wolves in a pack present their enemy with an ever-shifting blur of threats from all directions, uninhabited vehicles that can coordinate maneuver and attack could be significantly more effective than uncoordinated systems operating en masse.”
In 2014, however, the technology needed to make such machine behavior possible was still in its infancy. To address that critical deficiency, the Department of Defense proceeded to fund research in the AI and robotics field, even as it also acquired such technology from private firms like Google and Microsoft. A key figure in that drive was Robert Work, a former colleague of Paul Scharre’s at CNAS and an early enthusiast of swarm warfare. Work served from 2014 to 2017 as deputy secretary of defense, a position that enabled him to steer ever-increasing sums of money to the development of high-tech weaponry, especially unmanned and autonomous systems.
From Mosaic to Replicator
Much of this effort was delegated to the Defense Advanced Research Projects Agency (DARPA), the Pentagon’s in-house high-tech research organization. As part of a drive to develop AI for such collaborative swarm operations, DARPA initiated its “Mosaic” program, a series of projects intended to perfect the algorithms and other technologies needed to coordinate the activities of manned and unmanned combat systems in future high-intensity combat with Russia and/or China.
“Applying the great flexibility of the mosaic concept to warfare,” explained Dan Patt, deputy director of DARPA’s Strategic Technology Office, “lower-cost, less complex systems may be linked together in a vast number of ways to create desired, interwoven effects tailored to any scenario. The individual parts of a mosaic are attritable [dispensable], but together are invaluable for how they contribute to the whole.”
This concept of warfare apparently undergirds the new “Replicator” strategy announced by Deputy Secretary of Defense Kathleen Hicks just last summer. “Replicator is meant to help us overcome [China’s] biggest advantage, which is mass. More ships. More missiles. More people,” she told arms industry officials last August. By deploying thousands of autonomous UAVs, USVs, UUVs, and UGVs, she suggested, the U.S. military would be able to outwit, outmaneuver, and overpower China’s military, the People’s Liberation Army (PLA). “To stay ahead, we’re going to create a new state of the art… We’ll counter the PLA’s mass with mass of our own, but ours will be harder to plan for, harder to hit, harder to beat.”
To obtain both the hardware and software needed to implement such an ambitious program, the Department of Defense is now seeking proposals from traditional defense contractors like Boeing and Raytheon as well as AI startups like Anduril and Shield AI. While large-scale devices like the Air Force’s Collaborative Combat Aircraft and the Navy’s Orca Extra-Large UUV may be included in this drive, the emphasis is on the rapid production of smaller, less complex systems like AeroVironment’s Switchblade attack drone, now used by Ukrainian troops to take out Russian tanks and armored vehicles behind enemy lines.
At the same time, the Pentagon is already calling on tech startups to develop the necessary software to facilitate communication and coordination among such disparate robotic units and their associated manned platforms. To facilitate this, the Air Force asked Congress for $50 million in its fiscal year 2024 budget to underwrite what it ominously enough calls Project VENOM, or “Viper Experimentation and Next-generation Operations Model.” Under VENOM, the Air Force will convert existing fighter aircraft into AI-governed UAVs and use them to test advanced autonomous software in multi-drone operations. The Army and Navy are testing similar systems.
When Swarms Choose Their Own Path
In other words, it’s only a matter of time before the U.S. military (and presumably China’s, Russia’s, and perhaps those of a few other powers) will be able to deploy swarms of autonomous weapons systems equipped with algorithms that allow them to communicate with each other and jointly choose novel, unpredictable combat maneuvers while in motion. Any participating robotic member of such swarms would be given a mission objective (“seek out and destroy all enemy radars and anti-aircraft missile batteries located within these [specified] geographical coordinates”) but not be given precise instructions on how to do so. That would allow them to select their own battle tactics in consultation with one another. If the limited test data we have is anything to go by, this could mean employing highly unconventional tactics never conceived for (and impossible to replicate by) human pilots and commanders.
The propensity for such interconnected AI systems to engage in novel, unplanned outcomes is what computer experts call “emergent behavior.” As ScienceDirect, a digest of scientific journals, explains it, “An emergent behavior can be described as a process whereby larger patterns arise through interactions among smaller or simpler entities that themselves do not exhibit such properties.” In military terms, this means that a swarm of autonomous weapons might jointly elect to adopt combat tactics none of the individual devices were programmed to perform — possibly achieving astounding results on the battlefield, but also conceivably engaging in escalatory acts unintended and unforeseen by their human commanders, including the destruction of critical civilian infrastructure or communications facilities used for nuclear as well as conventional operations.
At this point, of course, it’s almost impossible to predict what an alien group-mind might choose to do if armed with multiple weapons and cut off from human oversight. Supposedly, such systems would be outfitted with failsafe mechanisms requiring that they return to base if communications with their human supervisors were lost, whether due to enemy jamming or for any other reason. Who knows, however, how such thinking machines would function in demanding real-world conditions or if, in fact, the group-mind would prove capable of overriding such directives and striking out on its own.
What then? Might they choose to keep fighting beyond their preprogrammed limits, provoking unintended escalation — even, conceivably, of a nuclear kind? Or would they choose to stop their attacks on enemy forces and instead interfere with the operations of friendly ones, perhaps firing on and devastating them (as Skynet does in the classic science fiction Terminator movie series)? Or might they engage in behaviors that, for better or infinitely worse, are entirely beyond our imagination?
Top U.S. military and diplomatic officials insist that AI can indeed be used without incurring such future risks and that this country will only employ devices that incorporate thoroughly adequate safeguards against any future dangerous misbehavior. That is, in fact, the essential point made in the “Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy” issued by the State Department in February 2023. Many prominent security and technology officials are, however, all too aware of the potential risks of emergent behavior in future robotic weaponry and continue to issue warnings against the rapid utilization of AI in warfare.
Of particular note is the final report that the National Security Commission on Artificial Intelligence issued in February 2021. Co-chaired by Robert Work (back at CNAS after his stint at the Pentagon) and Eric Schmidt, former CEO of Google, the commission recommended the rapid utilization of AI by the U.S. military to ensure victory in any future conflict with China and/or Russia. However, it also voiced concern about the potential dangers of robot-saturated battlefields.
“The unchecked global use of such systems potentially risks unintended conflict escalation and crisis instability,” the report noted. This could occur for a number of reasons, including “because of challenging and untested complexities of interaction between AI-enabled and autonomous weapon systems [that is, emergent behaviors] on the battlefield.” Given that danger, it concluded, “countries must take actions which focus on reducing risks associated with AI-enabled and autonomous weapon systems.”
When the leading advocates of autonomous weaponry tell us to be concerned about the unintended dangers posed by their use in battle, the rest of us should be worried indeed. Even if we lack the mathematical skills to understand emergent behavior in AI, it should be obvious that humanity could face a significant risk to its existence, should killing machines acquire the ability to think on their own. Perhaps they would surprise everyone and decide to take on the role of international peacekeepers, but given that they’re being designed to fight and kill, it’s far more probable that they might simply choose to carry out those instructions in an independent and extreme fashion.
If so, there could be no one around to put an R.I.P. on humanity’s gravestone.
Keep ReadingShow Less
The Perverse Rise of Killer Robots
Oct 09, 2015
The development of "killer robots" is a new and original way of using human intelligence for perverse means. Humans directing machines to kill and destroy on a scale not yet imagined is a concept that not even George Orwell could have imagined. In the meantime, the leading world powers continue their un-merry-go-round of destruction and death -mostly of innocent civilians- without stopping to consider the consequences of their actions.
Killer robots are fully autonomous weapons that can identify, select, and engage targets without meaningful human control. Although fully developed weapons of this kind do not yet exist, world leaders such as the U.S., the U.K., Israel, Russia, China, and South Korea are already working on creating their precursors.
The U.S. Government Accountability Office reports that in 2012, 76 countries had some drones, and 16 countries already possessed armed ones. The U.S. Department of Defense spends $6 billion every year on the research and development of better drones.
South Korea uses Samsung Techwin security surveillance guard robots in its demilitarized zone with North Korea. Although humans operate these units, the robots have an automatic feature that can detect body heat and fire a machine gun without human intervention.
Israel is developing an armed drone called Harop that can select targets with a special sensor. Northrop Grumman has also developed an autonomous drone called the X-47B. This drone can travel on a preprogrammed flight path while being monitored by a pilot on a ship. It is planned to enter active service by 2019. China is also moving rapidly in this area. In 2012, it already had 27 armed drone models, one of which is an autonomous air-to-air supersonic combat aircraft.
Killer robots follow the generation of drones and, as with drones, their potential use is also creating a host of human rights, legal and ethical issues. Military officials state that this hardware protects human life by taking soldiers and pilots out of harm's way. What they don't say, however, is that the protected lives are those of the attacking armies, not those of the mostly civilians who are their targets, whose untimely deaths are euphemistically called collateral damage.
According to Denise Garcia, an expert in international law, four branches of internationally law have been used to limit violence in war: the law of state responsibility, the law on the use of force, international humanitarian law and human rights law. As currently carried out, U.S. drone strikes violate all of them.
From an ethical point of view, using these machines presents a moral dilemma. Allowing machines to make life-and-death decisions removes people's responsibility for their actions and eliminates accountability. Lack of accountability almost ensures future human rights violations. In addition, many experts believe that the proliferation of autonomous weapons would make an arms race inevitable.
As the United Nations is trying to negotiate the future use of autonomous weapons, the U.S. and U.K. representatives want to support weaker rules prohibiting future technology but not killer robots developed during the negotiating period. That delay would allow existing semi-autonomous prototypes to continue being used.
The need for a pre-emptive ban on developing and using this kind of weapon is urgent. As Christof Heyns, the UN Special Rapporteur on Extrajudicial, Summary or Arbitrary Executions, stated recently, "If there is not a pre-emptive ban on the high-level autonomous weapons, then once the genie is out of the bottle, it will be extremely difficult to get it back in."
Keep ReadingShow Less
See More
Most Popular
To inform. To inspire. To ignite change for the common good.
News & Opinion
Trending
Common Dreams, Inc. Founded 1997. Registered 501(c3) Non-Profit
Privacy Policy