Report Urges Congress to Tackle Venture Capitalists Peddling AI 'Miracle Weapons'

A new report released Monday sounds the alarm on the growing influence of profit-hungry venture capital firms that are promoting weapons systems powered by artificial intelligence, a rapidly emerging technology that experts and watchdogs warn could be an existential threat to humanity if not strongly and properly regulated.

The report, published by the Quincy Institute for Responsible Statecraft, cautions that venture capital (VC) firms and their allies in Washington, D.C. are "determined to move full speed ahead on the development and deployment of weapons based on AI and other technological innovations, despite many unanswered questions about the costs and risks involved."

Michael Brenes and William Hartung, the report's authors, implore Congress to pursue concrete policy actions to regulate the torrent of VC money flowing into the development of AI-powered military technology—so-called "miracle weapons"—as the Pentagon actively courts Silicon Valley startups.

Citing data from PitchBook, The Financial Timesreported last year that "U.S. venture investment in defense startups surged from less than $16 billion in 2019 to $33 billion in 2022."

The Quincy Institute report observes that "the surge in VC investment in emerging arms technology is being spearheaded by a handful of firms and individuals," including "the Founders Fund, started by Peter Thiel, who is also the co-founder of PayPal and the arms technology firm Palantir; and Andreesen Horowitz, whose 'American Dynamism Fund' invests in notable emerging tech firms like Anduril and Shield AI."

"Given the risks of catastrophic malfunction and hair-trigger wars conducted with minimal human input, we need a vigorous national debate before moving full speed ahead on military applications of AI and other emerging technologies," Hartung, a senior research fellow at the Quincy Institute, said in a statement Monday.

Brenes, a nonresident fellow at the Quincy Institute, said that "hugely consequential decisions" about the role of AI in U.S. military technology and operations "cannot be driven by narrow considerations of corporate profit."

"Neither taxpayers nor the Congress should buy the hype surrounding these new technologies without careful oversight and scrutiny," said Brenes. "Otherwise, we will see yet another round of cost overruns for systems that do not work as advertised."

"With defense startups growing in number, and enticing military and political leaders, it will be exacerbated in an era of 'big tech.'"

The new report comes amid sustained outrage over the U.S. tech giant Google's AI partnership with Israel, which has used artificial intelligence in its devastating military assault on Gaza.

The report also comes months after the Biden administration announced its "Replicator" initiative, a project the Pentagon characterized as an attempt to counter China with an "AI-empowered military."

"Since we need to break through barriers and catalyze change with urgency, we've set a big goal for Replicator: to field attritable autonomous systems at a scale of multiple thousands, in multiple domains, within the next 18 to 24 months," Deputy Defense Secretary Kathleen Hicks said in a speech last year.

Hicks' remarks drew immediate alarm from watchdog organizations, which have criticized the Pentagon's lack of transparency surrounding its AI efforts.

In March, a coalition of groups spearheaded by Public Citizensent a letter to the Pentagon warning that "autonomous weapons are inherently dehumanizing and unethical, no matter whether a human is 'ultimately' responsible for the use of force or not."

"Deploying lethal AI weapons in battlefield conditions necessarily means inserting them into novel conditions for which they have not been programmed, an invitation for disastrous outcomes," the letter reads. "'Swarms' of the sort envisioned by Replicator pose even heightened risks, because of the unpredictability of how autonomous systems will function in a network. And the mere ambiguity of the U.S. position on autonomous weapons risks spurring a catastrophic arms race."

The Quincy Institute report specifically calls on Congress to "establish a revamped Office of Technology Assessment (OTA) that could provide oversight of the industry and ensure that Silicon Valley startups do not manufacture promises that cannot be delivered."

The report also urges Congress to shutter the revolving door between the federal government and military contractors, which gives private companies further influence over consequential policy outcomes.

"This is not a new problem," the report acknowledges. "But with defense startups growing in number, and enticing military and political leaders, it will be exacerbated in an era of 'big tech.' Republican Representative Mike Gallagher recently announced that he was joining Peter Thiel's Palantir after resigning from Congress. This is while Gallagher promotes belligerent views on China in mainstream outlets like Foreign Affairs, arguing that the United States is in the throes of a 'New Cold War' with China that must be won by 'rapidly increasing U.S. defense capabilities to achieve unmistakable qualitative advantages over Beijing.'"

"It will be up to interested members of Congress, working with the administration, to craft specific proposals and regulations to manage the role of private money in the development of emerging military technologies," the report states.