Report Urges Curbs on Development of 'Deceptive and Dangerous' Human-Like AI

Tech companies are creating and deploying artificial intelligence systems "that deceptively mimic human behavior to aggressively sell their products and services, dispense dubious medical and mental health advice, and trap people in psychologically dependent, potentially toxic relationships with machines," according to a report published Tuesday by Public Citizen.

The report—entitled Chatbots Are Not People: Designed-In Dangers of Human-Like AI Systems—asserts that "conversational artificial intelligence (AI) is among the most striking technologies to emerge from the generative AI boom kicked off by the release of OpenAI's ChatGPT. It also has the potential to be among the most dangerous."

"The subtle and not-so-subtle design choices made by the businesses behind these technologies have produced chatbots that engage well enough in fluid, spontaneous back-and-forth conversations to pose as people and to deceptively present themselves as possessing uniquely human qualities they in fact lack," the publication warns.

The report continues:

Deceptive anthropomorphic design elements... are fooling people into falsely believing AI systems possess consciousness, understanding, and sentience. These features range from AI using first-person pronouns, such as "I" and "me," to expressions of emotion and opinion, to human-like avatars with faces, limbs, and bodies. Even worse, AI can be combined with emerging and frequently undisclosed technologies—such as facial and emotional recognition software—to hypercharge its manipulative and commercial capabilities.

This, the publication says, is happening "with little or no testing, oversight, and accountability—including in places no one expects them, like the drive-thru at fast food restaurants, sometimes without any disclosure to customers."

The report contains a series of policy recommendations including:

• Banning counterfeit humans in commercial transactions, both online and offline;
• Restricting and regulating deceptive anthropomorphizing techniques;
• Banning anthropomorphic AI from marketing to, targeting, or collecting data on kids;
• Banning AI from exploiting psychological vulnerabilities and data on users;
• Special scrutiny and testing for all health-related AI systems—especially those intended for use by vulnerable people; and
• Severe penalties for lawbreakers, including banning them from developing and deploying AI systems.

"The tech sector is recklessly rolling out AI systems masquerading as people that can hijack our attention, exploit our trust, and manipulate our emotions," Public Citizen researcher and report author Rick Claypool said in a statement. "Already Big Businesses and bad actors can't resist using these fake humans to manipulate consumers."

"Lawmakers and regulators must step up and confront this threat before it's too late," he added.

In July, the Biden administration secured voluntary risk management commitments from seven leading AI companies, a move that was welcomed by experts—who also urged lawmakers and regulators to take further action.

A report on the dangers of AI published earlier this year by Claypool and tech accountability advocate Cheyenne Hunt urged a pause in the development of generative artificial intelligence systems "until meaningful government safeguards are in place to protect the public."