Consumer advocacy organization Public Citizen on Wednesday issued a new warning about the dangers of Sora 2, the artificial intelligence video creation tool released by OpenAI earlier this year.

In a letter sent to OpenAI CEO Sam Altman, Public Citizen accused the firm of releasing Sora 2 without putting in proper guardrails to prevent it from by abused by malevolent actors.

“OpenAI must commit to a measured, ethical, and transparent pre-deployment process that provides guarantees against the profound social risks before any public release,” the letter stated. “We urge you to pause this deployment and engage collaboratively with legal experts, civil rights organizations, and democracy advocates to establish real, hard technological and ethical redlines.”

Among other things, Public Citizen warned that Sora 2 could be used as “a scalable, frictionless tool for creating and disseminating deepfake propaganda” aimed at impacting election results. The watchdog also said that Sora 2 could be used to create unauthorized deepfakes and revenge-porn videos involving both public and private figures who have not consented to have their likenesses used.

Although OpenAI said it has created protections to prevent this from occurring, Public Citizen said recent research has shown that these are woefully inadequate.

“The safeguards that the model claims have not been effective,” Public Citizen explained. “For example, researchers bypassed the anti-impersonation safeguards within 24 hours of launch, and the ‘mandatory’ safety watermarks can be removed in under four minutes with free online tools.”

JB Branch, Big Tech accountability advocate at Public Citizen, said that the rushed release of Sora 2 is part of a pattern of OpenAI shoving products out the door without proper ethical considerations.

“The hasty release of Sora 2 demonstrates a reckless disregard for product safety, name/image/likeness rights, the stability of our democracy, and fundamental consumer protection against harm,” he said.

Advocates at Public Citizen aren’t the only critics warning about Sora 2’s potential misuse.

In a review of Sora 2 for PCMag published last week, journalist Ruben Circelli warned that the tool would “inevitably be weaponized” given its ability to create lifelike videos.

“A world where you can create lifelike videos, with audio, of anything in just a minute or two for free is a world where seeing is not believing,” he said. “So, I suggest never taking any video clips you see online too seriously, unless they come from a source you can absolutely trust.”

Circelli also said that OpenAI as a whole does not do a thorough job of protecting user data, and also questioned the overall utility of the video creation platform.

“While some of the technology at play here is cool, I can’t help but wonder what the point of it all is,” he wrote. “Is the ability to generate AI meme videos really worth building 60 football fields’ worth of AI infrastructure every week or uprooting rural families?”

Consumer Affairs also reported on Wednesday that a coalition of Japanese entertainment firms, including Studio Ghibli, Bandai Namco, and Square Enix, is accusing OpenAI of stealing its copyrighted works in order to train Sora 2 to generate animations.

This has spurred the Japanese government into action. Specifically, the government has now “formally requested that OpenAI refrain from actions that ‘could constitute copyright infringement’ after the tool produced videos resembling popular anime and game characters,” according to Consumer Affairs.

From Common Dreams via This RSS Feed.