TikTok AI Avatars Generate Videos With Hate Speech and Dangerous Messages

TikTok accidentally published a link to an internal version of its new AI-powered digital avatar creation tool without any restrictions, allowing users to generate videos with any kind of content. CNN was the first to report on this, revealing that it was possible to create videos featuring quotes from Hitler and messages encouraging people to drink bleach, among other phrases. TikTok has since removed this version of the tool, while the planned, more restricted version remains available.

The digital avatars, called Symphony, were launched earlier this week and are designed to let businesses create ads using the likenesses of paid actors. This tool also uses AI technology for voiceovers, allowing advertisers to input scripts for the avatars to speak, as long as the content complies with TikTok’s rules. Access to this tool is normally limited to users with a TikTok Ads Manager account, but the version discovered by CNN allowed anyone with a personal account to try it out.

In a statement, TikTok spokesperson Laura Perez said the company fixed a “technical error” that “allowed a very small number of users to create content using an internal test version of the tool for a few days.”

When CNN accessed the internal tool, they were able to create videos with text from Osama bin Laden’s “Letter to America,” white supremacist slogans, and videos encouraging people to vote on the wrong day. None of the videos created by CNN had a watermark indicating they were AI-generated, which is present in the official version of TikTok’s Symphony avatars.

CNN did not publish the videos on TikTok, but Perez noted that if they had, the content would have been rejected for violating the platform’s policies. Although TikTok has already removed this version of the tool, questions remain about whether people will find other ways to abuse the digital avatar creator and whether TikTok is prepared to handle such misuse.