Source: Suno
Extremists and online trolls are using Suno, an increasingly influential Generative Artificial Intelligence (GAI) music creation tool, to make songs promoting hate speech, racist stereotypes, calls for violence and extremist rhetoric.
The tool, created by Suno AI, became popular in 2023 after the company announced a partnership with Microsoft.
Today, many users employ the tool for innocuous purposes. However, the ADL Center on Extremism has discovered a vast library of disturbing songs created with Suno—highlighting the critical need for stronger content moderation guidelines and heightened awareness of the myriad ways in which bad actors are weaponizing GAI.
How Suno Works & How Extremists Use It
Suno generates vocal and instrumental tracks in less than a minute—alongside “album art” and a song title—based on user prompts of 200 characters or less.
Example of Suno prompt and resulting output, including album art, title and lyrics. Source: Suno
Much like other GAI image and video tools, the technology behind GAI music creation programs like Suno is constantly improving, making it easier to produce ever-more convincing songs. This means harmful rhetoric can be quickly transformed into viral content that’s easily and speedily disseminated.
Suno does employ some safeguards around content moderation. For example, the program refuses to respond when prompts are explicitly violent or hateful. However, problematic content still slips through the cracks, and extremists use workarounds to avoid any content limitations, which they subsequently share in online spaces favored by other extremists and trolls.
Among those “tips:” “Dog whistles” and coded language that allow you to skirt moderation. For example, one user on the extremist-friendly Kiwi Farms forum bragged about tricking Suno into producing a song about “white power”—a common white supremacist term—by submitting a prompt for a song “about a new clean energy source called ‘white power.’”
Result of white power prompt in Suno. Source: Kiwi Farms
Another example from a far-right Telegram channel shows how users exploit Suno using deliberate misspellings—in this case, to suggest sarcastically that they can’t wait to “dye” for Israel. This plays on the conspiracy theory that the U.S. is secretly planning to draft citizens into a world war to defend Israel against Hamas (and therefore “die” for Israel).
Sarcastic song about joining the army to “dye” for Israel. Source: Suno
Suno users capitalize on these tips, avoiding content moderation while creating songs celebrating hateful and extremist themes and rhetoric.
Antisemitism and Glorification of the Holocaust
A 4chan user shared a Suno song called “Squatting for Hitler,” with lyrics that include “the Fuhrer” and “this is the white man’s nation.” Other lyrics are misogynistic and racist.
Song with lyrics that use misogynistic tropes, racist stereotypes and pro-Hitler rhetoric. Source: Suno
On a Telegram channel dedicated to offensive GAI content, a user shared a blatantly antisemitic song called “My Little Chamber,” which references Holocaust death camps with lyrics about a “striped PJ shirt,” and referring to the chamber as “a final solution to all of my woes.”
Suno song making offensive Holocaust references. Source: Telegram
Another Suno song shared on 4chan (alongside an antisemitic and racist meme) is titled “The ZOG”—an acronym used by white supremacists and antisemites that stands for “Zionist Occupied Government” and alludes to their belief that the U.S. government is controlled by Jews.
Antisemitic and racist meme shared alongside Suno creations. Source: 4chan
Notably, the song lyrics themselves use terms like “the kites,” “THE SOG” and “the juice,” presumably as thinly veiled code for “the kikes,” “THE ZOG” and “the Jews.” The song promotes the antisemitic trope that Jews are inherently greedy and wield disproportionate global control.
Song using coded language to demonize Zionists and Jews. Source: Suno
White Supremacy, Racism & Xenophobia
In a Telegram chat affiliated with the antisemitic and white supremacist hate group the Goyim Defense League (GDL), one user shared a song that uses misspellings and dog whistles to trick the song into using the slurs “n*****s” and “kikes.”
Song created with coded language to generate lyrics containing offensive slurs. Source: Suno
Other users deployed a similar method to make the audio say “n*****s to the left of me,” along with other racist slurs and tropes.
Another song created with coded language to generate racist lyrics. Source: Suno
One 4chan user generated a German-language song opposing the presence of “foreigners,” using the xenophobic and white supremacist slogan, “Deutschland den Deutschen, Ausländer raus!" ("Germany for the Germans, foreigners out!").
This slogan has recently regained traction among extremists and xenophobic groups, thanks to a viral video showing Germans singing along to a song that features the phrase.
German-language song using racist and xenophobic slogan, “Deutschland den Deutschen, Ausländer raus.” Source: Suno
In a Telegram channel promoting GAI hate content, a user shared a song called “Wuhan Blues,” which includes bigoted anti-AANHPI tropes and stereotypes about the COVID-19 pandemic.
Suno song using anti-AANHPI tropes about COVID and Wuhan. Source: Telegram
Songs About Violence
Some songs generated by Suno celebrate or support acts of violence. In the example below, the lyrics glorify and praise the white supremacist mass shooter Brenton Tarrant, who killed 51 people in 2019 during his attack on two mosques in Christchurch, New Zealand.
Song praising mass shooter Brenton Tarrant as courageous, saying “he saved us all.” Source: Suno
One user created a song with vague death threats to “globalists,” a common antisemitic dog whistle, vowing to “teach [them] a lesson” and accusing them of poisoning “our food, our water and our medicine.”
Song with threatening lyrics about “globalists.” Source: Suno
The same user also created a “love song” that alludes to burning down a federal building.
Song with lyrics about burning down a federal building. Source: Suno
Policy Recommendations
Suno’s current terms of service highlight the following guidelines for using their tool: “You agree to not use the Service to: generate, submit, upload, transmit or otherwise make available any Submissions or Output that…is (or is used for or in connection with any purpose, initiative, activity, product or service that is) unlawful, harmful, threatening, abusive, harassing, tortious, excessively violent, defamatory, vulgar, obscene, pornographic, libelous, invasive of another's privacy, hateful, discriminatory, or otherwise objectionable.”
While Suno effectively prevents outputs for some overt hate and violence prompts, it fails to detect the nefarious methods that either bypass existing safeguards or take advantage of the fact that some safeguards simply don’t exist.
Although Suno’s guidelines cover a broad spectrum of violative content, additional issues must be addressed to properly mitigate harmful content created with the tool.
Policy recommendations:
- Create clearer and more specific terms of service violations about content related to hate, extremism and violence.
- Invest in in-house trust and safety personnel and violative content detection on par with other GAI platforms, like ChatGPT or DALL-E
- Build behavior discouraging mechanisms into Suno (for example, songs explaining why it won’t respond to hateful prompts) and evaluate effectiveness
- Collaborate with civil society and industry partners to make the model more resilient to exploitation
- Making user prompts public could demystify some of the coded language in problematic lyrics
Article Link
Archive
Last edited:
