Crime AI Voice Simulator Easily Abused to Deepfake Celebrities Spouting Racism and Homophobia - vice snitches bc apparently having fun is not allowed

https://archive.ph/wdsWM

Who could have possibly seen this coming? AI image generators have been used to create non-consensual pornography of celebrities, but why would that same user base dare abuse a free AI text-to-speech deepfake voice generator?

UK-based ElevenLabs first touted its Prime Voice AI earlier this month, but just a few weeks later it may need to rethink its entire model after users reported a slew of users creating hateful messages using real-world voices. The company released its first text-to-voice open beta system on Jan. 23. Developers promised the voices match the style and cadence of an actual human being. The company’s “Voice Lab” feature lets users clone voices from small audio samples.


Motherboard first reported Monday about a slew of 4Chan users who were uploading deepfaked voices of celebrities or internet personalities, from Joe Rogan to Robin Williams. One 4Chan user reportedly posted a clip of Emma Watson reading a passage of Mein Kampf. Another user took a voice reportedly sounding like Justin Roiland’s Rick Sanchez from Rick & Morty talking about how he was going to beat his wife, an obvious reference to current allegations of domestic abuse against the series co-creator.
On one 4Chan thread reviewed by Gizmodo, users posted clips of the AI spouting intense misogyny or transphobia using voices of characters or narrators from various anime or video games. All this is to say, it’s exactly what you would expect the armpit of the internet to do once it got its hands on easy-to-use deepfake technology.
On Monday, the company tweeted that they had a “crazy weekend” and noted while their tech was “being overwhelmingly applied to positive use,” developers saw an “increasing number of voice cloning misuse cases,” but didn’t name a specific platform or platforms where the misuse was taking place.


ElevenLabs offered a few ideas on how to limit the harm, including introducing some account verification, which could include a payment up front, or even dropping the free version of voice lab altogether, which would then mean manually verifying each cloning request.
Last week, ElevenLabs announced it received $2 million in pre-seed funding led by Czech Republic-based Credo Ventures. The little AI-company-that-could was planning to scale up its operation and use the system in other languages as well, according to its pitch deck. This admitted misuse is a turn for developers who were very bullish on where the tech could go. The company spent the past week promoting its tech, saying its system could reproduce Polish TV personalities. The company further promoted the idea of putting human audiobook narrators out of a job. The beta website talks up how the system could automate audio for news articles, or even create audio for video games.
The system can certainly mimic voices pretty well, just based on the voices reviewed by Gizmodo. A layperson may not be able to tell the difference between a faked clip of Rogan talking about porn habits versus an actual clip from his podcast. Still, there’s definitely a robotic quality to the sound that becomes more apparent on longer clips.
Gizmodo reached out to ElevenLabs for comment via Twitter, but we did not immediately hear back. We will update this story if we hear more.
There’s plenty of other companies offering their own text-to-voice tool, but while Microsoft’s similar VALL-E system is still unreleased, other smaller companies have been much less hesitant to open the door for anybody to abuse. AI experts have told Gizmodo this push to get things out the door without an ethics review will continue to lead to issues like this.

 

[The handsome tard]

I cant wait for the government to be asked to step in and legislate this, aka, the public wont be able to use it but the elites surely will.

 

[Chilson]

Better Archive your deepfaked celebrity racial rants while you still can.

 

[ZazietheBeast]

NO FUN ALLOWED. Really hate this pro-censorship garbage that is being touted as a virtue. A man must be able to say the word nigger without getting mobbed.

But this is going to be a clear; "Rules for thee, not for me" as you can bet they'll deepfake your voice, use it as evidence and there is nothing you can do about it. Open source needs to be the way to go to keep our unworthy rulers from gaining even more power from the plebs.

 

[an retard]

I feel like making youtubers and videogame characters recite the total niggerdeath copypasta is probably the least dangerous use case for this kind of technology, but I guess we're too early to be dooming about that in news articles

 

[WAHbert Plant]

HAIII, IT’S ME TOM CRUISE, VAGINA POOP

 

[Pro. Memer]

Copying some of my comment from the 4chan thread

Also Vice wrote an article about it. "AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse" - https://archive.md/wPeTw
"In one example, a generated voice that sounds like actor Emma Watson reads a section of Mein Kampf. In another, a voice very similar to Ben Sharpio makes racist remarks about Alexandria Ocasio-Cortez. In a third, someone saying “trans rights are human rights” is strangled. In another, Rick Sanchez..."

Some examples, including the ones referenced in the Vice article.
https://vocaroo.com/17RdfEecyZ41 - Emma Watson
https://voca.ro/1o43MUMawFHC - Ben Shapiro
https://voca.ro/1lc2TEoGEFjJ - “someone ”
https://voca.ro/15r3Y5ZXhn1M - Rick and Morty

https://vocaroo.com/188adfpKIXaz - Trump
https://www.youtube.com/watch?v=9Xqw11NPC40 - Star Wars
https://vocaroo.com/1i0R1PfOcLof - David Attenborough

 

[Bland Crumbs]

I cant wait for the government to be asked to step in and legislate this, aka, the public wont be able to use it but the elites surely will.

Common Sense as read by Taylor Swift will be lit.

 

[XYZpdq Jr.]

AFE all day erry day

 

[byuu]

Now everyone who was ever recorded saying nigger can claim that it was just a deepfake.

This is the biggest moment in civil rights history since the March on Washington.

 

[mlanguishi]

I didn't used to believe in the tech singularity talk.. Now I think it was just a bit later than 2020 or so.

 

[Kermit Jizz]

They can try to stem the flow all they want, eventually the model will end up in the open and there will be nothing they can do to undo it once that happens.

 

[Breadbassket]

I didn't used to believe in the tech singularity talk.. Now I think it was just a bit later than 2020 or so.

We are still quite a distance away from any kind of technological singularity. Keep your eyes open though since technology often has a habit of not standing still.

 

[Nick Gars]

But this is going to be a clear; "Rules for thee, not for me" as you can bet they'll deepfake your voice, use it as evidence and there is nothing you can do about it. Open source needs to be the way to go to keep our unworthy rulers from gaining even more power from the plebs.

I definitely have mixed feelings about AI in general, it's honestly pretty fucking spooky thinking about just how much shit it can automate and make obsolete, and we're only seeing the infancy of it right now.

That being said, it is still absolutely something plebs need to continue having access to, because just as you've stated, it will definitely be used against us.

Copying some of my comment from the 4chan thread

Also Vice wrote an article about it. "AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse" - https://archive.md/wPeTw
"In one example, a generated voice that sounds like actor Emma Watson reads a section of Mein Kampf. In another, a voice very similar to Ben Sharpio makes racist remarks about Alexandria Ocasio-Cortez. In a third, someone saying “trans rights are human rights” is strangled. In another, Rick Sanchez..."

Some examples, including the ones referenced in the Vice article.
https://vocaroo.com/17RdfEecyZ41 - Emma Watson
https://voca.ro/1o43MUMawFHC - Ben Shapiro
https://voca.ro/1lc2TEoGEFjJ - “someone ”
https://voca.ro/15r3Y5ZXhn1M - Rick and Morty

https://vocaroo.com/188adfpKIXaz - Trump
https://www.youtube.com/watch?v=9Xqw11NPC40 - Star Wars
https://vocaroo.com/1i0R1PfOcLof - David Attenborough

AFE all day erry day

I was waiting for someone to post em. Thank you, this shit is fucking gold. My fucking chest hurts from laughing.

 

[XYZpdq Jr.]

I definitely have mixed feelings about AI in general, it's honestly pretty fucking spooky thinking about just how much shit it can automate and make obsolete, and we're only seeing the infancy of it right now.

That being said, it is still absolutely something plebs need to continue having access to, because just as you've stated, it will definitely be used against us.





I was waiting for someone to post em. Thank you, this shit is fucking gold. My fucking chest hurts from laughing.

my fav so far is Nixon
"the sign is a subtle joke..."

 

[CharlesBarkley]

I want a hot anime robot sex slave cept she sounds like Gilbert Godfrey so those moans mean something.

 

[XYZpdq Jr.]

and another

 

[Iron Jaguar]

Someone should do Mel Gibson saying nice things about the Jews.

 

[glue eater]

 

[Kendall Motor Oil]

I have a theory they wanted this to happen for publicity. The 11.ai faggot knew people were making the ponies read erotic fanfiction but only sperged out when someone accidentally used a generated audio file commercially.