I’ve been meaning to write this for a long time, but I’ve never been good at writing things on social media. I know it’s been a while since I posted anything, but I want to reflect on how
15.ai came to be, share some of my thoughts, and talk about the future of the project.
The idea of
15.ai started as early as 2016 when I stumbled upon a paper written by DeepMind called “WaveNet: A Generative Model for Raw Audio” as an undergrad at MIT – I was 18 years old. That paper lit a fire under me. I didn’t just want to learn about AI voice generation – I wanted to push it further, to see what was really possible. I dove headfirst into it, fully convinced I could refine this technology and explore its potential like no one else had.
For three years, I worked on this project alongside my undergraduate studies, and in 2017, the famous Tacotron2 paper was released. In 2019, I gave a lecture and presentation about my findings, as I was able to replicate the results of WaveNet and Tacotron but with about 25% of the data they claimed was necessary (shoutouts to Dr. Edelman at MIT, he’s a great guy and none of this would have happened without him). I had originally planned to base my PhD dissertation on this work and bring that percentage down even lower (my extremely audacious prediction at the time was that you only needed
15 seconds of data to replicate a person’s voice; hence, the name
15), but when the startup I was working on with friends was accepted into the Y Combinator incubator on the very day I had to decide, I chose to enter the industry instead.
Fast forward about a year and a half, I left the startup in 2020 for various reasons. While I had made a good amount of money working in the industry, my exit wasn’t exactly on the best terms. I felt pretty angry that I had given up my dream of pursuing a PhD and becoming a professor for something that ultimately left me feeling unfulfilled. So, I threw myself back into research. I wanted to prove that the ideas I had once planned for my dissertation weren’t just credible – they were groundbreaking. But the grad school application cycle had already passed, and I wasn’t about to wait a year to apply.
Instead, I decided to take matters into my own hands. The best way to get my work noticed was to show it off. No gatekeeping, no barriers – just a free, accessible tool for anyone to use. I wanted to democratize AI research. I wanted to give people something that didn’t require coding skills or expensive hardware, something they could just use and be amazed by.
I got to work right away. I hacked together a functional frontend and backend for the website while scouring the Internet for interesting data sources, since well-known speech corpora like LJSpeech were boring. The whole point of the project was to prove that it was possible to replicate speech accurately with as little data as possible. Cloning a monotone voice that enunciates syllables slowly and coherently wasn’t all that impressive; real speech has complex undertones and nuances, and I wanted to capture that challenge.
That’s when I found the goldmine:
My Little Pony: Friendship Is Magic. I was familiar with the show – I had watched it when I was in middle school, but I hadn’t engaged with the fandom in years because of my studies. What truly impressed me was the dedication of the show’s fans from the “Pony Preservation Project”, who had compiled an extensive speech corpus unlike anything I’d seen before. Every single line from the show had been meticulously trimmed, denoised, transcribed, and even emotion-tagged. This was work that no other fandom had ever achieved at the time. (This was 2020 before any of that could be automated – this had all been done by hand.)
With this newfound data source, I found myself at a turning point. I realized that with this, I could not only push the boundaries of my research but also demonstrate the true potential of what this technology could achieve. I extracted the data from the PPP along with multiple other data sources that I had to manually transcribe (like the voices for GLaDOS, Wheatley, SpongeBob, the Narrator from
The Stanley Parable, etc.), trained separate models on the data, and hosted them on the website. The design of the website was intentional – while it was supposed to be very easy to use, I didn’t want my research to go unnoticed. That was why I had included a bunch of relevant numbers, graphs, etc. next to the generated audio files.
As I added more voices to the website, I realized it was possible to encode all the speakers into a single embedding, which would allow me to train all of the voices simultaneously instead of sequentially, saving me a huge amount of time on research and development. Near the end of 2020, I released a version of the website that added over 50 character voices to the website at once – a huge step up from the 7 or 8 or so I had previously.
Then, 2021 happened. The website exploded. It was all over Twitter, YouTube, and eventually, news outlets. Before I knew it, I was getting slammed by millions of requests every day. Autoscaling on AWS quickly turned into a nightmare, and as I watched the charges rack up, I realized I was in for a long ride. At its absolute peak, I was charged $12K for a single month (yes, you read that right), which included costs for training, inference, hosting, and everything else needed to keep the site running. But honestly? I was too stubborn to stop. I knew what I was getting into, and as a 23-year-old living alone, it was terrifying – but also kind of thrilling.
The attention came with offers – job interviews, acquisition proposals, you name it. I turned them all down. In hindsight, maybe not the smartest move, but I didn’t want to monetize the project or turn it into a job. I was afraid that would kill the joy I had for it. I just wanted to build something cool and keep improving it. So, I kept quiet and decided to focus entirely on expanding the list of characters and improving the underlying technology.
In early 2022, the whole Voiceverse NFT plagiarism thing happened, which pissed me off, but ultimately it didn’t do anything in the long run. So there’s that.
Then, in the middle of 2022, things started to go wrong. I received multiple complaints of copyright violations, and I received a cease-and-desist letter. I dismissed it as unimportant and chose to disregard it, since, technically, copyright law surrounding generative AI at the time was on my side. But due to certain other details that I can’t share here, I was effectively forced into stopping operations of the website immediately without warning or preparation.
I wanted to bring back the original website as quickly as possible, but my only option was to pivot to something that steered clear of copyright issues. That was easier said than done. I had built my reputation on doing things differently, on showing that I could take on challenges others wouldn’t touch, and now I was in a position where I had to tread carefully. It was frustrating as hell, but I knew I wasn’t going to let this project die – not when I’d come that far.
Looking back, I’ll admit I was a bit egotistical during this time. I thought I could handle everything on my own: the scaling issues, the legal headaches, the insane costs. I thought I was untouchable because, honestly, I believed I was doing something no one else could. And maybe I still believe that to some extent, because even now, I’m proud of what I built. But I can also see now that my stubbornness might’ve cost me. Maybe if I’d accepted a few offers or reached out for help, things could’ve been different.
Even so, I don’t really regret the core decisions I made. I wanted to create something that mattered, something that made people think, “Wow, someone really built this and gave it away for free?” And I like to think that I succeeded.
15.ai wasn’t just a tool; it was proof that cutting-edge technology and AI doesn’t have to be locked behind paywalls or reserved for corporations. It was a challenge to the status quo, and it was also a little bit of me flexing.
As for what’s next, I’m still figuring that out. The copyright issues, the shutdown – it all sucked, but it didn’t break me. I’m still working, still thinking about how to bring this back in a way that’s better, smarter, and maybe just a bit more sustainable. I have some ideas, but if I’ve learned anything from all this, it’s that nothing goes exactly as planned. So, I’m going to keep pushing, keep experimenting, and keep doing things my way. Because if nothing else, that’s what got me here in the first place.
Thanks to everyone who stuck around during the highs, the lows, and everything in between. Whether you loved the site, hated it, or just thought it was interesting, you’re part of what made this whole thing worth it.
- 15
P.S.: For journalists, researchers, or anyone else with questions about the project, feel free to reach out to me at
15@15.ai. I’m always open to discussing the journey, the tech, or whatever else you’re curious about.
