Postmortem September 17th outage and rollback

[AngryAboutInternets]

all four enterprise raid NVMe harddrives had failed simultaneously

This is lottery-tier luck, except backwards.

 

[X Prime]

My wife and I got into an argument so I hit her over the head with the Kiwifarms hard drive. A grand number of post stickers lost.

Of course you'd be responsible for the Eclipse outage. I'm not sure why I expected anything else.

 

[Georgio Cocklord]

Thank you Sneedlord.

 

[Jaded Optimist]

I was worried when even tor was down.

Thanks boss

 

[TheGuntinator]

I am perfectly fine with nuking all the old stickers left on everything and starting fresh if they really cause that much trouble.

Problem with that would be all the highlights in every thread would be lost if all the dtickers were wiped

 

[Dire Dire Cocks]

Thanks, Jersh.

 

[Slav Power]

I don't know the terminology for drives, I don't even like touching them. They're WD M.2 1.6TB enterprise drives in a frontloading form factor which should not shit out unless I am seriously abusing them with writes continuously.

Hard drives are specifically magnetic platter drives. NVMe are pretty much always SSD drives.

 

[Pignat]

>All the autism from the site turned hard drives into tard drives

 

[200MB]

My reaction score will never financially recover from this.

 

[Balr0g]

Did Hillary send someone to have the hard drives commit suicide by magnet wave? Good to see us back and thank you for all the effort

 

[Pissmaster]

Problem with that would be all the highlights in every thread would be lost if all the dtickers were wiped

That‘s true, though they’ll regrow as newcomers read through threads and the rest of us resticker things we care about.

 

[KingCoelacanth]

Did someone accidentally knock over a server rack?

 

[Runescape Sperg]

But you know which servers ranked least in customer satisfaction? 9/17 servers. What a horrible name for a server, it reminds me of that tragedy

 

[Weebburner]

it only took about 7 hours, where most of that was just waiting on the database to import over 120,000,000 post stickers.

just fucking remove sticker already. Theres enough bitching about them already and if were seriously having extra downtime because some fucking retard wants to give tophat to folks hes mad at then just nuke that shit.

 

[Barry Williams]

NO

I miss when you could look at someone's profile and see a meticulous breakdown of each sticker they'd received and how many times, along with how many times they'd given out said sticker to others.

 

[JambledUpWords]

Can we add this react

 

[Maude Snew]

Kiwi Farms: back
Liz Dong: gone
the coping and seething is off the charts.

 

[Tiny Chinaman]

which one u niggaz chewin on the wirez?

 

[Consider Lizärds]

Nice work Null. I was minutes away from browsing Reddit. MINUTES.

 

[TexOffender]

Well glad it's back up. An entire RAID dying sucks.