Postmortem Postmortem for August 16th

  • 🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.
Status
Not open for further replies.
Did you do it on your first try or did you have to refresh a few times?
I kept checking back every 15 or so minutes and trying a refresh once on the 522 error page; url was straight to kiwifarms.net and not a subforum or anything
 
  • Like
Reactions: Dork Of Ages
Transaction backups would have helped here but possibly the upgrade wrecked them? They can help to turn a pretty big fuckup into a smaller one and have saved my ass a few times - who the fuck designs a language where no where clause just applies it to everything? Why couldn't UPDATE ALL IN `Table` have been part of the spec?

I don't know what's available for MariaDB - XtraBackup works nicely for regular MySQL and I've anecdotally heard of it being used at scale.

Also: it's a horrible feeling when that happens and I know it too well.
 
  • Informative
Reactions: Dork Of Ages
We should start a rumor that some dangerous/top secret information was leaked all over the farms and Null had to roll it back to hide it and just used the failed upgrade as a coverup. Youtube horror/conspiracy channels will call it "The Ghost Hours" or "The Lost Day" or something else super gay.
 
Null plans a couple weeks break from the farms...
raw.gif
 
Status
Not open for further replies.
Back