Postmortem Postmortem for August 16th

Status
Not open for further replies.
Did you do it on your first try or did you have to refresh a few times?
I kept checking back every 15 or so minutes and trying a refresh once on the 522 error page; url was straight to kiwifarms.net and not a subforum or anything
 
  • Like
Reactions: Dork Of Ages
Transaction backups would have helped here but possibly the upgrade wrecked them? They can help to turn a pretty big fuckup into a smaller one and have saved my ass a few times - who the fuck designs a language where no where clause just applies it to everything? Why couldn't UPDATE ALL IN `Table` have been part of the spec?

I don't know what's available for MariaDB - XtraBackup works nicely for regular MySQL and I've anecdotally heard of it being used at scale.

Also: it's a horrible feeling when that happens and I know it too well.
 
  • Informative
Reactions: Dork Of Ages
We should start a rumor that some dangerous/top secret information was leaked all over the farms and Null had to roll it back to hide it and just used the failed upgrade as a coverup. Youtube horror/conspiracy channels will call it "The Ghost Hours" or "The Lost Day" or something else super gay.
 
Null plans a couple weeks break from the farms...
raw.gif
 
Status
Not open for further replies.
Back