I really think the "worst day ever" was Nick finding out about her affair(s) and her blaming him by saying she was driven into the arms of another man by him being wrapped up in work and emotionally unavailable. He always had problems but ever since then the weird sex stuff got out of control, his already bad takes on religion went totally off the rails, and he became obsessed with scheduling/being busy/working too much. It just fits too well, IMO.
I take no position on how to blame Nick actually was, but the fact the a she seems to have been a pill zombie since atleast the first kid makes me think there were always deep problems that were ignored. Ditto for Nicks extremely casual relationship with the truth, it was always there - he just used to at least seem to feel bad about it.