#TIL on Kiwi Farms user pfps are a true and informative representation of who they are irl.
You're going to be terribly disappointed.
@Gargamel So, back in the day online videos used to work similar to downloading any other file in that the server would send you the entire video file, usually an mp4 though once upon a time you had .mov files, and it's ilk. Once your browser had enough of this file it would begin to play it and pause if it was running out of the file. And I am
mildly simplifying because you could do what is called a Ranged GET request which will just ask for part of the file but in general the principle was that video was a file transfer like any other. And if your browser asked for the 240p or 480p version of a file or, if you were better than everybody else the 720p resolution the server would send you the specific this_movie_480.mp4 file or whatever they had named it.
Simple and you're with me so far, I'm certain. Then streaming protocols were invented. There were advantages to this. Significant ones. The streaming protocols would send a manifest file first which was a fairly simple list of all the bitesize (not bytesize) chunks of video. Typically 5 second blocks. Each block would have a name, simple as blah_1.mp4, blah_2.mp4 and a time stamp showing which bit had what. If you open developer tools in your browser you'll often be able to see these manifest files named with the .m3u8 extension. Anyway, that's not that important. The essential thing to take away is that streaming protocols sent the video in many, many cut up little bits.
With this method, the server didn't waste bandwidth sending an entire video file if they only watched parts of it, skipping ahead to parts not yet downloaded became trivial, a server wasn't perma-tied up transferring data for a given user continuously regardless of how fast the user was actually watching them, instead it was a series of small requests with several seconds in between - and in computer terms several seconds is a very long time that lets you send out all the tiny video files all your other users are asking you for at the same time. And that handling of resolutions I mentioned earlier? Well they're still different files for different resolutions but now if a user suddenly clicks a higher resolution (or their browser automatically changes it based on bandwidth) then instead of now swapping between two large files you just send the next segment at the new resolution they requested.
So what's the downside of all this? Well you've just found it - browsers are now optimised around Just In Time requesting files as you watch, optimised for a world in which bandwidth is precious and you should only request what the user is ready for. In ideal conditions this works well - the video doesn't block all the other things your computer might be using the Internet for (or others on your network who share it with you), 5 seconds is lots of time for a computer to request the next block and get it without any visible delay to a human. And in theory you can cache all the segments anyway so if you do want a previous one, you don't have to go to the server, you just check in your disk cache for the already downloaded video segment. In non-ideal conditions you end up with an endlessly pausing video or long delays when you click to a different time.
You say your bandwidth is fine. I'll take that as an accepted statement from you but be advised there are lot of ways you can think things are fine but in some subtle way your network is now what you hope. But as this is occurring when jumping backwards I feel like your browser is probably removing its cached versions OR YouTube is deliberately flagging their files as having a short cache life (browsers trust what the server says about how long they should rely on a file before going back to the server to see if there's a new one).. Try adjusting the cache as others have suggested. It would be lovely if there were a way to adjust the default amount of video to buffer but short of redirecting the video stream to VLC (a media player that you can set a different default to) I've not found a way to do that. It's annoying but for anything you know you're likely to watch the whole thing of, using YouTube Downloader is actually very simple once you know how to use it.
Anyway, none of this will necessarily resolve your problem but your question was why the fuck does it happen, not how the fuck do I fix it. Good luck.