Doesn't help that, in Fallout 4's case especially, the main quest is extremely personal.
This is why it feels much more egregious in Fallout 4. Your BABY is stolen from you, the SS is characterized as deeply caring about the child and almost everyone understands that the moral imperative to rescue a small child is more important than gunning down ghouls.
At least in Fallout 3 you can rationalize fucking around a bit because your dad is a grown ass adult who abandoned you. You're not the one with the moral imperative to find him, he was in the wrong for leaving you. On the surface of it, 'you have to find this person you like' is a good inciting incident for getting people out into an open world to explore, but it winds up grating on the back of the player's mind and reflects poorly on them as a moral judge of character.
New Vegas is smart in that the objective at the start is characterized through the lens of vengeance, or at the very least personal curiosity, which is something you will have to go out of your way to seek out. The world won't end if you take a stop in a town to do some odd jobs for extra pocket money. Benny might get away, someone might come after you to ask questions about the chip, but there's nothing that you can't shrug off at the end of the day.
But in Fallout 4 someone might be fucking dissecting your infant. That's always going to be niggling at the back of the player's mind and makes the SS look like a lunatic, even if you want to play them as someone who is morally upright.
At the very least, with Mass Effect 3, the sidequests are sort of part of the main mission. Just about everything you do bolsters the war assets a bit.
Except for stuff like dancing in the club. Shepard gyrates while Earth burns.
It's the same deal with both games, and you people just don't want to admit it.
Seething.
Next Fallout should start as you just being a generic vault citizen, the vault gets raided/malfunctions or whatever, and you're ejected in to the wasteland. No pre-amble, no macguffin, no lost relatives, just a blank slate with no background beyond what your own build and headcanon dictates.
I'd personally prefer being a generic wastelander since it offers more room for character history. I'd love it if the game had like a 'background select', maybe tied into classes, that offer you a potential background and different set of starting gear for your character. That'd be a lot of fun, I think.
I actually almost like that about 76, that you wake up after a big party in the Vault and just waltz out to begin your adventure in the open world. No obnoxious tutorial setpiece, no heavy-handed exposition, just you, your pip-boy, and the wide open world. Shame the rest of the game is such a shitshow.
Let the main quest line be something you can get tangled up with later, it doesn't need to be outlined and given a personal connection to your character before you start playing.
And for the love of God don't force a random backstory on the PC with the final DLC.