Bad Dad Goku truly starts after Freeza when he refuses to return home because he's found new training. Not that it was not important and useful - no matter the outcome, a Goku that can't teleport can't get Cell out of Earth before it blows up - but it stands out more compared to other times.
His first training period was when he was 12 and had no family. Then he spent some time out in the wild to improve his tail after meeting Grampa and otherwise get stronger by age 15, still no family. He kills Piccolo and has to be trained by Kami-sama to fight Piccolo later, taking another few years. We then have the years of peace until Raditz shows up and he dies, now he HAS to be trained and can't meet anyone because, again, dead. He leaves Earth as soon as he's healed in part because his friends are in danger and while he trains on the way he's got time pressure.
When Porunga tries to bring him back, it's the first time where it's framed as his stubborn decision to leave family and friends behind for 'unnecessary' training - Freeza's dead, Freeza's people are dead, Vegeta isn't starting a rampage yet, and there's nothing on Earth to threaten him. So in a sense, there's no reason for Goku not to come back then, AND there's no reason other than him being bad at psychic powers to take so long to master teleportation and so on.
Something similar happens post-Cell, of course. Goku does care about his family and friends since he feels it's his fault everyone's been in danger, but he's also excited at being able to fight the strongest in history. And then you add that Goku projected himself on Gohan the one time he shouldn't and we get Bad Dad Goku.