It can't be ruined if you just enjoy the thing the way you want to and associate with people in it who think like you do. It's the rule of thumb that there is no fandom without toxicity in it, and if you like something you should probably avoid the fandom unless you know someone who can point you to parts of the fandom you'll actually enjoy. I'm not saying there aren't toxic elements in a given hobby and that they might not be something like 'I'm not involved in this because I like it, I'm involved because I want the poon.' I'm saying that I've never seen something actually ruined by those people existing in that hobby. Just people saying it's been ruined by them. I'd say the SJW-esque types are far more damaging, because not only will they go out of their way to make sure No One Gets To Have Fun, but if they're allowed to fester, they'll strike at the core material and try to rewrite it into their tumblr awful ways. Casuals and thots don't actually do anything inherently, They're just around enjoying the thing in their own weird little way, not hunting down people who do it differently or trying to wrest control of it from the makers so that they might destroy it.