Liking anime and actually having access to far-off shows used to command a lot of nerd cred. Back in the 80s (and a little of the 70s), you either had to be connected via a fan group to get the stuff or just pay a bootlegger at a convention. There were also a handful of mail-order bootleggers, but it's not like they were putting ads in the New York Times. This was well before the rise of anime-dedicated cons.
And this doesn't even touch the subject of understanding what you were seeing. Back then, it was fansubs or squat, as companies like AnimEigo and US Manga Corps were still years away from existing. So, if you wanted the whole package, you needed to be a part of a fan group who, in turn, had access to fansubbed tapes, which you could beg for copies or simply watch during showings.
Now, most everything is handed to people on a silver platter and it's no longer "cool" or "fringe" to be into the medium. And with all that availability came all the degenerate downsides, like loli stuff. People began to realize that, not only was a lot of anime not for kids, but a certain segment would probably be illegal (at least, in the U.S.).
These are, unfortunately, cultural divides that will never be reconciled. The Japanese are gonna do what has worked for them for decades, no matter how much pushback they might get from American interests. There's a lot of repressed sexuality over there that has to get expressed somehow and anime is one of those outlets (manga, too).
The long and the short of it is that anime comprises both family-friendly Oscar-winning material and stuff that might land you in prison, with everything in-between. Just like anything else, really, but the cartoon and manga aspect, which has always been regarded as a kid's medium ever since Disney, makes stuff like loli particularly distasteful and offensive.
