One of things about Vic being a devout Christian that strikes me a weird for the accusers is that they say he's not very approving of the gay lifestyle, which is true, it's a prerequisite of the faith to consider it a sin.
It's not even a requirement depending on whichever particular form of Christianity he believes in, and there's a number of churches which have a more 'we don't have a problem with it, we're just not a big fan of calling them gay marriages' view as well.
Because again, as far as we can tell, the only reason he's being called homophobic is because he's openly religious and didn't sign yaoi artwork, with the stated reason being he can't sign non-official things. And there's definitely room for an unstated reason to be 'stop forcing your headcanons, especially when it involves shipping underage characters or some form of incest, on everyone else' - which is also not homophobic unless you're a dumbass.
Basically, to me it makes a great deal of sense that these nebulous accusations would be coming from a former fan who felt betrayed. So now things that they previously loved, like getting a hug and a cheek-kiss from their favourite VA, are now nonconsensual acts directed at underage girls that are basically rape.
It really can't be stated enough how creepy, entitled and obsessive fans can get, and for all the shit fanboys get, fangirls are just as bad and often worse. So I think that at least part of the puzzle, the 'everyone has a Vic story/has known about the rumours about him for years', can really just be put down to crazy fans. That doesn't explain all the rest of it, like the actions of the professionals involved, but I do think it explains why there's this background noise of 'anime fans know that Vic's not a good guy' that they all seem to think bolster their accusations.
And when they find out it's all smoke and mirrors, like with any hoax, they'll just pivot to other reasons why they hate him and pretend they haven't been claiming the entire industry knew he was a sexual predator for over a decade based on fuck and all.