Ironmouse's collabs with Connor have actually been a big boon for her since it gives people who don't necessarily watch her streams a good glimpse of her personality outside of her interactions with the other VShoujo girls. She's still profanity-laced as ever but the Connor collab works only because Connor is great playing the normie that shits on you and your takes (which is often the position he takes contra Joey/Garnt in their podcast) and is more than happy to counter and shit on her rather than go along with the "Hee-hee-pee-pee" jokes. It's like watching a great manzai routine, only unscripted.
Anitube has long been toxic for years and actually has a lot of crossover audience with vtubing fandom. I'd reckon a lot of the Gigguk hate will only echo the usual reasons people have pointed out about disliking him: he has shit humor, his anime takes suck, his content caters to normies, his previous
involvement with Flying Colors Foundation, etc.
In short, his brand of anime autism doesn't appeal to them - they like other brands of anime autism, the "better kind" of the DigiBro/SuperEyepatchWolf/Scamboli/insert-your-favorite-animefaggot-here variety. I've always just liked his content simply because he he's always been self-aware about how shit his taste is and is more than willing to make fun of himself. Which is more than what can be said about the rest of Anitube.
I'm of the mind that a lot of that has more to do with how much work and involvement the logistics of the in-person guesting is and the cost-benefit of another stint. There's really not much to be gained from a second guest stint and the first guesting on the podcast still rates as one of their most viewed episodes and was great advertising for Cover and its talents - along with establishing a working relationship with Kadokawa/Bookwalker. Cover got everything that they needed off that first guest stint alone. There's no point coming back to it.
Also have to add that the Shadowverse streams are paid promotions and that Gigguk has also been sponsored by Shadowverse before.