GF was always focused on the casual market. If anything the removal of stuff showed that they realised their core market was not big enough for them to waste the effort.
Actually, during Gens 3-5 you could tell they were on top of their game as far as gameplay mechanics and competitive were concerned. They made sure to include interesting Pokemon, great moves that complement said Pokemon, add mechanics that change everything completely and in general seemed like they were in-tune with the more hardcore side of the fandom. Even when they removed Battle Frontier in Gen 5, they at least added PWT, one of their better additions to date and some great fanservice, not to mention flipping Double Battles on their head by introducing Triple and Rotation battles. Nothing like that has happened since and they even mocked the fans expecting PWT to come back in one of SwSh DLCs.
I disagree they were always pandering to casuals: These were games for kids, but it was clear they were catering to the more hardcore side of the fandom as well, especially in Gens 4 and 5 since they knew a lot of the kids who played their older titles were now adults and were ready for something harder than the singleplayer in the competitive side of things.
Gen 6 is where the rot really started. There were far less Pokemon introduced from that point on, much less moves, nothing other than your run of the mill Battle Tower clone, no new changes other than Fairies and Megas. The last one would continue to be a theme here, rather than actually trying to balance the metagame anymore they just add ridiculously powerful gimmicks and make sure that the new Pokemon overshadow the ones added in the previous gens. Gen 8 broke the camel's back and they couldn't even be bothered to test everything accordingly, leaving us with dexit and moveset cut. It's clear that now they are pandering to the casuals, well that and Masuda openly stating they were chasing the mobile phone market during Gen 6.
It didn't used to be this way, there was a time when anyone could play Pokemon and get a worthwhile experience, not just the casual audience and kantosoys that get pandered to more than anyone else. There was a whole lot of care and love put into these games that you might not have seen, but I sure have. It's only starting with Gen 6 that they changed course like this, and it has less to do with them "realizing their core market was not big enough for them to waste the effort", more like they just stopped caring, period. It had to hurt knowing that no matter what, people will buy your games for the name alone, so why bother putting in any effort? Last time Masuda did, people cried that they replaced Pikachu with an ice cream cone. Give the menchildren their yellow rodent and make some shitty tech demo, it will sell anyways. The franchise has followed this bottom of the barrel formula ever since, you can trace it back to how fans reacted to Masuda's "magnum opus" during Generation 5(and oh boy were they loud enough for him to hear. He made sure to Kanto pander ever since)