It arises naturally rather than being dictated. Compfags want to win tournaments, so they look at all the rules and see what seems best and follow in the trail of what seems to work. Since the game isn't really solved, it's perfectly possible for something good to go under the radar, or for someone to pilot a suboptimal list to a tournament win through some combination of luck and skill (a 40% win rate has just over a 1% chance to go 5-0 at an event on the face of it, after all), plus, of course, an established meta naturally having some openings for lists tailored around countering it that might not do well in another environment.
Granted, the video just came out so I don't know if this sentiment would hold, but that seems a bit of a dumb way to decide the meta if it's based purely on which faction won a recent tournament. It also seems like an easy thing to counter. Just bring anti-horde weapons.
That all ties into this - you generally need to win 4-5 games, depending on the tournament, to take a win, and given that it often comes down to points and there will be other undefeated players in the top few spots, you need to win those games with good point totals, without knowing what you'll be facing. So if there are a lot of Knights in the current meta, for example, horde Guard would be at its strongest since the average player would be putting more points into anti-armour and that's by design going to be inefficient into hordes and so the cycle repeats, although if something is sufficiently broken it can warp the meta around itself for a while, and you're also always going to be dealing with some baselines - half the tournament lists in various aggregators are some flavour of Marine, so it's almost always going to be meta to bring at least some reasonably efficient way to kill MEQ.
ETA: On the meta note, it's also worth noting that, like Trudeau, you don't win games by killing your enemies. It's about scoring points, and the meta will recognise that. Ususally, that will require getting opponents out of your way, but as an example of that not always being the case, you had a time in late 9e where you had a meta dominated by what were effectively two solitaire lists - both Necrons and Sisters could somewhat consistently score perfect 100 point games regardless of what their opponents did while barely having to interact with them, and that's about as close to the platonic ideal of a meta list as you can get.
The other thing I wanted to ask about is some random videos and comments complaining about Horus Heresy. The complaint that "it's just space marines" I get, but there are other factions like militia and admech, so you don't have to play SM (though I'd argue you should). I don't agree with the idea that other factions like Eldar, Orks, and Necrons should be included. I understand that they were around, but the game is about the Horus Heresy. It would be like having a new space hulk, and someone going "but I play imperial knights. I should have rules and a new model line for the game too!".
If you really want to play heresy era necrons, either homebrew some rules or play any of the other editions of the game. My understanding is that HH is just 7th+, so maybe the codex would be compatible?
This comes down, in my view, to why someone might want to play HH. It's hard to believe it's for the rules, given that 7e was the absolute nadir of 40k's game design and 8e's soft reboot probably saved the game from going the way of WHFB, but I suppose it's been enough years that there's now nostalgia for even that. Given that it's another game with a related ruleset, it's incumbent on Heresy to convince people to play it over 40k. It does this through two avenues - the Marine hyper-focus and tie-in with the Heresy novels, to which it seems that you don't object but which doesn't really appeal to me at all, and through being more focused on narrative than competition and more friendly to rivet-counters (to the extent that FW hired an actual rivet-counter to lead the design on the HH tanks) and those latter two points appeal to everyone, not just Marine enthusiasts, so you're naturally going to get some disappointment when you look to the last bastion of what FW used to do and see that it's purely Imperials vs Chaos - it's not that everyone has to be present at everything, as you mention, so much as that HH is all that's left of that style when Imperial Armour used to cover a plethora of different conflicts (along with other specialist games that generated their own lines) with a plethora of different factions, and now HH and its Marines vs Marines (where maybe Mechanicum and Solar Auxilia get to watch from the sidelines if they're lucky) forever is all that remains.
