For me, the strongest argument against free speech has been that it's almost unprecedented, historically. The means, methods and targets change, but every system protects itself.
We are right, you are wrong it's always been.
I've always felt that in spirit free speech meant the ability to not have a good faith viewpoint restricted, not the restriction of anything literally defined as "speech"—especially when taking
intent into account. What I mean is, it's consistent to interpret criminalizing the yelling of fire in a movie theater when there's no fire as
not a violation of free speech, because it's an intersection of speech and intent that is explicitly for the purpose of causing a commotion. There's no debate here.
Even so, this realistic and theoretically concise definition becomes problematic in practice because we tend to "mindread" our political opponents as not arguing in good faith when they say something we deem controversial. Using my own definition of free speech, criminalizing holocaust denial is indeed a violation, but if you look at the arguments of ANYONE who justifies those laws, in addition to other things, they're heavily predicated on the explicit belief that 100% of holocaust deniers/revisionists are bad faith actors and foaming at the mouth anti-semites who deep down KNOW they're wrong.
Personally, I know for a fact that the latter belief is mostly not true; in my experience, holocaust deniers genuinely believe what they're saying. The rest is true just sometimes. Yes tons of them are Neo-Nazi spergs who hate Jews, but functionally all of them are legitimately misinformed about the more nuanced details of the Holocaust that explain away those apparent "discrepancies" that tend to lead to revisionism when ruminated on too much. Confirmation bias is also not really the same as knowingly lying.
On a broader level concepts like "hate speech" are pretty fuzzy, and can fall into the "fire in a movie theater" category, but also can fall into "technically true, but has offensive connotations". Stating that Jews are overrepresented in various institutions is an objective fact—stating "jews control X" sometimes just MEANS that, not per say an organized conspiracy, let alone that most Jews are involved or accountable, but many people do not interpret that as a neutral statement.
Admittedly, it frequently isn't, so I'd imagine it's theoretically possible you can get arrested for hate speech for simply stating a fact. Pure utilitarianism is used to justify restrictions on free speech, and the issue is so many ideas can theoretically be used as a justification to commit violence independent of whether they're factual or logically consistent or not. Invariably you're just gonna restrict the truth if you believe people's lives are at stake, and unfortunately, from a utilitarian perspective, you wouldn't necessarily be "wrong".