1. Fanatics do no like free speech.
2. Fanatics like government powers that limit their opponents.
3. Fanatics use, or advocate the use of, violence to enforce the first two points.
4. Fanatics always have an enemy they are fighting against.
That is exclusive to authoritarian extremes on the left and right that seek wider social change.
Libertarian extremes can (in theory since they tend to get squashed by governments) end up with anarchists living on isolated plots of land and so long as you do not step foot on said land they don't really care (the crazy mountain man prepper whose expecting the race war/nuclear holocaust to happen any day sort). Or your more hippie types end up in communes (these have been tried but fail because human nature doesn't coexist well with the ideological extreme) also perfectly happy to do their own thing.
Where the problem likely comes in is that individual people have a limited capacity to make larger social change. You as a individual can physically only do so much, no matter how big, strong, or smart you are your individual capacity at achieving anything will always be less than a groups.
So those extremist ideologies who seek wider social change (as I've tried to point out, there's a lot of ideological extremes but not all seek to change all of society) necessarily need a collective of some kind and to maintain group cohesion a authoritarian bent is required, which leads to limited individual freedom (you're not going to establish your utopia whatever shape that takes if everyone has their own opinion on what this utopia looks like). Group cohesion is also supported with a "us vs them" tribal mentality, and this hooks into the desire for wider social change because the status quo likes being the status quo and inherently will give opposition to any change (and so we end up with violence being a means of changing or retaining the status quo).
Government simply provides an existing power structure that obviously if subverted offers an avenue for social change, so necessarily it becomes the main target for subversion to the ends of any ideology seeking wider social change.
So I think you're very narrowly defining "fanatic" to be exclusively applicable to a very narrow set of ideological extremes namely more authoritarian subversive ideologies that seek to change wider society. So yeah if we look at very specific ideological extremes on the left and on the right and look at them only superficially in terms of some of their actions then horseshoe theory fits... but that's exactly what I'm saying, it's a very limiting tool to understand extremes of the left and the right as it ignores a lot of the variety in the extremes and only looks at very specific things very superficially.
The core similarities between fanatics makes living under one quite similar to the average person even if the stated goals are different.
I refuse to believe that living under a right wing Christian ultra-conversative authoritarian regime is going to be very similar to living under the Tranny Communist authoritarian regime in anything other than a very narrow set of tactics used to enforce the status quo. Firstly I don't expect the Christian Ultra-Conservatives to force me to suck the tranny cock or for the Trannie Regime to flog me for not going to Church on Sunday.
You're perhaps viewing this incorrectly. You think that if all of society gets overthrown by a new status quo your life and immediate day to day will remain unchanged and only the part of your life that interacts with the current larger systems of society will be impacted (i.e. your private life remains unchanged while your public life is impacted and thus why you list public activities such as free speech, or political violence on the streets, or the galvanizing of public opinion against a common enemy). Sadly that has never once ever been the case in any authoritarian system that has ever been enacted.