Better question; WHY would I want to get involved with people who think it's good fun to harass strangers?
Yes, it is. They didn't ask for you giving them shit. You have no authority to give them shit. They didn't pay for it. You decided to just give them shit.
Unlike you, they were smart enough to realize that screaming insults at someone isn't worth getting your head busted.
That's why when you attack, you don't hold back. This isn't a boxing match, after all. Use a weapon. Go for the nuts. Make it HURT. I wasn't wanting to beat them in a contest, I was trying to hurt them.
So someone deserves it for wearing glasses or being the wrong race? They deserve it for being related to someone you don't like? They deserve it because they engage in hobbies you don't like? You are not in the moral authority to harass people and not expect retaliation. Go up to strangers on the street and call their mothers whores. You'll not have people say "thank you sir, I will work on myself to improve that fact." You'll get ignored at best, and your ass kicked at worst. Both are richly deserved for harassing.
THAT is society. Not people screaming at each other and demanding no retaliation.