You know how people also use buddy? To talk to children much younger than them, and... to the mentally disabled. There is a good chance many troons had to be put in the special needs classes and got frustrated being talked to like they're slow in the minds, whether it was justified frustration (from condescending) or not (because they were actively stupid but were too stupid to realize it).
I am certain a lot of trans autists grew up being talked down to with autism because people didn't realize they were high functioning autists instead of mid-low functioning. The idea of a large hon flying off the handle if someone bends over his table to go "Hey buddy, what 'cha doing?" in a tone used for the retarded is hilarious.