Pedantry is used by people who often can't argue against the main point of your argument, and must attack it on a technicality. It's a stupid version of the Motte and Bailey tactic.
Well, if you're going to set the parameters of the debate such that "pedants" are specifically those pedants who engage in variants of the Motte and Bailey tactic due to a lack of better arguments, then
of course pedants will be annoying people who often can't argue against the main point you're making, naturally leading us to support the assertion that we hate pedants. (assuming, of course, that hate is the just and natural emotional attitude to adopt vis a vis annoying pedantic people who cannot argue the main point but must instead endeavor to derail debates on a technicality)
But pedantry is a wide and varied discipline, and pedants encompass all manner of skills, positions, and debate styles. Not every pedant engages in pedantry for the purpose of derailing threads, and even those who do, may not always do so for wont of any "better" arguments. They may be making a valid, though pedantic, point against your position, or they may be being pedantic simply because it's funny and they don't want to gild the lilly with more sophisticated arguments. They may even just be making conversation!
So while yes, technically, the answer to OP's question must be affirmative, as clearly someone else hates pedants (universally, as a group), I myself do not. And I don't think you should, either.
I hate
some pedants, and
sometimes the hatred I feel for that set of pedants whom I hate stems, at least in part, from their pedantry, but it all depends on the context.