Jacobins were considered a dire threat to America at the time by the Federalists, and Jefferson's enthusiasm for (and personal association with) a number of them was viewed as akin to treason, especially after the XYZ Affair and the subsequent Quasi-War. While actually arresting Jefferson himself would have been problematic, the Federalists went after a lot of his supporters under the Alien and Sedition Act, jailing them and attempting to silence them. This generally backfired, with the jailed critics writing more polemical material even from behind bars. Ultimately, despite the dire predictions of French threats to the United States, the whole thing fizzled out rather uneventfully. However, the Federalist crackdown on speech was so deeply unpopular that Adams was thrown out as America's first one-term President (a club Trump is probably about to join), and replaced with Jefferson. The Federalists more or less collapsed and were never a dominant party again. They formally dissolved in 1824 although they had more or less ceased to exist years before that.
It's somewhat ironic how beloved Jefferson is of modern libertarians, often considered right wing, who love quoting things like this famous one:
This kind of rhetoric owes more to the Jacobins than anyone, and it is pretty doubtful the people who quote it approve of those guys much. Jefferson was considered to be the equivalent of a modern day Communist in some senses, although there's been a weird realignment of policy views between what is considered "liberal" and "conservative" and what was then.
For instance, the belief in a strong federal government, by the Federalists, was considered "conservative." In fact, the Federalists would have preferred the executive branch be led by someone nearly a monarch. Meanwhile, Jefferson would be considered one of the wilder "liberals," at least among the most prominent politicians of the day. There were certainly more radical liberals. Now, the belief in a strong federal government is generally liberal, or at least left-wing (among those who are not outright anarchists), while opposing a strong federal government is conservative. So conservatives like Jefferson for his opposition to a strong federal government, yet they probably would not like the actual Jefferson if he were around today.