I can say from experience that the current-gen Celerons/Atoms are really decent chips speedwise. They are pretty much able to do anything you'd want that's not gaming related. (light indie games and emulation of the older 16 bit systems are fine though) If you're not a retard and install things like ublock in your browser you'll get a snappy experience. In power consumption, they're a tossup with the high end ARMs and sometimes might even consume less.
The driver support is absolutely atrocious and pretty much all current ARM SoC manufacturers don't really care about anything besides shoveling their chips onto disposable android devices, because that's where the real money is. Almost all Linux driver support for ARM SoCs are the result of reverse engineering and usually, when some voluntary developers make some headway into a SoC, a new one comes out everyone flocks to and development is basically abandoned.
That list of fancy features the SBC seller is promising on his premium board? Forget about all of them. If you're very lucky, you might get 3D acceleration (that'll forever be subpar to even intel iGPUs or the performance of the same SoC on android) and hardware video decoding, usually not even of all the codecs the SoC supports. If you're lucky.
All the ARM desktop nerds have a serious case of stockholm syndrome. They celebrate things about their ARM SBCs as huge breakthroughs that are a complete standard not worth mentioning on every sub $100 celeron ITX board. Most of all, with x86 shit just works. The driver support is excellent and done by the actual manufacturer (even in Linux) and you're compatible to all x86 binaries. As desktop user, there's literally no reason to go with ARM outside Apple's stuff, if you can live with it.
I can't imagine intel's gonna change the game there.