For a lot of lesbians, especially the ones who prefer butches/masculine women, they just can't really see FtM as males because feminine body structure is so apparent to them. The feminine quality jumps out, even with facial hair or without breasts. Just as many gay men seem to see HSTS MtF as skinny twinks, found them hot, and ended up focusing on the masculine body shapes and their brain bypass all the getup and bolted on tits. 'I've never been attracted to guys/girls except you' from these people is largely from politeness - these troons are likely just being clocked on spot by hornybrain instinct.
Basically, transphobic 'sex preference' goes both way.