Yes and no. The things traditional morality (western, Judeo-Christian, American, etc) considers a breach, or immoral, usually relate to some character flaw that is dangerous in public office.
Someone who cheats at the monthly poker game with the guys is likely to cheat when he gets put in charge of a trillion dollar budget. Someone who screams at their kids is likely to be bad with managing employees. Someone who avoids hard work and coasts through life is likely to avoid doing the hard work of governing. Someone who abuses animals is likely to discount the human cost of life-and-death decisions.
So for Bill Clinton, the problem wasn't immoral sexual appetites per se. But someone who cheats on his marriage is willing to betray the people closest to him; someone who lies about affairs is willing to, well, lie to you, even when the only thing on the line is his own image. Saying "we want someone who won't lie or have affairs" is shorthand for saying "we want someone who is honest and won't betray us".
It's not a 1-to-1 causal thing, but it's a good heuristic.
No, I don't expect every politician to be perfect, but discounting morality altogether leads to, well, Clinton and Trump. I like a lot of what Trump does, and that's my argument in favor of voting for him. But his morality and his character have reflections in his governing record too.