To be fair, it's been a while since I've seen a Lyle episode. Maybe he has improved, I wouldn't know.
That being said, Lyle obviously loves the sound of his own voice and he frequently inserts himself into Chris and Zach's bits when he isn't just talking over them. He can also be kind of mean-spirited. I suspect most of these problems arise when Zach is present. It's like he's threatened by him in some way so he has to compete with him. I've seen a few Lyle episodes without Zach and he wasn't quite as bad. He seemed more willing to play the straight man.
Dynamic is important. It's amazing how the presence or absence of just one person can completely change the tone of an episode. RLM's Best of the Worst has this same problem I think. Ultimately it's subjective but I do believe that Lyle and Corey just don't mesh with the other three. They are loud personalities that dictate the flow of the episode.