Reality game shows switch their hosts all the time, they very rarely survive from doing so. It almost always ends up getting the show cancelled because people stop caring really quickly. When The Apprentice changed hosts from Donald Trump to Arnold Schwarzenegger the show was cancelled almost immediately. When Fear Factor switched hosts from Joe Rogan to Ludacris the show was cancelled almost immediately. People lost interest in those shows extremely quick because the personality behind the hosts changed the whole vibe of the shows.
I know I sure as fuck don't watch this show for Jet Neptune, so if Sam left the show I would 100% lose all interest. Imagine watching Hell's Kitchen without Gordon Ramsey, it would be retarded to even consider such a thing.