It depends because not all aliens are equal.
We are aliens to them, if they exist, and there's not even a consensus on Earth. So maybe planet X has a 40% of the population being believers, and the rest not, while Planet Y having a different percentage.
Conclusion: If they follow basic rules of logic they should be agnostic, strongly leaning towards atheism or non theistic beliefs.
If they ignore things like solipsism, the brain in the vat theory or the limitations of empirical knowledge for obtaining truth, then they'll be biased, which shouldn't be that strange. At the end of the day, we're assuming the thoughts of living creatures that are exposed to this cruel world, maybe even more infernal than in the Solar System.