Temperature has a huge influence in genetics, though.
"Bergmann's rule is an ecogeographical rule that states that, within a broadly distributed taxonomic clade, populations and species of larger size are found in colder environments, while populations and species of smaller size are found in warmer regions. The rule derives from the relationship between size in linear dimensions meaning that both height and volume will increase in colder environments."
Size, of course, is positively correlated with IQ. The correlation breaks apart in extreme cases because both pituitary issues and Berkson's (pseudo)paradox.
View attachment 7213262
The reason is that all the brain is bigger, not just the parts necessary for motor control. And brain size is one of the best biomarkers of intelligence, together with NAA concentrations (this last one also breaks apart in extreme cases due to Canavan disease).
View attachment 7213277
But you would need probably at least 50 generations to start seeing the clear influence of climate on genetics. It is obvious that Karl is not genetically adapted to his environment.
At any rate, what you are describing is founder effect, which is usually classified as one form of allopatric speciation, like I said before.