It's because they both are Valyrian. It's like saying Celtic or Nordic. They are not all related, but they have the same phenotypes. For Valyrian, is white, purple eyes, silver hair. If you being two typical families from Sweden, they're very likely are looking white and blonde and won't look like Slavs.
In the books, there is a Velaryon guy called Aurane. He looks so much like Rhaegar Targaryen that Cersei absolute loses her marbles over him and does something very stupid.
View attachment 5858260
This is how his ancestor, according to the Unseen Westeros exhibition looks like, Corlys the Sea Snake:
View attachment 5858279
We got this:
View attachment 5858280
The actor is good and all, but come on!