Obviously, the caste system made sure that Brahmin kept their gene pool clean and superior, so that's one thing.
And as we learned from countless examples through history, hardships and harsh environments breed top quality people, so keeping their country carefully polluted and on the edge of habitability makes them stronger and more resilient than the lesser human races.