i think another factor is that inner city blacks try very hard to emulate elegance and riches, in a very superficial way. they like their big coats and shiny chains and overly done hair, and swimming doesnt mesh well with that... its too... "simple", almost inelegant to them
and that goes back to what you said about them not doing it because they dont believe they should do it, they are very susceptible to the "this is WHO I AM this is THE CHARACTER I AM and i will behave ACCORDING TO THE SCRIPT" mentality