Because we actually have no intuition at all for what the world would look like if random people could just take off and fly. The Grand Canyon is an impassible barrier to humans and just nice scenery to a hawk. At-will flying is fundamentally intuition-breaking, trivializing all kind of barriers and designs, makes hash out of lots of premade dungeons, and generally makes running the game world more a pain the ass, because now I have to rethink things, and I can't just pull a module out of a sheaf and run it.
And yet, the players would probably object if I redesigned the game world on the fly so that things like security made sense, e.g. every walled city having tall watchtowers throughout with sharpshooters who shoot airspace violators on sight, so you can't just bypass the gate guards.