Neoconservatives --- mostly (((neoconservatives))) --- were behind it. The "Project for a New American Century" document spelled out the goals and stated it was unlikely to happen any time soon unless there was another "Pearl Harbor" event. That PNAC document was actually just a reworking of a paper written for the Israeli Government called "A Clean Break: A New Strategy for Securing the Realm". All they did was rewrite with the spin that it would be in America's interests to pursue this plan.
Richard Perle led the group that wrote that report for Israel. Richard Perle was also a signatory on the PNAC document -- along with Dov Zakheim (who was comptroller of the Pentagon when Rumsfeld announced that 2.3 trillion dollars was missing from the Pentagon), Dick Cheney, Paul Wolfowitz, Elliot Abrams, Elliot Cohen, Scooter Libby, Jeb Bush, Donald Rumsfeld, etc.
Were the terrorists real and just used by these people to do the attack or were they merely patsies? No idea. Given Mohammad Atta's love of cocaine and hookers, I'm inclined to think that they weren't really fundamentalist terrorists killing themselves for Allah. They may have wanted to attack America, but I don't think religious fervor was the driving force. But, I think the government not only knew about it and let it happen -- they actively assisted them because it would make it possible for them to pursue their regime changes in the middle east at the behest of our BFF Israel and defense contractors (like Cheney's Halliburton) would make a shit load of money.
If the government didn't assist them (like having the Air Force stand down rather than engage the hijacked planes as is standard protocol), then our military leadership sucks ass and heads should have rolled over the massive amount of incompetence that would have had to happen to watch a hijacked plane approaching DC and not do anything about it, allowing it to hit the pentagon. That level of fuck up shouldn't even be possible.