If Fat really retained everything he read, he'd remember that the Christian crusades happened hundreds of years before oil became a valuable commodity that reshaped the Middle East. If Fat retained everything he read, then he wouldn't constantly be butchering common sayings ('through a force of will' vs. 'through sheer force of will'). If he retained everything he read, than he'd be able to flesh out the above argument with details from the book. I do believe he's the slowest reader in the world, though, and that he's therefore very poorly exposed to literature. Notably, his pop culture references were all from visual or audible media: the Wolverines (80's band), the three stooges, Star Wars, teenage mutant ninja turtles... For the references that were adapted from books (Harry Potter, The Wizard of Oz), you know he watched the movie instead of reading the book. He doesn't read deliberately; he reads with great difficulty, and calling the fact that he has to reread the same page 3 times to remember anything 'deliberate' is a massive cope.
Also, only total douchebags call entry level jobs 'beneath' them. It lets you know he's a total ass to serving staff because they'd have to be beneath him, if they're willing to take such a menial job. It's possible to communicate reasons for quitting a job after one shift without being a dick towards the people who work that job. You say, 'My oppositional-defiant disorder meant I didn't work well with the micro-managing culture of retail,' or 'I needed a more challenging job to keep me engaged, so I really respect the people who can endure that kind of tedium.' He needs every statement he makes to let the reader know he's better than someone, and that's why he's so hateable.