GTAIII
Best: it was the first real open sandbox game and gets points for that. Not to mention being involved with a gritty NYC mafia scene.
Worst: The fact that Claude says nothing at all and spends the entire game as just an errand boy with no real agency of his own. And the limited radio station soundtrack.
VC
Best: The Ambiance. The radio stations, the look, the sheer feel of the game was like being in an episode of Miami Vice. It remains the standard to world immersion.
Worst: How limited it really is in size and options. If it had the outfit and car customization later games like SA did, it would be perfect. And the aforementioned inability to swim.
SA
Best: How CJ feels like a real character who wants more than just being a gang banger and how he has to deal with his brother who doesn't. Not to mention the betrayals by his friends Smoke and Ryder.
Worst: The game feels like it doesn't know what it wants to be and meanders. Is it Boyz N The Hood? Ocean's 11? A UFO movie? It just wanders all over the place like it lacks focus.
Liberty City Stories
Best: We get to go back to LC and see the subtle changes that happened between this game and GTA III
Worst: Some of the controlling when it was ported over to PS2 made it feel clunky and unresponsive at times.
Vice City Stories
Best: The same thing with LCS, just noticing the subtle differences in town between 1984 and 86, not to mention the ambiance of stepping into an episode of Miami Vice
Worst: The same clunky control issues LCS had. Plus the opening cut scene in VC had Victor Vance speaking with some weird Jamaican accent, but in VCS he is an American. It just nags at me.
GTAIV
Best: A larger and more expansive and realistic version of Liberty City and how Niko has to interact with it and his moral choices along the way.
Worst: GTA seems to have lost the tongue-in-cheek satire stance around this time in favor of some kind of realism mirror. At least there feels like there's a lot less of it.
The Lost and the Damned
Best: Mid-mission checkpoints, which saved so much time grinding or getting around compared to previous games.
Worst: Johnny cuts ties with Ashley and the gang...just to end up getting his head kicked in by Trevor a few years later.
Ballad of Gay Tony
Best: I just liked the throwbacks to the other games, like how Johnny and Niko both show up
Worst: I can't think of anything except how small the game is. It's only like 12 hours or so long. As a side game that was to be expected, but it was fun so it seemed to go by so quickly.
GTAV
Best: The Stranger encounters and the side missions like finding the parts for the UFO guy and getting the souped up sandrail as a reward.
Worst: How much of the game map is just wasted space. The Air Force base, the mountains, the secret lab, stuff where a single mission takes place and then is just there or areas like the prison where nothing happens at all except you get a massive wanted level just for going over it.