I'd like to know what role optimism played in the formation of these zeitgeists as you describe them. Why are those things linked in a way that would make us able to form a coherent cultural identity then, and not now?
The death of monoculture has to do with this, I think.
It's already been mentioned, but (from what I understand) before the internet you basically went through public media or nothing. There was outsider stuff still, sure, but it was so niche and ineffectual to the world at large that it wasn't enough to change culture until some big suit found out about it and pushed it into the public sphere
/some outsider betrayed the culture and went full suit, then commercialized that shit. Everyone saw the same stuff at the same times and it created a unified narrative that was extremely hard to break. This culture also planned for people trying to break it, and created a prevailing narrative through the media it published that certain types of thought were bad and made you a nerd/danger/retard (see: demonization of communism in the Cold War, the Satanic Panic, etc.)
People were further encouraged not to rock the boat by being told everything was fine. Optimism, from what I can tell, has mostly existed in mass media as a tool of the media establishment to keep people happy even while shit is going down so that the establishment retains an income source. In the late 90s and REALLY early 2000s (like, 2000-first half of 2001), around the time of the Y2K bug and Columbine, you had shit like All Star on the radio and nonstop sitcoms on TV. Tons of poppy happy stuff, somewhat vapid, meant to keep people generally thinking good thoughts and looking forward to their futures as they ignored the oncoming specter of constant school shootings and the death of American industry. Even if there was still some dark or offensive stuff on TV, it was often tied into comedy to keep it light.
When the Internet came around and broke all that shit down with a bunch of blood, gore, memes, and sarcasm, the mass media started trying to adapt and lost a lot of the vapid happiness that'd characterized it before. That's part of why you got piss filter modern shooters, dark gritty reboots, constant military shilling and sadder whiny songs on the radio. (The other part of this was, obviously, 9/11 and the neverending Middle Eastern wars, but that mostly died out (in culture) past the early 2010s while all of this depressing shit lived on.) Then the Internet went and
killed mass media's cultural relevance, but it simultaneously infected itself with algorithms that made everyone see a different version of itself.
That leaves us with a fractured zeitgeist mostly dominated by the Internet, which is no longer the same centralized force as it once was
(go back 20ish years and everyone will mention the same few Newgrounds animations and games, the same handful of YouTube channels and YouTube Poops, the same damn search engine without any personalization; talk to anyone now and they'll say the same few sites, but neglect to mention that all of these new sites personalize your content so almost nobody watches or sees the same shit anymore) and defined itself by deconstructing old media without teaching itself how to reconstruct anything out of it.
The best attempts the Internet ever made at reconstructions were fanfictions or shitposts, which is why the modern media industry is now dominated by fanfiction and shitposts. The people being hired by the modern industry are hired in an attempt to compete with the Internet using its own tactics, which is why everything is now the same millennial-humor slop with lesbian couples and zero originality. Major media networks are now hiring literal ao3 writers because society as a whole has become so dominated by the Internet that anything unlike it is unattractive to the masses.
I think I'm really starting to ramble here, so the long and short of it is:
Optimism mostly existed in media, no matter the worldly circumstances of the time,
as a way to generate constant revenue even while the world was going down the toilet. It
played a part in forming zeitgeists by clouding peoples' memories with rosy visions of sitcoms and dark humor to coat over the bad stuff like children being shot at school or almost every electronic planet on the device being at risk of complete malfunction. It essentially
sold people happiness to keep itself afloat, and
unified people through distributing the same "family-friendly" content to everyone at the same time in the same places.
The Internet fundamentally broke this distribution just by existing, and went on to deride the entire concept of selling people happiness as outdated and fake. It built itself on subverting mass media through gory Flash shorts, constant parody, and shitty fanfiction. This became so fundamentally the Internet's "thing" that now, where we reside in a future in which mass media is essentially
dead or a pale imitation of internet media, we have completely forgotten how to create.
The Internet saw no reason to be optimistic, because it had nothing to sell. Its profits did not rely on keeping the masses happy, and its reputation only relied on keeping the masses informed. It was the "real" alternative to the "fake" normal of optimistic mass media, and now that the alternative has completely overtaken the normal we have lost sight of what normal used to be.
Nobody sees the same things anymore because the internet is meant to be a tool of information, while all its entertainment is meant to correct or "make real" the fakeness of (what used to be)
actual mass media entertainment.
(This correction often involved turning optimism and happiness on its head both as shock value and because it was generally more accurate to real events or feelings about certain situations.)
This has led to the modern internet, where everyone is "informed" but nobody is happy. Optimism is dead in media and the youth because the youth were raised on media that treated optimism as silly, childish, fake, or an illusion to generate revenue (which, in the sense that we're talking about (toxic positivity, the fake laughter of sitcoms, playing Smash Mouth while the twin towers fall as opposed to having a good attitude and hope for the future), was not exactly an incorrect assertion). It has remained dead because there is no financial incentive to revive it, no regulations forcing people to keep in line with what's acceptable for which audience, and almost no conception of a better future due to the entire last quarter-century being one long downturn in quality of life.
TL;DR OF THE TL;DR: Optimism in media is gone because there is no reason to keep it alive. It only ever existed in such huge swathes because networks and producers had huge financial incentives to entertain and soothe people (that incentive being their continued existence, given that media only existed in the first place to provide reprieve from real life's negativity). The Internet never needed such optimism to exist, and it proceeded to kill the one industry that did, leading to the death of optimism in media.
The Internet also supplanted the old methods of distribution and regulation that kept people happy and soothed by old media (strict television schedules, content regulation, radios playing the same thing on the same station at certain times, etc.) by becoming personalized to every user who engaged with it, leading to everyone seeing the current zeitgeist completely differently to everyone else. Hence, the death of monoculture: there cannot be one current culture when every person considers it to be something different.
(I hope this still makes sense, sorry for the enormous tet wall. I'm trying to elaborate on what I mean but i'm running on like 3 hours of sleep and a few months of terrible circadian rhythm so I'm sure it's just an incoherent mess lmoa.)