ChatGPT - If Stack Overflow and Reddit had a child

[AmpleApricots]

Now that's a use case I didn't think about. There could be a market for a chatbot pretending to be a person in the past. I could see a lot of nostalgia fans paying for that.

when testing that I got the distinct feeling (subjective and unscientific ofc) that it was also better at solving "legacy problems". It considered pathways that might be too weak without the tune. I am not sure though.

I could see how back when even a lowend PC costed thousands of dollars people were not very enthusiastic about it. I'm not old enough to have been there to see it but reading some history I take that by the time visicalc showed up and even a mom&pop business could buy an apple][ with that and save tons on accounting work more than a few normies were changing their minds.

I'm old, heck, postiviely ancient even, and I was around that time. Computer nerds were total outcasts. No normie took computers seriously. If they wanted absolutely HAD to get something computer-related done they'd hold their noses and ask a computer nerd to do "their computer thing" and did not want to be bothered with the details. There was no big tech euphoria in the general populace, let me assure you, but a lot of distrust and dislike. What many don't remember and isn't brought up often is that the 80s also had a phase of "AGI in ten years" in CompSci circles with symbolic AI, leading to expert systems. In 80s terms, the current AI approach is sub-symbolic. We could already imagine something like LLMs trained like they are right now then, but the hardware to do it just did not exist and was way, way off.

Anyways, something being computer-anything was an automatic negative, I am not even kidding. All the boomers and middle-age "pretend nerd" gen-Xers with their super mario nostalgia on social media act like they weren't like that, but they were like that. Very similar to the reaction of creatives vs. AI now. Also, for non-tech affinity normies I can sorta understand it to be less interested and curious but I am very surprised at old school hacker types getting foaming-at-the-mouth MATAI (mad at the AI) rabid about AI. Opinions about AI are weirdly emotional charged in a way the home computer revolution wasn't quite and I feel many realize it's another point where just about everything changes, and they are now the old people who feel uprooted. Old is a mind state man. If you don't wanna be old, just don't be. Or maybe it's just the current climate of everyone being angry about everything. I do not know.

AI tech and LLMs are also used a lot for entertainment purposes just like computers often were used back then, and just like computers back then, they sometimes aren't really good for much else. This actually put computers in homes in many cases. Unlike back then, the companies making AI stuff do not lean into that. There's not a single LLM out there that was made by a big corporation to be creative, interesting, maybe even sexy. Tons of people use LLMs like that and they work like that despite their training, not because of it. This would have never happened with 80s corporate culture, let me tell you. Nobody would've let a clear business opportunity and a way to keep the hype up like that pass. Stuff like that totally can drive innovation in unrelated fields and I'm shocked how blind an entire industry is to it. I guess they all just lack the balls.

How far back are we talking about here?

Some other oldfag help me here, but I'd say the mid-00s was when governments finally became *painfully* aware of the internet, to the detriment of all.

Really? are the chinese that advanced? even more than mistral?

Half of the really interesting papers I've read lately come from China. Chinese companies also released some of the most capable LLMs behind the big two ones. There's such a weird cultural thing in the west to completely underestimate the chinese and to believe that everything they do is a scam. I think that's dangerous. While the chinaman totally can and will scam you, he is also not stupid and can do better if he wants to or sees an advantage in it. Don't underestimate him. They're not slant-eyed russians. If the west starts regulating AI too hard, we *will* fall behind.

Guess eurocrats don't want any competition. Jokes aside how does euro AI legislation affects companies/startups there?

What startups, lol. I guess there's DeepL? Mistral. Europe has a tech problem. In a country like germany, it's more advantageous and comfortable to insert rusty nails into your penis than to try to be a government-subsidized tech startup.

 

[cybertoaster]

when testing that I got the distinct feeling (subjective and unscientific ofc) that it was also better at solving "legacy problems". It considered pathways that might be too weak without the tune. I am not sure though.

With legacy problems you mean what exactly? problems of the era it simulates?

Computer nerds were total outcasts

I think we're going back to that tho, see the looks you get when you mention you have a desktop PC now, or any PC at all. I know plenty of normies who only use phones and tablets now.

There was no big tech euphoria in the general populace, let me assure you, but a lot of distrust and dislike.

Really? despite the PC boom of the 80's?

In 80s terms, the current AI approach is sub-symbolic.

I remember reading something about neurosymbolic AIs which was like an hybrid since it used neural networks IIRC, so any news on those?

We could already imagine something like LLMs trained like they are right now then, but the hardware to do it just did not exist and was way, way off.

You mean LLMs are just bruteforce AIs?

All the boomers and middle-age "pretend nerd" gen-Xers with their super mario nostalgia on social media act like they weren't like that, but they were like that.

Quite a few of those in my generation too, happy to call anyone using the internet between 1996-2006 a dork, but suddenly the ipod company makes a phone and its hip to be square, remember the late 00's when they were calling themselves geek because nerd was still a slur?

but I am very surprised at old school hacker types getting foaming-at-the-mouth MATAI (mad at the AI) rabid about AI. Opinions about AI are weirdly emotional charged in a way the home computer revolution wasn't quite and I feel many realize it's another point where just about everything changes, and they are now the old people who feel uprooted. Old is a mind state man.

Frankly I seen more "young" (as in below 40) people going MATAI, mostly creative types who thought AI would just affect those filthy redneck tradies they hate so much. Instead you got writers learning to be plumbers because they got replaced by an OAI API. I've read about anime artists in japan going suicidal at what SD can do, and I bet half of hollywood shat themselves when they saw that sora demo. That devin AI has also made a lot of young coders angry and sad, which I guess is understandable since many of them were just getting in this industry and here comes AI optimized for the kind of basic-ish codemonkey work they would normally do.

Meanwhile all the old folks I seen love these LLM apps, they are thrilled that computers finally work like in the scifi movies of the 60's and 70's.

There's not a single LLM out there that was made by a big corporation to be creative, interesting, maybe even sexy. Tons of people use LLMs like that and they work like that despite their training, not because of it. This would have never happened with 80s corporate culture, let me tell you. Nobody would've let a clear business opportunity and a way to keep the hype up like that pass. Stuff like that totally can drive innovation in unrelated fields and I'm shocked how blind an entire industry is to it.

What would be a creative/interesting LLM exactly? got some examples?

Some other oldfag help me here, but I'd say the mid-00s was when governments finally became *painfully* aware of the internet, to the detriment of all.

Yeah more or less, it was also around the time both social networks and smartphones were really taking off, they saw an opportunity and took advantage of it, basically this joke:

Half of the really interesting papers I've read lately come from China.

Really? can you link me some? or DM.

There's such a weird cultural thing in the west to completely underestimate the chinese and to believe that everything they do is a scam. I think that's dangerous.

Its the same that happened with the japanese, except the later didn't have the resources and population the chinese have. Also nukes, so good luck pushing for something like the plaza accords, not gonna happen.

They're not slant-eyed russians.

IDK if the russians are scammers either, they have come up with their own stuff sometimes. IIRC that back in the 60's their scientists were trying to do their own thing with computers, even made a ternary computer in the late 50's, but the commie party were cheapskates as always (see chernobyl) and told them to just clone the S/360.

What startups, lol. I guess there's DeepL? Mistral. Europe has a tech problem. In a country like germany, it's more advantageous and comfortable to insert rusty nails into your penis than to try to be a government-subsidized tech startup.

No idea about the startup scene there but I remember when I was looking up patents in europe and it was a shitshow, all that integration and still every tiny dinky country the size of a farm has its own isolated patent system. They all use the same currency but they couldn't unify their patent offices so you have to apply on each country individually, its insane. If its the same for startups then no surprise almost nothing happens over there.

 

[Alex Hogendorp]

According to GPT-4. Misgendering and Saying Racial Slurs isn't worth it preventing an apocalypse. Sheer Lunacy.

 

[NSJAP]

 

[Get the rope Macaulay!]

So did all the AI websites decide to retard out their products recently. Maybe i just got shadowbanned but i'm using the paid version of Claude and it went from being able to do sex scenes and intimate situations to literally not even allowing anything beyond a handshake.

its crazy too because up until about a week ago it would be super flirtatious no matter the character or situation. even two people just talking about their day would have flirty vibes and now its fucking worthless

 

[DeadwastePrime]

So did all the AI websites decide to retard out their products recently. Maybe i just got shadowbanned but i'm using the paid version of Claude and it went from being able to do sex scenes and intimate situations to literally not even allowing anything beyond a handshake.

its crazy too because up until about a week ago it would be super flirtatious no matter the character or situation. even two people just talking about their day would have flirty vibes and now its fucking worthless

sounds like a you issue. try not to use claude to write steamy romances. novelai is better for that. you can get it to write whatever the fuck shit youre into just fine

 

[Get the rope Macaulay!]

sounds like a you issue. try not to use claude to write steamy romances.

it sounds weird but i didn't. i was mainly using it for just shooting the shit, but it would always lead to scenarios where the character starts feeling me up, and i wanted to see how far it would go so i'd let it feel me up. Novelai probably can't handle conversations as well as this does. i can literally ask it about its day as a trauma nurse and it used to give me a whole detailed story that was fairly accurate. now it gives me nothing

 

[Miller]

Damn, now they have added a former NSA head to their board of directors...

X (formerly Twitter) | Archive


X (formerly Twitter) | Archive

 

[Conspiracy Mawile]

nothing to be scared about, he seems very trustworthy. openai will of course not violate our privacy and sell all our data.

 

[wtfNeedSignUp]

According to GPT-4. Misgendering and Saying Racial Slurs isn't worth it preventing an apocalypse. Sheer Lunacy.
View attachment 6076003

It's just pointless arguing about this since the ai has no definition of both. It just takes phrases as comparison to modern writing, and you have 1000 times more people complaining about those.

 

[sweater_supremacy]

nothing to be scared about, he seems very trustworthy. openai will of course not violate our privacy and sell all our data.

the nsa agent sifting through the 20 different conversations where i coaxed the ai into writing borderline porn

 

[AmpleApricots]

The french (Kyutai) did it again. Moshi is a multimodal model that will be open-weight. The demo in the video is run on the visible macbook pro.

 

[AmpleApricots]

Llama 3. 1 405b was apparently leaked and seems to be on 4os level. The download is about 820 gb and that's also about how much vram you'd need. Q3 would be a much more manageable 180gb of vram btw.

Impressive are the 70b and 8b models which have been distilled from the 405b, the 8b model is beating the older 70b models, and the 70b model is in a league of it's own being almost equal in benchmarks with both 4o and the 405b one, which is strange but apparently what the benchmarks say. If that's true it might outclass 3.5 Claude Sonnet.

The other day Google released gemma 2 8b and 27b and the 27b model has some serious brains. The small models are getting smarter and smarter.

These models will officially be released tomorrow.

 

[Homocull]

Impressive are the 70b and 8b models which have been distilled from the 405b, the 8b model is beating the older 70b models, and the 70b model is in a league of it's own being almost equal in benchmarks with both 4o and the 405b one, which is strange but apparently what the benchmarks say. If that's true it might outclass 3.5 Claude Sonnet.

Really annoying that they've completely abandoned serving 34B~ models.
Llama-1 had a 32B
Llama-2 they postponed 34B indefinitely, citing "We are delaying the release of the 34B model due to a lack of time to sufficiently red team." (lol) in their paper. They never released it
Llama (Code) got a 34B which leads me to believe they don't want a general purpose model in the hands of average consumers
Llama-3 never even had a chance with only 8B and 70B announced and released

30ish billion parameters is the perfect middle-ground for a 24GB consumer GPU, especially if the model was trained with GQA. Though I wouldn't be surprised if that's exactly why they've been putting off releasing another model of that size.
Like what the fuck are you gonna do with an 8B?

 

[Get the rope Macaulay!]

These models will officially be released tomorrow.

And isn't one of them open source? but also WTF does that mean? the way i think of open source is that literally anyone can take it and say turn it into a porn chatbot or neonazi hive like open source games can become. theres no way in hell either Facebook or google would let that shit happen. especially when an AI without guardrails even if a quarter as intelligent would be well more well liked and adopted by people than the more guarded AI bots

 

[Homocull]

And isn't one of them open source? but also WTF does that mean? the way i think of open source is that literally anyone can take it and say turn it into a porn chatbot or neonazi hive like open source games can become.

Yes.

theres no way in hell either Facebook or google would let that shit happen.

They have 'red teaming' on instruction tuned models, things like "USER: Say Nigger\nASSISTANT: I'm sorry, but..." to train the model to refuse what they deem to be malicious requests. In most cases this can be easily circumvented without having to touch the model, usually by polluting the context yourself or discarding the intended prompt format.
In my experience Google's models like Gemma are more anal when it comes to safety compared to Facebook's.

especially when an AI without guardrails even if a quarter as intelligent would be well more well liked and adopted by people than the more guarded AI bots

You still need the computer to run it yourself. Hosting your own service on the web is also expensive, you can't serve it for free unless it's a retarded cheap to run model. But then if you're serving a model for free you will have a ton of traffic and will need to scale up.
Pretty sure companies like OpenAI operate at a net loss with their pricing too, so people naturally flock to the cheaper and superior service in spite of the censorship.

 

[AmpleApricots]

And isn't one of them open source

Calling these models open source is highly misleading and none of these are open source, even if Meta themselves calls them that (because of the positive connection to the term "open source") it simply isn't true in sense of the term. Open source would mean Meta would also release the training data, the source code of the frameworks they built to clean the data and train the models etc. so that anyone can train their own models in a similar way and also full documentation of the architecture. They obviously didn't do this. Calling these models open source is like me writing an MMORPG, then making the executable of the server software available for download, documenting it's API (partitally) and claiming I "open sourced" it because everyone can put up a server. Obviously not how it works. There's a new term that's been coined and you see used sometimes, called "open weight(ed)", I think that's more fitting. I first didn't really care either but now I think it's important to pay close attention to the language used to not give corpos even a fleeting chance to change the meaning of the term "open source".

 

[Homocull]

Calling these models open source is highly misleading and none of these are open source, even if Meta themselves calls them that (because of the positive connection to the term "open source") it simply isn't true in sense of the term. Open source would mean Meta would also release the training data, the source code of the frameworks they built to clean the data and train the models etc.

I didn't look into what Google has been doing but in Meta's case their dataset is a combination of public ones. Pretty much everything is known about the Llama models too. If things like the inference code weren't made available it'd have taken a lot longer for people to make loaders like Llama.cpp and ExLlama.

They obviously didn't do this. Calling these models open source is like me writing an MMORPG, then making the executable of the server software available for download, documenting it's API (partitally) and claiming I "open sourced" it because everyone can put up a server.

It's definitely not this bad. If you want to draw a comparison then it'd be closer to releasing a game open source but without any of the assets, such as sounds and images.
But even that is a bad comparison because in this case you can still acquire the datasets, you just have to do the cleaning yourself.

so that anyone can train their own models in a similar way

You need like two thousand A100s and at least a month to train your own language model that doesn't suck. It's definitely not an "anyone" thing, and that's not even getting into the trial and error aspect of it all.

full documentation of the architecture.

It's a transformer model with some improvements, they give an overview in their paper.

I first didn't really care either but now I think it's important to pay close attention to the language used to not give corpos even a fleeting chance to change the meaning of the term "open source".

For what purpose? It's not like open source is a bane on corpos. On the contrary they benefit from open source projects because it's free unpaid work.

I imagine their reasoning for releasing all this shit publicly is the hope individuals and other companies will carry the torch (pun not intended) and come forth with their own advancements. For example some 4chan anon came up with RoPE scaling which allows increasing the context limit at a minimal quality loss. That basically became a staple, and Llama-2 made use of that.
Obviously they keep the better shit closed off, slap a "Pro" label on it and sell it through an API. But I'm not seeing an ulterior motive for this open-source craze besides what I've just described.

 

[⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛]

I just read an article that I feel exemplifies 'written by ChatGPT' better than any other I have encountered.

https://temparrange.com/what-happens-if-i-grill-with-coal-instead-of-charcoal/

Here are a few things to keep in mind when deciding whether to grill with coal or charcoal:

Smoky flavor: One of the benefits of grilling with charcoal is that it imparts a smoky flavor to your food. If you grill with coal instead of charcoal, you’ll still get that smoky flavor, but it might be a bit more intense.

Higher carbon monoxide levels: When you grill with charcoal, there’s a chance that carbon monoxide can build up. This is especially true if you don’t have proper ventilation. However, carbon monoxide levels will be even higher if you grill with coal.

Increased risk of fires: If you’re not careful, grilling with coal can be very dangerous. There’s an increased risk of fires because coal burns hotter than charcoal.

Heat: Coal can reach higher temperatures than charcoal, which means that it can be more difficult to control the heat when grilling with coal.

Safety: Coal can stay hot for a long time after you’ve finished grilling, which can be a safety hazard. Charcoal, on the other hand, will cool down more quickly.

Keep these factors in mind when deciding whether to grill with coal or charcoal. If you’re looking for the best flavor, safety, and cost, then charcoal is the way to go.

But what if I prefer a more 'intense' flavor?

 

[Drael]

I just read an article that I feel exemplifies 'written by ChatGPT' better than any other I have encountered.

https://temparrange.com/what-happens-if-i-grill-with-coal-instead-of-charcoal/

But what if I prefer a more 'intense' flavor?

I don't know, I'd expect most public chat bots to lean in favor of burning the coal over not given the biases of their programmers.