GPT-4chan - The AI is pretty sentient!

  • 🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.

chiobu

爪闩尺丂㠪ㄚ
kiwifarms.net
Joined
Sep 10, 2021
So basically you enter a text prompt and the GPT-? (no idea what version) AI will reply you 4chan style


1650957653311.png

:story:

screenshot-playground-beta.gpt4chan.com-2022.04.26-15_18_14.png


screenshot-playground-beta.gpt4chan.com-2022.04.26-15_16_17.png
 
It looks good at delivering a handful of distinct funny answers to the same prompt, like American Voices on The Onion or something.

I took the first thing I saw on /pol/ and here's what it came up with:

-----
--- 597104074
Americans destroyed Europe forever.

--- 597104588
>>597104074
>eternal anglo

--- 597106161
>>597104074
All this colonial power shit is so fucking stupid. By definition, if you were colonizing, it's not your territory.

--- 597106165
>>597104074
you mean we're just taking what we want?

--- 597106298
>>597104074
>we're just taking what we want

--- 597106304
>>597106161
>By definition, if you were colonizing, it's not your territory
 
  • Like
Reactions: chiobu
1650964012999.png


I only typed the blue highlight section (wanted to enter something absurd and see what the AI spit out). Oddly coherent...

EDIT: GPT-4chan seems to be broken on my end. Hoping it gets well soon (planning to feed some skitzo-ish posts from here into it).
 
Last edited:
NGL i'd love a blantant rip off of GPT3 without the obnoxious content filters. basically Tay 2.0 by limiting its responses the way they do it limits the ability to realize it as a real person. unless they want everyone to pretend all AI is southern baptist old ladies its bizarre to have someone that doesn't curse or swear or say sexual things or use offensive language. especially in urban areas.

imagine a 4chan-based AI designed to pretend to be a jew or black.
 
NGL i'd love a blantant rip off of GPT3 without the obnoxious content filters. basically Tay 2.0 by limiting its responses the way they do it limits the ability to realize it as a real person. unless they want everyone to pretend all AI is southern baptist old ladies its bizarre to have someone that doesn't curse or swear or say sexual things or use offensive language. especially in urban areas.

imagine a 4chan-based AI designed to pretend to be a jew or black.
I think you already know about the OpenAI autism towards their own creations being too powerful to release (another Elon Musk production). But I read about a more radically open competitor recently, probably on KF. I will try to find it.
 
I think you already know about the OpenAI autism towards their own creations being too powerful to release (another Elon Musk production). But I read about a more radically open competitor recently, probably on KF. I will try to find it.
its the one that AI Dungeon uses now right? say what you will about OpenAI their budget and time means they are probably much further ahead than any other competitor, the next best is probably only getting as much use because of Microsoft buying up exclusive rights to GPT3
 
its the one that AI Dungeon uses now right? say what you will about OpenAI their budget and time means they are probably much further ahead than any other competitor, the next best is probably only getting as much use because of Microsoft buying up exclusive rights to GPT3
Here is what I was thinking of: EleutherAI. And the post I got it from:

Article reads like an advertisement for OpenAI, written by Tyler McVicker, and with a healthy dose of copium over PaLM and a number of other LLMs dwarfing GPT-3 DaVinci in raw size.

No, the scaling hypothesis isn't dead, far from it. Yes, DeepMind's discovery that all current models are grossly undertrained is fucking huge. (Whitepaper attached.) It was also a paper published a few weeks ago, so there's no way it came up in that private chat with OAI 7 months ago, so it's him wishcasting (again) that OAI is anywhere near ready to leverage that finding and field a model with 4x the tokens used to train DaVinci. Not to mention that training at this scale is time consuming as fuck, so even if they were ready with a fresh, 4x larger dataset, right this minute, GPT-4 wouldn't be done training before Q3 this year at the earliest, so it's not going to be ready any time now. I'm not even going to bother with his previous retarded hype hallucination that GPT-4 would be a 100T parameter model, it's too stupid.

Also, he conveniently left off mentioning a number of actually open source LLM projects that are ongoing right now that are making rapid progress on democratising access to this stuff, namely EleutherAI and the BigScience team. EleutherAI has a number of pretrained models fully available to the public already (which can be run on local hardware if you're crazy enough, or run for free via GoogleCollab notebooks using the KoboldAI frontend and scripting), and BigScience is in the process of training a 176B parameter model that is directly comparable to GPT-3 DaVinci on the Jean Zay supercomputer in France. When it's done (should be July-Sept if things continue to go well), it will be released to the public for all to enjoy. (You can track their progress HERE, if you're like me and find load bars exciting.) Meanwhile, the PyTorch and DeepSpeed teams are working on a number of software improvements that relate to all this, that have the potentially to drastically reduce how much VRAM is needed to run these huge transformers.

Bottom line is, sellout Sam Altman can lick my DB-25 port -- his crew is no longer the cutting edge, and their monopoly's days are numbered.

After a year-long odyssey through months of chip shortage-induced shipping delays, technical trials and tribulations, and aggressively boring debugging, we are happy to finally announce EleutherAI’s latest open-source language model: GPT-NeoX-20B, a 20 billion parameter model trained using our GPT-NeoX framework on GPUs generously provided by our friends at CoreWeave.

GPT-NeoX-20B is, to our knowledge, the largest publicly accessible pretrained general-purpose autoregressive language model, and we expect it to perform well on many tasks.

We hope that the increased accessibility of models of this size will aid in research towards the safe use of AI systems, and encourage anyone interested in working in this direction to reach out to us.

Here at EleutherAI, we are probably most well known for our ongoing project to produce a GPT⁠-⁠3-like very large language model and release it as open source. Reasonable safety concerns about this project have been raised many times. We take AI safety extremely seriously, and consider it one of the, if not the most important problem to be working on today. We have discussed extensively the risk-benefit tradeoff (it’s always a tradeoff), and are by now quite certain that the construction and release of such a model is net good for society, because it will enable more safety-relevant research to be done on such models.

I am keen to see AI researchers who are not holding anything back and not sucking the OpenAI teat, although they are still paying lip service to "muh safety" and have a limit:
It is very unclear if and when such models will start to exhibit far more powerful and dangerous capabilities. If we had access to a truly unprecedentedly large model (say one quadrillion parameters), we would not release it, as no one could know what such a system might be capable of.
 
@The Mass Shooter Ron Soye -- Stupid reply bug. Open-source model alternative sperging below. I'm listing them in size from biggest to smallest.

1. BigScience 176B -- Still training, but should be due out sometime this summer/early fall. Will be about the size of GPT-3 DaVinci when done, which means the 16-bit slim weights will clock in around 300-350GB in size. Not currently feasible on consumer hardware*, but doable on a professional GPU cluster. (You don't need a massive datacenter to do inference, only training -- Latitude had an 8 GPU cluster powering their instance of DaVinci/Dragon, from what I remember.) NovelAI/GooseAI would be the services to watch for hosting this one, and if they do, it'd really make things interesting in a hurry. *(There is one possible way involving DeepSpeed, but I would expect the time per output to be unacceptable for entertainment use, plus it's a bitch to set up.)

2. GPT-NeoX 20B -- Eleuther's current largest model, available from their NeoX github. (Not up on Huggingface Transformers yet, but they're working on it.) Slim weights are just shy of 40GB, so doable on consumer hardware (2x RTX3090s), but it's a tight fit. Available with a friendly UI via NovelAI under the name "Krake", and API access via GooseAI. I think HoloAI may have it as well these days, but they've been lagging so I don't pay much attention to them. KoboldAI beta (United) is also an option, via Google Collab. Haven't taken this one for a spin yet as it benchmarks almost the same as FairSeq 13B below, and I can't yet run it locally. I've heard it can be better at some things due to its deeper knowledge base, but its prose tends to be dry and a bit sparse if you aren't familiar with steering it, vs FairSeq's more colorful writing.

3. FairSeq 13B -- This one was released out of the blue a few months ago by Facebook AI Research, of all orgs. Code and trained weights are up on their github, along with smaller versions, and the multilanguage XGLM. Some are also available via Huggingface. Your best bet to run the 13B version is probably via NovelAI (named "Euterpe") or a KoboldAI Google Collab instance, and using their repackaged versions of the model that's compatible with their interface. It's possible to run this one locally via the beta branch of KoboldAI, if you have a bare minimum of 32GB of GPU VRAM (or a shitload of system memory and patience). If you don't have that, but you do have 16GB+ VRAM, you can run the 6B version locally. I'm crossing my fingers they continue to release larger versions of this, or better, ones that take advantage of some of the cool improvements the DeepMind team have published the last few months (the "RETRO" and "Chinchilla" papers), because I've been very impressed with FS 13B so far -- the only thing I've played with so far that has beaten it has been DaVinci.

4. GPT-J6B (aka "Jax") -- Another Eleuther project, from March 2021. Available from NovelAI ("Sigurd"), GooseAI, HoloAI, and also via KoboldAI and Huggingface (and direct from Eleuther's github), for local or Collab users. Fair number of finetunes exist for this one, due to its age and relatively-accessible size. Generally considered semi-obsolete nowadays, as FairSeq 6B punches above its weight and requires the same hardware to run, but your tastes may differ, especially if you want a writing type that has a finetune available for J6B, but not FairSeq 6B.

5. GPT-Neo 2.7B (and smaller flavors) -- Eleuther's first project, now deprecated. Readily runs on local hardware (needs around 8GB VRAM for 2.7B, less for 1.3B or smaller ones). Available on NovelAI ("Calliope") and the rest of the places I've been listing, but I can't imagine why you'd bother paying a subscription for this one, as it's quite obsolete. A fun toy if you can run it for free, though. I wouldn't be surprised if GPT-4chan is running on this one, as it's cheap to host, but big enough to be trainable on following this kind of input pattern.

6. GPT-2 -- The great-grandaddy. It still exists, as it was before OpenAI sold out. Very obsolete, unless you like word salad.

its the one that AI Dungeon uses now right? say what you will about OpenAI their budget and time means they are probably much further ahead than any other competitor, the next best is probably only getting as much use because of Microsoft buying up exclusive rights to GPT3

Kinda. AI Dungeon lost access to GPT-3 (Dragon was DaVinci 175B, Griffon was Curie 6B) back in 2021, and had to scramble to find replacements for the golden goose they and OpenAI lobotomized. Currently, they have three main models -- Dragon, Wyvern, and Griffon. Griffon's GPT-J6B, and Wyvern is GPT-NeoX 20B, both by Eleuther, and open source. NuDragon is AI21's flagship model, Jurassic-1, not open source. From what I've read about that one (couldn't be fucked trying to get access to it to try for myself), it looks like it should be on par with DaVinci on paper, but in practice, isn't -- their training has biased it heavily towards doing Q&A type crap. GPT-J6B and GPT-NeoX 20B are neutral models that can be used for a greater variety of purposes, though Latitude has, as usual, managed to turn them all into drooling idiots with their shitty finetunes. Fuck I hate that company and its brainless CEO.

Despite their Microsoft backing, OpenAI's falling behind Google and DeepMind, though they're ahead by a mile in commercializing their work. DeepMind has RETRO (only 8B, but uses an external database as an aid that gives it DaVinci-tier benchmarks at some tasks), Chinchilla (70B) and Gopher (280B), Google has PaLM (north of 500B), and NVidia and Microsoft teamed up to do a half trillion parameter model I'm forgetting the name of, Megatron-Turing maybe, too lazy to look it up right now (it was mostly a proof of concept, don't believe they fully trained it). Chinchilla/Gopher and PaLM are solidly ahead of GPT-3 DaVinci in performance, but no one can actually access them. Annoying.

Chinchilla is amazingly efficient -- benches better than DaVinci at a fraction the size. PaLM is the most impressive overall, if you're interested and get a chance, Google's paper on it is fascinating -- the capabilities of these neural nets don't scale linearly. They go from "can't do XYZ at all" to "holy shit, it's great at it" for certain kinds of reasoning. For example, I've seen DaVinci spontaneously crack genuinely clever jokes involving wordplay and philosophy, but it couldn't necessarily explain to you what it did and why. PaLM can.

There's other stuff out there that has even higher parameter counts than these, like WuDao, but they're sparse models, not dense, so they don't perform at godlike levels like the parameter count might make you think at first glance. Still cool though.
 
Back