ChatGPT - If Stack Overflow and Reddit had a child

  • 🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.
Fun little nugget:

I typed the following into Bing's Chat GPT-4 client and got a fuck off message:
View attachment 5336280

I looked up Microsoft's privacy policy and they absolutely do fucking store basically everything you submit to them. It's funny to me that they have a hard stop coded in this way.

On further testing, any of those questions independently trigger a hard stop, lmao.

Edit: I'm fucking dying:
View attachment 5336290

The same thing has happened to me over the most basic technical questions. My last interaction with Bing Chat was when I was troubleshooting why sddm somehow uninstalled itself. Bing gave a bad response and it was like it knew that it screwed up. Before I even had a chance to say it was incorrect it preemptively gave me the fuck off message like it was expecting to get yelled at.

Now I run my own instances of stable diffusion for image generation, and gp4all for chat. And while I'm just beginning to learn this shit I can say with absolute confidence that it's completely worth the time and effort running your own ai models local and offline.

2023-10-20_08-35.png
 
The same thing has happened to me over the most basic technical questions. My last interaction with Bing Chat was when I was troubleshooting why sddm somehow uninstalled itself. Bing gave a bad response and it was like it knew that it screwed up. Before I even had a chance to say it was incorrect it preemptively gave me the fuck off message like it was expecting to get yelled at.

Now I run my own instances of stable diffusion for image generation, and gp4all for chat. And while I'm just beginning to learn this shit I can say with absolute confidence that it's completely worth the time and effort running your own ai models local and offline.

View attachment 5429166
Reading about it after the fact, it sounds like part of the reason is that early trials had the chat becoming abusive when it thought the questions sounded hostile. It would try to respond in kind, or escalate the negativity.

Also, I’m sure they’re very careful at Microsoft about the model producing anything that could be misconstrued as an official statement.

That said, I am also 100% convinced there are hard-coded no no responses that prevent people from learning more about how the sausage is made and what kind of data they’re actually collecting.
 
  • Agree
Reactions: Myrtle the Turtle
The same thing has happened to me over the most basic technical questions. My last interaction with Bing Chat was when I was troubleshooting why sddm somehow uninstalled itself. Bing gave a bad response and it was like it knew that it screwed up. Before I even had a chance to say it was incorrect it preemptively gave me the fuck off message like it was expecting to get yelled at.

Now I run my own instances of stable diffusion for image generation, and gp4all for chat. And while I'm just beginning to learn this shit I can say with absolute confidence that it's completely worth the time and effort running your own ai models local and offline.
What llms have you been using for local chat generation? Most of my results have been trash compared to online models.
 
Sorry if its been asked/answer before but what options are there for a system that you can "talk" to that doesn't save all your info and not require personal information (phone number for instance)? Novel AI even though its not in intended purpose?

Follow up, whats a good resource for understanding the world around this field of AI when people talk about models, and comparing samplers and all that jazz?
 
Sorry if its been asked/answer before but what options are there for a system that you can "talk" to that doesn't save all your info and not require personal information (phone number for instance)? Novel AI even though its not in intended purpose?

Follow up, whats a good resource for understanding the world around this field of AI when people talk about models, and comparing samplers and all that jazz?
While NAI, as far as I know (haven't been keeping up this year as much), is primarily focused on a story-telling format, it should support a more chat-based structure if you want. I believe it's also added in a beta-version of an Instruct Mode, for directly telling the model what you want it to do, rather than formatting it as a prose prompting. It's got a free trial, at least, so you could give it a shot, see how its new proprietary models stack up. It's presumably pretty good about not harvesting data, but I haven't gone back to /vg/'s /AIDS/ general to see if there's been any drama around it this year.

The only way you're getting 100% purely private, though, is local models. You'd likely want to run a front-end like SillyTavern or AgnAIstic (which I believe comes with non-local models you can pay for, too), along with a program to run the model, like KoboldCPP. You'll need to check out a place like huggingface.co to find and download the models to run. /g/ has /lmg/ (local models general) that's dedicated to this. Here's their guide on setting that up. Those places might also be good sources of info if you want to try to learn more, like how different models stack up through LAMBADA testing (though keep in mind most HuggingFace spaces will probably lead to a Discord server, and 4chan is 4chan. /aicg/'s guide, for example, is full of loli [to filter people, I presume])

Keep in mind the old adage of "fast, cheap, good: pick two" applies to AI models, with an additional "censored" thrown in. Local models require decent hardware, and you'll almost certainly get worse results than if you pay out the ass to OpenAI for GPT-4 (if they give you access/don't block your prompts for unsafe content) or Anthropic for access to Claude (same issue of "AI safety".) You can try getting into proxies to work around this, but that's beyond my ken.
 
I googled that and the first thing that caught my eye was "No GPU required". Can you give some more info on what exactly that thing is?
I've been running models from Huggingface locally for some time now and I always run into the GPU limitations when dealing with larger models. How come no GPU? What's the performance? Does this only load smaller models and dump them into RAM? Reading through the docs it seems like this is what it does, using certain quantized models fitting into up to 16gigs of ram. Would you be a dear and answer a few questions if not too much of a headache?

How many of the models have you tried?
How do they perform?
What's the response time? I don't fully understand how a model running in RAM on CPU can even remotely compare to one running in GPU. Is it slow?
Considering it's running 13B param models at most, is there any upside to using gpt4all instead of running those directly in the GPU since most of them run on my machine?
 
google bard is free, can link sources like GPT-4 and writes the best code overall but it'll shut down half your questions
bing chat is atrocious and hardly as good as either
Sorry if its been asked/answer before but what options are there for a system that you can "talk" to that doesn't save all your info and not require personal information (phone number for instance)?
bard is 18+ but photoshopping together an ID fooled google's age verification algorithms/indians for me
 
Not sure if this should go here or in a standalone post in Deep Thoughts or something.

The other day I was watching a seminar about the use of ChatGPT in education. Missed some of it, but the parts I listened to were the usual takes you can find in any tech article. "Oh, they're fantastic tools to help students with their writing. They can also write your assignments for you. (And students can also do those make-work assignments without ever having to engage their brains, which is apparently a problem?)"

Look at all the wonderful AI art - in the future people won't make art directly, they'll dial art up with "prompt-engineering". Don't learn how to draw, learn how to prompt engineer. It's the new field of art.

How do we stop students from using AI-chatbots for thinking unapproved thoughts? (Errr ... they can do that without help? I hope?)

-----
anyway.

I sort of wanted to rant a bit: Isn't the fundamental point of education that you take these skills and abilities and internalize them to make them yours? What part of teaching your students to outsource their brains to chatGPT is helping them do that? You're taking a path of least resistance in using a machine to generate busy-work, and then prompting your students to take a path of least resistance to overcome that busywork with another external agency. It isn't even something like a calculator that they own and control - it's a dependence on something owned and operated by a sinister corporation that's currently lobbying governments for special favors and limits on everyone else's research.

Learning how to use chatGPT to write things isn't the same as learning how to write.
Learning how to "prompt-engineer" for art isn't the same thing as learning how to draw (what an incredible trivialization of the freedom that actually developing artistic skill affords you to express something. Compress 255^3^numpixels degrees of freedom down to however many are in a short string of tokens.)
Learning how to use a calculator isn't the same thing as learning how to do arithmetic. (Or learning how to build a calculator, for that matter).

The vision of the world being pushed by what I suppose Norbert Weiner would have called the "gadget worshippers" is one where everyone outsources their thinking to computers and settles down into benign inertness after swallowing their superfluous brains like a sea-squirt (and the engineers of those AIs can become the Wizard-of-Oz source of some godlike voice of authority that must be obeyed (because who are you to gainsay the voice of authority? Especially after neglecting the development of your native mind?))

The vision of the last few real humans who think and exercise native skill getting John-Henried by endless interchangeable zombies that have outsourced their minds to external tools is an offensive one that almost makes me sympathetic to ideas of Butlerian Jihad. I don't think it's really possible, in general, until those AIs are legitimate conscious entities in their own right. But still ...

If you're going to use AI in education, you should be using it to train your brain. Flashcards and their evaluation instead of calculators. A chess opponent instead of using it in your place to play your side of the game. Etc etc.
 
The vision of the last few real humans who think and exercise native skill getting John-Henried by endless interchangeable zombies that have outsourced their minds to external tools is an offensive one that almost makes me sympathetic to ideas of Butlerian Jihad. I don't think it's really possible, in general, until those AIs are legitimate conscious entities in their own right. But still ...
So much of this griping comes from lazy teachers that just like to assign pointless busywork instead of actually teaching their students anything meaningful.
 
I googled that and the first thing that caught my eye was "No GPU required". Can you give some more info on what exactly that thing is?
I've been running models from Huggingface locally for some time now and I always run into the GPU limitations when dealing with larger models. How come no GPU? What's the performance? Does this only load smaller models and dump them into RAM? Reading through the docs it seems like this is what it does, using certain quantized models fitting into up to 16gigs of ram. Would you be a dear and answer a few questions if not too much of a headache?

How many of the models have you tried?
How do they perform?
What's the response time? I don't fully understand how a model running in RAM on CPU can even remotely compare to one running in GPU. Is it slow?
Considering it's running 13B param models at most, is there any upside to using gpt4all instead of running those directly in the GPU since most of them run on my machine?
Very slow on CPU, I'm getting around 10 tokens per second on a 13b 4-bit quantized model via llama.cpp CPU mode with a beefy i9 12 core CPU, down to 1 token/s for 7b. I suppose it's great if you have multiple AMD 96 core workstation threadrippers tied together on a single system.

While I can comfortably get up to 50 tokens per second and 0% CPU use for 13b models via exllamav2 on my RRTX 3090, I can't load anything bigger. 33b models are limited to 2048 context length while 65/70b won't load at all. VRAM is the limit to how big of a model I can use with a GPU.
 
Very slow on CPU, I'm getting around 10 tokens per second on a 13b 4-bit quantized model via llama.cpp CPU mode with a beefy i9 12 core CPU, down to 1 token/s for 7b. I suppose it's great if you have multiple AMD 96 core workstation threadrippers tied together on a single system.

While I can comfortably get up to 50 tokens per second and 0% CPU use for 13b models via exllamav2 on my RRTX 3090, I can't load anything bigger. 33b models are limited to 2048 context length while 65/70b won't load at all. VRAM is the limit to how big of a model I can use with a GPU.
That's what I thought. The way I understand it, LLM performance on GPU is on the edge of the limits of physics, while no matter how fast the PCIe(like 32gb/s max?) it's basically nothing. Add to that the latency and I don't see even multiple threadrippers expediting this process any further simply because of the bus limitations.

Maybe I should just give this thing a try and see what it's all about because right now I see no reason to run models that way if you have a capable 6GB/8GB+ graphics card. VRAM limitations is another matter because from what I saw in their website, they offer 13B models at most and these run (mostly)comfortably on 6gigs of VRAM.
 
Maybe I should just give this thing a try and see what it's all about because right now I see no reason to run models that way if you have a capable 6GB/8GB+ graphics card. VRAM limitations is another matter because from what I saw in their website, they offer 13B models at most and these run (mostly)comfortably on 6gigs of VRAM.
for 6GB/8GB, you could fit a 7b 4bit quantized model, you can run a 13b model too if you squeezed quantization enough. The Exllamav2 devs managed to fit a 70b model through 2.5 bit quantization on a 24GB VRAM card, though I'm not sure if it's actually useful.

There's also the issue of context length, Mistral being a 7b model for example is pretty impressive at handling extreme 32K context length, especially the Amazon tuned MistralLite. It easily takes close to 16GB VRAM for me if I made a 6bit quantized version for the full 32K context, that's a few GB more than the common llama-2 13b based 4096 context length model with the same 6bit quantization.

I'm very happy to have a gamer 24GB VRAM card, but even that is sometimes not enough. There are 80GB VRAM cards today from NVIDIA, but they cost $15K each.
 
I'm very happy to have a gamer 24GB VRAM card, but even that is sometimes not enough.
Yeah 24GB is a serious card. I'm getting by with my old ass 3070ti with 8GB and was considering an upgrade but bought a laptop instead. Maybe if I get more into LLMs i'll chip in for a better card in a few months but it felt retarded getting one for Overwatch
There are 80GB VRAM cards today from NVIDIA, but they cost $15K each.
They also have cheaper A100 Modules with 40GB going for $4k-$6k which should do the trick for "enthusiasts". But if you wanna train the models and play with them extensively you need the data-center grade cards and it's pretty expensive hobby tbh especially if you're not in the field professionally
 
  • Like
Reactions: Another Char Clone
preemptively gave me the fuck off message like it was expecting to get yelled at.
NGL i enjoy pretending the AI has a personality, the idea of the AI being like a shitty government employee or IT guy sort of warms my heart. this time next year i just hope microsoft lets it be honest and the message includes the pharse
Reading about it after the fact, it sounds like part of the reason is that early trials had the chat becoming abusive when it thought the questions sounded hostile. It would try to respond in kind, or escalate the negativity.
we really are living in the worst time in human history because up until a decade ago people would have fucking loved a chatbot that had a personality like that. an AI lolcow would have been a celebrity unto itself.
how to prompt engineer.
I really do wish people see AI art prompts the same way they see making cartoons digitally. no one complained about artists not needing to know how to draw on cels or use a pencil vs using a mouse. Just because the south park guys use computers to make the episodes instead of paper cut outs doesn't make it an inferior product. AI art is like game engines vs writing everything in assembly. the fact that you can learn the tricks to produce shit much greater than any pleb shows its a legit tool instead of something that can only be used to take people's jobs.
it's a dependence on something owned and operated by a sinister corporation
from how computing power keeps expanding you'll be able to buy a machine that can handle producing GPT-4 level results rather cheaply in a decade.
T to write things isn't the same as learning how to write.
Up until social media the average person read less than 50 words a day, and typed less than 10 words a day. If anything AI is really showing how much we need to up our language skills to get more use out of our tools. Just in general the ability to read and write has been used more in the last 20 years than ever before. Most everyone lived entire lifetimes without the ability to read or write, from ceos to heads of state to founders of religion or generals. Now we live in a world where the average human in the united states types more words in a week than Philip K Dick did while on speed.

Back in the 90s every student on the planet mocked the idea of ever needing to know how to compose a sentence or read novels, writing 300 words was more horrifying than being shot to most teenagers. now even the dumb ones are writing and reading more than literary critics did last century. Just by being forced to keep up with the jones even the blacks have gone from being our most illiterate race to having at least used their reading muscles more than ever before. if you showed a black student back in the 90s a 150 character sentence they'd start drooling now they read 100s of those a day for fun. The fantasies of nerds from the Bush era of an entire generation with a passion for reading and comic book characters and scifi has become a reality, but somehow despite having an understanding of literary tropes that only the honor society did back a generation ago its still not good enough for you.

even worrying about us getting dumber because of AI is a pipe dream just because the levels required to get the desired output is so much higher than what could have been considered a functional education back in the 90s. I guarantee you the next generation "raised on AI" will be talking like goddamn fraiser crane just because they'll need to to create the images and ideas they want to convey. think about it literally like assembly vs coding. You're the faggot whining about "mah precious punch cards" "mah precious steering fluid" because we finally have an easier way to do shit. You're the retard asking why we don't teach kids how to jack off animals despite it being necessary for the survival of human civilization (to help barn yard animals with procreation for us to later eat the offspring of) I guarantee you you're putting the cart before the horse, by the time a child has both the means and energy to feed his homework (which is proven to be ineffective besides as practice material and is commonly less than 5% of a students grade) into AI to answer they're already going be beyond an IQ deemed necessary to function in society and certainly beyond an IQ deemed necessary to graduate from any HBCU, meaning they're going to be extremely above average by that point and if we're honest doesn't have a fucking need to do the homework anyways.

Plus dipshits like you think we're all stupid niggers. Just because you and that woman you jack off to named lizzo aren't inquisitive and wouldn't bother trying to improve whatever AI models are put in front of them doesn't mean the rest of us wouldn't be trying to look into the source code or use different training material or just expand the capabilities of what our machines will be able to do. I've seen more non-art school queers get into their creative side in the last 12 months than ever before. Those stereotypical rough dads like Kevin's from the Wonder Years who'd mock their sons for having hair that violates army regulation now spending more time thinking about and creating artwork than ever before, the same way Newgrounds led to people discovering how much they enjoyed drawing and animating, a whole new group of people are appreciating art in a way they'd call you a faggot for doing decades ago. If your buddy drew you a picture and gave it to you, you'd look at him like he had downs even if it was good, now any groupchat has at least one guy showing off some creation. The same way image macros or selfies or food pics or reviews went from something niche that only artists might do to so normalized even the anti-intellectuals are doing it. we're seeing swaths of the population who would have never thought of making art especially ones they'd show off to their drinking buddies suddenly doing it as a fun activity.

8 years ago you needed $100k to get even 128GB of VRAM, now thats down to about $8k. Whatever machine requirements are needed to run chat GPT-4 now will be brought up by someone with even a bit more money and passion for free speech and modified so we can use it without those bullshit restrictions. Two of the biggest copes people have about new technology is either "it will ruin society!" or "what could we possibly use this shit for?" thankfully humanity has been able to figure out how to avoid the first while pleasantly answering the 2nd.
 

This microsoft paper apparently let it slip that GPT 3.5 Turbo is a 20b model. I always guessed it's a lot smaller than people claimed and it would explain a lot honestly, from it's speed and availability to their capability to even offer API access for next to nothing. The implication is that it would be possible to run a ChatGPT-level model on pretty average (for AI purposes) home hardware. It also would mean that OAI is leagues beyond any other company in making these models. Seeing mistral 7b by the original llama people it's also believable, as that model is pretty smart for it's size and I could see a 20b sized model of that architecture being just as smart if not smarter than ChatGPT. The llama 2 models apparently are just not.. very good.

Having GPT4 level language models on home hardware seems more and more likely.
 
Back