What LLMs do is predict the next word (token) in a sequence based on a series of probabilities. It's very easy for people, even retards like myself, to understand how they work if you just put a little effort into actually looking it up.
3blue1brown has an excellent video on this subject. The math behind LLMs isn't all that complicated either. Chances are if you took College Algebra you can understand it.
People without computer literacy will anthropomorphize the output of LLMs because it resembles natural language. In reality it's a two-way search engine. We are far from general purpose "AI", and what modern LLMs are is a trillion dollar magic trick. I am not surprised at all that when you throw the entirety of the Internet (and more) at a computer model it is capable of producing said results.
It frustrates me to no end to see how overhyped and misunderstood this technology is. And just how much it's trying to be shoehorned into things where it's an ill fit. When you live in the funny money economy where nothing has any meaning anymore I guess that's how it goes. It's like everyone around you is constantly living in a manic episode.
Sad thing is that kids now use AI to do as much of their work as is allowed. I've argued with some online who feel this is fine. They don't have to learn anything because they can just ask the AI. So they lack the computer literacy and math skills to understand the AI's workings and they lack the critical thinking skills to see when it's totally wrong.
This is a similar problem to when calculators were introduced. Lots and lots of students never fully understand the four basic arithmetic functions of a simple calculator. So they can plug in 3 + 8 and find 11, but they have no idea how the plus sign relates those two numbers and combines them to 11. I've seen high school students at this level. 27/3 can easily be calculated as 9, but there is no understanding that that means the 27 got split into three pieces each worth 9.
So by the time they're in high school the teacher's expected to get them to grade level. So at 11th grade, that should be about precalculus where I was, or calculus 1 for the smarter ones. Let's say they're starting prealgebra, where I was in 6th grade. The problem looks like this:
Solve for x: 2x + 5 = 15
So you can hold their hand and get them to subtract 5 from both sides. That requires the calculator, of course, for 5-5 and 15-5. Then they have 2x = 10. At this point the student considers the problem done and is already typing in the next problem:
Solve for y: 3y + 9 = 21
So at the point the student is already writing 12 as the answer because they typed in 3 + 9. And if you tell them the first problem still isn't complete, they say "but I did it."
Now imagine you've got 30 kids like this. And they're getting driver's licenses.
I've messed around with Brave AI and I've seen it flub up things like converting kilograms to pounds. It can't get significant figures right either - like let's say I ask it for the approximate weight of apple in ounces. Maybe it would say approximately 6 ounces. Then ask it to convert to metric units. Suddenly the answer is calculated down to the hundredth of a gram.
So anyways I think our schools should go back to pencil and paper, lock up the smartphones, and instead of calculators go back to the fucking abacus. This is not as obsolete an idea as it sounds. I've seen videos of Asian schools were they do that and eventually the kids don't even need the abacus because they can multiply two 8 digit numbers in their head, just operating an imaginary abacus in the air with their fingers. For more advanced math, you can still use the abacus, or learn to use a slide rule. This forces a student to actually understand math. They can have a TI83+ when they get a job.
For geometry, you have to draw the problem yourself with a pencil.
If you're working from a book then copy it onto paper.
If you're working from a computer, copy it onto paper.
This is like pulling teeth but I've tutored F students in geometry and got them up to a B in a matter of days if I can get them to draw. They have to draw the shapes but that seems too difficult for them. Here is an example of the type of drawing I'd want them to copy onto paper. They are always like "I caaannnn'ttt".
I'll leave it as an exercise for the reader to solve these problems to completion. You should be able to do them all in your head if you've got a high school diploma or GED. If you need AI, go back to school. Don't get me started on reading and science.
EDIT: I'm sure the Matrix squid robots are watching me, but I just got recommended this video. I recommend you all watch it. This is horrible. When the teacher wanted those two guys to spell "philosophy" they at least did that phonetically, and I've heard of worse. I urge any American with even the most passing interest in the future of education to listen to the podcast
Sold a Story.