GPUs & CPUs & Enthusiast hardware: Questions, Discussion and fanboy slap-fights - Nvidia & AMD & Intel - Separe but Equal. Intel rides in the back of the bus.

[The Mass Shooter Ron Soye]

Now, you couldn't practically go this big for a PC SoC, because DDR5 doesn't have the bandwidth to keep a GPU that big busy.

Strix Halo will do just that. Too big to fit on AM5, up to 16 cores and 40 CUs (RDNA3.5), coping with the bandwidth problem by supporting 256-bit (quad-channel) DDR5 memory.

Will there be a mini PC with Strix Halo soldered onto a board, and four DDR5 DIMM slots, actually supporting quad-channel memory? Beats me. We could see fully soldered LPDDR5(X) solutions instead. Maybe 2x CAMM could work, but they can't get to quad-channel with stacking.

(Strix Halo should be considered a laptop-first product. Mini PCs are more of an afterthought.)

Consumer-level AI runs on the CPU. You can't assume the presence of a dGPU for a mass-market application, which is why AMD, Intel, Apple, and Qualcomm are all putting NPUs on their CPUs.

XDNA2 may have doubled or tripled die area compared to XDNA1 (on the same TSMC N4 node) to make its big jump to ~45-50 TOPS, and every CPU/APU should have this "consumer-level" of dedicated AI performance eventually because Microsoft demands it. But how much will they improve in the future? Meanwhile, Nvidia advertises "242 AI TOPS" using tensor cores for the RTX 4060. TOPS ratings aren't always comparable (I'm seeing RTX 3080 gets about doubled RTX 4060 Stable Diffusion performance at the same TOPS) so we'll have to see how well these accelerators do in practice. But someone who wants to use AI for more than 15 minutes before getting bored of it can probably afford a discrete GPU.

 

[The Ugly One]

Strix Halo will do just that. Too big to fit on AM5, up to 16 cores and 40 CUs (RDNA3.5), coping with the bandwidth problem by supporting 256-bit (quad-channel) DDR5 memory.

Yeah, more memory channels certainly fixes the problem, especially now that we're at DDR5-6400. A SoC with 4 DDR5 interfaces could be a very capable game machine while being far cheaper than one with a dGPU.

XDNA2 may have doubled or tripled die area compared to XDNA1 (on the same TSMC N4 node) to make its big jump to ~45-50 TOPS, and every CPU/APU should have this "consumer-level" of dedicated AI performance eventually because Microsoft demands it. But how much will they improve in the future? Meanwhile, Nvidia advertises "242 AI TOPS" using tensor cores for the RTX 4060. TOPS ratings aren't always comparable (I'm seeing RTX 3080 gets about doubled RTX 4060 Stable Diffusion performance at the same TOPS) so we'll have to see how well these accelerators do in practice. But someone who wants to use AI for more than 15 minutes before getting bored of it can probably afford a discrete GPU.

Consumer-level AI is going to be stuff like opening up PowerPoint, telling it, "I want a six slide presentation on how the marketing department has improved its reach in SouthEast Asia" and getting a nice skeleton to work with, batch organizing and labeling your photos, more sophisticated browser histories, better adjustment of lighting & color during conference calls, and so on. It's going to be in constant, frequent use for countless tasks, most of them not very flashy, but they add up. NVIDIA has no inroad here, because paying hundreds of dollars to do these tasks faster is not a good value prop.

In fact, software developers are already betting on NPUs over GPUs. There's lots of code already out there for Qualcomm & Apple NPUs, and while efforts are already underway to port it to AMD & Intel solutions, there's very little interest in porting to CUDA. For example, Windows Studio effects are only available on Qualcomm-powered Surface machines right now, but will soon be on Meteor Lake and Ryzen 8000 laptops as well. Microsoft isn't even bothering to port them to CUDA.

 

[AmpleApricots]

That's also why I find the expression "SoC" more fitting. What people call a CPU nowadays is really more of a System-on-Chip. Cyrix kicked them off, as in were the first that commercially somewhat mattered, then later that technology went to AMD. First time I saw the AMD logo was on PAL chips, which were sort of a revolution themselves. MMI? Fairchild? Who the hell remembers.

Higher integration has been like, the goal of all of computer technology forever. I think the striking difference is most visible in late 80s to early 90s tech:

386 mainboard, late 80s

Same thing, early 90s:

Almost all that stuff from board 1 disappeared into that big chip in the middle. That big grid of RAM in the lower right disappeared too and now goes into the white sockets, at much higher density. I think people who have not lived it can't imagine how fast these developments can go if there is interest.

I think it's too early to really say where the AI race will go because there still will be fundamental architecture changes and something that'll be high end in a year will probably be a joke the next. But you'll totally not need a dedicated card for big-brained AI either one day, I am deeply convinced of it. Hell, the hardware requirements changed already dramatically in the last 12 months. What people now run at home on higher end GPUs would've been unfeasible to run at home altogether just a year ago.

 

[The Ugly One]

Oh yeah, I forgot to mention how this used to be L2 cache:

But you'll totally not need a dedicated card for big-brained AI either one day, I am deeply convinced of it.

As a general rule, anything that requires the consumer to buy an additional, expensive accelerator is not going to be mainstream. You just can't expect enough people to have it for it to sell. Hell, even 3D accelerated gaming didn't actually get big until consoles. Voodoo sales never hit anywhere near what the Playstation was doing. IIRC even 2D gaming didn't actually get big on PCs until die shrinks had made it inexpensive for every motherboard to have an integrated EGA adapter at minimum.

 

[Freedom Fighter]

Oh yeah, I forgot to mention how this used to be L2 cache:

View attachment 5662987

True 3D V-cache!

 

[AmpleApricots]

Oh yeah, I forgot to mention how this used to be L2 cache

There used to be fake ones. Empty plastic package, nothing (no die) inside. It was worth it because RAM was HORRENDOUSLY expensive, fast one even more so. Since the way how this cache used to work you could only really find out via benchmarks if you didn't build a dedicated testing setup to see if this was actually memory, which was a lot harder to do back then. The 90s were a wild mix of cutting edge technology and abysmal production quality, because the market was expanding by the minute and if you wanted to keep up to date you basically had to buy a new system every six to eight months and everyone needed cheap parts and many of them STAT. On upgrade day you'd go to half a dozen computer stores, take their printed out price lists and compare them later and hope the hardware you selected wasn't out of stock. China emerged as cheap country to do shit in but businesses didn't yet know how to wrangle the eternal chinaman and his schemes so many fly-by-night operations making hardware of questionable quality, I guess you'd call them "start-ups" now. I remember taking a power supply apart and being so appalled by what I saw that I just threw it away as it was actually dangerous, as in "this will eventually burn my house down if I use it". Another time a regulator on one of my harddrives exploded and caused a small fire. (No, this was not caused by the power supply) I know of somebody who's pentium era Northbridge just... melted.

Nowadays you can buy the lowest price components and they will be fine. (excluding of course 4TB usb sticks for $10 from aliexpress, like I said, the chinaman is eternal) Things have changed a lot in that regard.

As a general rule, anything that requires the consumer to buy an additional, expensive accelerator is not going to be mainstream.

Early personal computing was strictly in the realm of the hobbyist. A normal person did not need nor wanted to have a computer. They would put up with this, your average consumer wouldn't, that is true. That said, just like the early PCs, the early AIs are strictly in the realm of the enthusiast. I mean, the desktop itself is a niche product nowadays so whaddya expect.

 

[The Ugly One]

That said, just like the early PCs, the early AIs are strictly in the realm of the enthusiast.

Windows 11 already has AI features that run on your CPU. If you take a screenshot, you can extract text from it. It uses AI to do so. It's a pretty simple AI, but you don't need a 600W datacenter chip for that sort of neural net.

 

[Fcret]

I don't know what the situation with Intel or Apple is in that regard. There's *a ton* of money poured into AI right now.

Understanding Apple is a simple matter of comprehending their specific corporate autism. Their goal for AI/ML is pretty straightforward - on-device. Most of their resources have been poured into improving local models with things like the neural engine and CoreML rather than chasing the cloud AI service dream. And they're already extremely strong in this space - for memory hungry stuff, a fully kitted out Mac Studio is unironically one of the most affordable ML workstations you can get compared to what nvidia wants.

Like Ugly One said, there's no interest in the HPC Datacenter market because that market is kind of the polar opposite of their on-device ethos.

The reason I expect 3D accelerator cards to die is every single other accelerator card is dead or relegated to a meaningless niche. At some point, the tech is good enough that paying big money for an expensive add-on is just not something consumers want. The most logical place for the GPU to end up is on the CPU rather than the motherboard.

"but MUH UPGRADES MUH MODULAR SYSTEM"

I wonder how many people back in the day were upset when the Pentium came out because they were "forced" into paying for an FPU they "weren't going to use". It has to be non-zero.

 

[The Ugly One]

Understanding Apple is a simple matter of comprehending their specific corporate autism. Their goal for AI/ML is pretty straightforward - on-device.

More important is understanding that Apple isn't a technology company. It's a consumer appliance company. Datacenter racks aren't consumer appliances.

 

[AmpleApricots]

If you take a screenshot, you can extract text from it.

While OCR is AI, it isn't exactly new. Is that a feature they recently introduced? Are we talking "recognizes Arial in black on a white website" or "recognizes shitty handwriting on a piece of paper in a youtube video"? I haven't used a Windows since XP. LeCun (Meta's head AI guy) did OCR stuff in the 90s.

back in the day

With every piece of new tech there was always at least some guy against it, claiming his old piece of tech worked "just fine". I sometimes feel I'm that guy now.

 

[SilentDuck]

Higher integration has been like, the goal of all of computer technology forever. I think the striking difference is most visible in late 80s to early 90s tech:

386 mainboard, late 80s

Bro is just flashing that math co-processor like its a given!

 

[AmpleApricots]

Bro is just flashing that math co-processor like its a given!

I didn't even see it my vision went blurry when I saw that LIF socket. I still have some of the original intel "combs" to pull chips out of them. I once desoldered one from a 68040 Mac because I'm completely insane.

 

[SilentDuck]

I didn't even see it my vision went blurry when I saw that LIF socket. I still have some of the original intel "combs" to pull chips out of them. I once desoldered one from a 68040 Mac because I'm completely insane.

The mathco was the ultimate flex for a while there. In the same position I'd save and hoard them like an old lady with those fancy spoons and cat plates.

 

[AmpleApricots]

The mathco was the ultimate flex for a while there. In the same position I'd save and hoard them like an old lady with those fancy spoons and cat plates.

Oh, now I understood - these are not my systems, just random pictures off the internet. I have some similar ones, just their batteries weren't as kind to them.

 

[LegoTugboat]

It's a pretty small image, but for context to the consoles, Apple's M series iGPUs are between 20-35% of the total chip. It depends a bit on what actual chip it is.

https://i.ytimg.com/vi/8bf3ORrE5hQ/hqdefault.jpg

(Blue is the GPU, red are the performance cores, purple is the neural engine, green in the middle are the efficiency cores)

So yeah, consoles iGPUs are goddamn massive.

 

[ZipDisk]

I watched a restore video on an ancient Packard Bell PC and Necroware had the misfortune of "rescuing" one that was hostile to L2. Like he went into the dinosaur internet and found out about it after benchmarking and trying to find out where "the damage" was, and it turned out "the damage" was in the design.

 

[DavidS877]

More important is understanding that Apple isn't a technology company. It's a consumer appliance company. Datacenter racks aren't consumer appliances.

Apple: "Hey, our new iPhone Skynet Edition has on-device AI and only costs $100 more"
Consumers: BUY!
Apple: Finally, we can be rid of that giant AI cluster we had to build.

 

[ZMOT]

In this thread's humble opinion, is there a point to VR right now or is it just a big toy/gimmick?

FAT FUCKING TIDDIES
RIGHT IN YOUR FACE
'nuff said.

but seriously, depends on a gimmick you need/want. do you "need" a fight stick to play a fighting game? no, but some want one. same for hotas and sims. so if you think the experience is worth the price (sadly the cheap options have disappeared/sold out over the years), there's no shame in giving it a try. worst case you can always buy used, you could get WMR headsets for 200 bucks back in the day, should be cheaper now. won't be as comfortable/sturdy/feature rich as current models, but will do the trick.
or if you have a mate with a headset ask to give you a demo (or borrow it).

it's important tho to go in with the right expectation, it won't be like the matrix or holodeck, but the 3d immersion effect is neat and can greatly enhance certain games. what it will never do is replace pancake gaming. there's very little improvement playing something like anno in VR when mouse and laying in your chair is much simpler and more comfortable. otoh something like synth riders will never really work on traditional screens. FPS depends but is often hampered by motion - I'd go as far and say sony somewhat fixed it with the aim controller, but that's not something anyone would buy on PC, let alone with the fragmented market (and splicing up your knuckles for a 3d printed option isn't really an option due to it's price, since valve had to add a lot of hardly used crap for some reason, making it more expensive than it needs to be and should).
stuff like simulations are also damn great, just check any project wingman vr review.

you need a fairly good dedicated area to do it versus my cramped home office.

not really, it was a marketing feature in the beginning. lot of games you can play seated, or standing while taking a step in each direction. I play most stuff on a 3-4m² space. there's simply not that much point having a large play area when you run/port around virtually anyway. why walk around the last meter in meatspace? and for games that play in an enclosed space, you can pretty much get the same experience from playing it seated without devs limiting their game to the people who have a dedicated VR room.

TLDR: do you "need" VR? no, but sometimes it makes shit so much fucking better porn included

 

[Second Sun]

Discrete GPUs will never go away, there's just too many orders of magnitude difference in performance. But igpus are increasingly 'good enough' for many games.

Nintendo Switch: 393.2 GFLOPS
Intel UHD 770 iGPU: 793.6 GFLOPS
Intel Iris Xe iGPU: 2.2 TFLOPS
AMD 8700G iGPU: 4.5 TFLOPS
PS5: 10.3 TFLOPS
Nvidia 3060: 12.7 TFLOPS
Nvidia 4080: 48.7 TFLOPS

 

[LegoTugboat]

Mm. There's quite a difference between 4k ultra-high, and 1080p or 1440p on medium settings.

Apple M1: 2.6 Tflops
Apple M2: 3.6 Tflops
Intel Arc A370M: 4.1 - 4.2 Tflops (in fairness, a dedicated low power GPU)

When you compare it to something like the previous Intel iGPUs...

Intel UHD Graphics 630 (2017): 403.2 - 460.2 Gflops
Intel Iris Plus G7 (Ice Lake) (early-mid 2020): 1.07 Tflops
Intel Iris Xe G7 96EU Mobile (late 2020): 1.6 Tflops

Honestly, a roughly 4x boost in 3 years is pretty damn good, even if starting from a relatively low baseline