GPUs & CPUs & Enthusiast hardware: Questions, Discussion and fanboy slap-fights - Nvidia & AMD & Intel - Separe but Equal. Intel rides in the back of the bus.

[Car Won't Crank]

it looks like HA if you want to use plugins you have to install the OS in a virtual machine or docker container, and it does not like Debian 11. It does have more overhead, but I guess that won't be as big of an issue in my next server

The website and docs steer you to install HA on bare metal like a Rpi or NUC as it makes using it easier for normies. If you dedicate a whole machine to it, HA will automatically take care of addons by creating containers and linking ports and whatnot. For power users, containers are best imo as it provides much more flexibility. Just requires a bit more searching to get the docker url for each addon you want to install, which has to be done manually. The main HA is a container and all the additional addons are their own containers like Mosquitto, Frigate, etc. Perfect use case for Podman pods.

 

[WelperHelper99]

would argue NVIDIA cards are a much better buy than AMD cards due to DLSS3. Even at 2X upscaling, DLSS2 doesn't noticeably shimmer. So in practice, especially the way @WelperHelper99 is likely to use his machine, a $550 4070 is going to produce better graphics than the similarly priced 7800 XT. Additionally, I've noticed that if something devs do is going to run poorly on a particular card, it's going be AMD. Warhammer: Darktide inexplicably started puking frames after the latest update. Seems to be related to screen space reflections...and it only does it on my 6700 XT, while my much shittier-on-paper 3050 Ti Laptop has no problem.

Really how I plan on using it is a gaming rig first, everything else second. A.I. is cool, but I'd rather have my games run well. For that, the 4070 is good. And future proofed. I don't plan on upgrading for years, so I'd the card is going to be the most expensive part of the rig, I want it to be fast at gaming first. A.I. is cool, but I see myself playing with it for a month and getting bored.

still don't agree with the 4070 though. The 16GB 4060 is going to be able to keep both SDXL and the refiner in memory simultaneously, which is an enormous performance boost. The 4070 would be swapping half of the refiner into system RAM, which would take you from something like 2it/s to 10s/it. For gaming you're right, the 4070 is a stronger card, but he specifically wants to play with AI, and for that the 4060 is actually significantly better. And DLSS means the performance impact is really going to be something like going from rendering at 75% resolution, to rendering at 67%. Which isn't huge, DLSS genuinely is very good even when it's being made to do a lot.

To be clear, A.I. is at the bottom of the list. Computers are a hobby in my life, really I just want to play games, and if I'm REALLY bored, generate a image or two. I've looked at the 4060 Ti, I can get one fairly cheaply, it's just long term I worry about, because I'll probably get bored of A.I, but gaming is forever.

 

[The Ugly One]

Yeah. Much as it pains me, I basically always recommend Nvidia,

NVIDIA 's corporate culture has their people very comfortable with overpromising & underdelivering and treating anyone they do business with like they own you. I was working for publicly traded company with billions in revenue a few years back, and one of their people refused to collaborate with us unless we changed the target market of an upcoming product. It was insane, like fuck off, this isn't your call, asshole. I think this goes a long way toward making people dislike them and hope somebody, anybody can unseat them. On the flip side, AMD is very cool to work with.

For gaming you're right, the 4070 is a stronger card, but he specifically wants to play with AI, and for that the 4060 is actually significantly better.

I expect @WelperHelper99 to play around with AI for about 3 hours and then never touch it again, so that's why I haven't really even brought it up.

And DLSS means the performance impact is really going to be something like going from rendering at 75% resolution, to rendering at 67%. Which isn't huge, DLSS genuinely is very good even when it's being made to do a lot.

It can be quite a lot:

	Vertical scale factor	Reduction in rasterized pixels	Internal resolution @4K	Internal resolution @1080p
Quality	0.67	56%	1447p	720p
Balanced	0.58	66%	1252p	626p
Performance	0.5	75%	1080p	540p
Ultra Performance	0.33	89%	720p	360p

I put 1080p in there because that's my laptop's screen. Even if the game runs fine, I use it on Performance mode to keep the heat down. It's really quite astounding to me just how good of a job it does when rasterizing only 1/4 of the pixels and using a neural net to generate the rest.

 

[WelperHelper99]

I expect @WelperHelper99 to play around with AI for about 3 hours and then never touch it again, so that's why I haven't really even brought it up.

That's what I see happening. I know my attention span. I'll goof around with it a little, sure, but actually use it? I doubt it. The entire rig is a generation behind. Actually going full into A.I. would melt it. It's a multi media device. That's what I'm building.

 

[Just Some Other Guy]

It's cool. Don't feel pressed into it.

 

[WelperHelper99]

It's cool. Don't feel pressed into it.

Don't worry, my wallet is preventing any impulse purchases, especially after Christmas. It'll be until February when I buy it, which by then I should have weighed the pros and cons of what I exactly plan on doing with the rig.

 

[DavidS877]

I want to take back any nice thing I may have inadvertently said about Intel in this thread.

Story Time:

I have cameras around the outside my house to keep an eye on what's going on. A few years ago I decided to check out object detection. The cameras detect motion and store it to my NAS with video and 1 frame/second JPG. I added a Raspberry Pi to test out. It was able to detect in about 4 seconds per frame. Plenty to test.

Then I found out about these USB accelerators. Coral AI and Intel NCS2. Both pretty new. I tried the Coral and it was still a pain. Switched to the NCS2 and OpenVINO and had it working fairly quickly. Detection in a couple milliseconds, and all the image processing on the Pi CPU, resize, drawing boxes, etc. was the slow part.

Fastforward. Got a Pi 5. Setup the new OpenVINO, new YOLOv8, got it all working on the CPU, detection slow, but image processing much better. Went to move the NCS2 over to the new Pi. Went to find the needed udev rules so normal users could use it. It wasn't in the main package. Went on-line, found the NCS2 was discontinued. Ok, fine, where are the setup for the newest OpenVINO... gone... like the product never existed, and all of the Intel Movidius ML accelerators were ripped out. So, my options are backtrack to an older version or say "Fuck Intel" and switch to something else.

Guess I'll figure out where I left that Coral Ai stick and get it running since all the other Pi stuff seems to be a giant pain in the ass, or Chinese, AI sponsored by Tencent.

Of course, I also bought an Arc GPU since I wanted to see how it did AI with more VRAM. It was/is poorly supported too and is sitting on the shelf waiting for something, not sure what.

Thank you for coming to my "Fuck Intel" talk.

 

[The Ugly One]

Fastforward. Got a Pi 5. Setup the new OpenVINO, new YOLOv8, got it all working on the CPU, detection slow, but image processing much better. Went to move the NCS2 over to the new Pi. Went to find the needed udev rules so normal users could use it. It wasn't in the main package. Went on-line, found the NCS2 was discontinued. Ok, fine, where are the setup for the newest OpenVINO... gone... like the product never existed, and all of the Intel Movidius ML accelerators were ripped out. So, my options are backtrack to an older version or say "Fuck Intel" and switch to something else.

Discontinuing a product always pisses off its existing users. I had customers mad at me for no longer supporting something we released a full decade earlier. Not that it makes it any less annoying, but it's worth understanding what is happening at Intel.

During the Krzanich era, it was just assumed that Intel's near-monopoly (90%+ market share) in server & desktop CPUs was untouchable, and they hurled money at countless initiatives and acquisitions that were largely unrelated to their core business and producing niche products that never had a big market. This includes the 2016 acquisition of Movidius and the NCS sticks. So Krzanich let their process node fall behind, and now they are in deep shit in their core business as AMD has basically taken their entire workstation business (Threadripper PRO has a 95% market share), is gobbling up their lucrative server business, is taking more and more of their desktop business, and is even starting to make inroads into their once-inviolable laptop business.

Intel is in big fucking trouble, and Pat Gelsinger is basically cutting everything that isn't going into either dekstops or datacenters at blinding speed. Cold comfort to you, I know, but the lesson here is that any Intel product released up to about 2019 that isn't a mainstream device isn't something for which you should expect long-term support.

 

[Car Won't Crank]

I want to take back any nice thing I may have inadvertently said about Intel in this thread.

Story Time:

I have cameras around the outside my house to keep an eye on what's going on. A few years ago I decided to check out object detection. The cameras detect motion and store it to my NAS with video and 1 frame/second JPG. I added a Raspberry Pi to test out. It was able to detect in about 4 seconds per frame. Plenty to test.

Then I found out about these USB accelerators. Coral AI and Intel NCS2. Both pretty new. I tried the Coral and it was still a pain. Switched to the NCS2 and OpenVINO and had it working fairly quickly. Detection in a couple milliseconds, and all the image processing on the Pi CPU, resize, drawing boxes, etc. was the slow part.

Fastforward. Got a Pi 5. Setup the new OpenVINO, new YOLOv8, got it all working on the CPU, detection slow, but image processing much better. Went to move the NCS2 over to the new Pi. Went to find the needed udev rules so normal users could use it. It wasn't in the main package. Went on-line, found the NCS2 was discontinued. Ok, fine, where are the setup for the newest OpenVINO... gone... like the product never existed, and all of the Intel Movidius ML accelerators were ripped out. So, my options are backtrack to an older version or say "Fuck Intel" and switch to something else.

Guess I'll figure out where I left that Coral Ai stick and get it running since all the other Pi stuff seems to be a giant pain in the ass, or Chinese, AI sponsored by Tencent.

Of course, I also bought an Arc GPU since I wanted to see how it did AI with more VRAM. It was/is poorly supported too and is sitting on the shelf waiting for something, not sure what.

Thank you for coming to my "Fuck Intel" talk.

I share some similar angst with the google Coral stuff. While there is some support and documentation for installing and using Coral TPUs, it appears Google doesn't care about maintaining it regularly. When I went to add the google repos in my Debian server, they all returned errors of some sort. I forget what exactly. It was a gpg key or repo link itself was not found. I had to dig through some forum posts for the correct links to get it installed. Bad first impression. Additionally, the Coral drivers use an older version of python so it makes initial setup slightly more annoying if you're not using containers.

 

[Susanna]

Coral drivers use [...] python

That's really scary.

 

[Two Dollars]

I still don't agree with the 4070 though. The 16GB 4060 is going to be able to keep both SDXL and the refiner in memory simultaneously, which is an enormous performance boost. The 4070 would be swapping half of the refiner into system RAM, which would take you from something like 2it/s to 10s/it. For gaming you're right, the 4070 is a stronger card, but he specifically wants to play with AI, and for that the 4060 is actually significantly better. And DLSS means the performance impact is really going to be something like going from rendering at 75% resolution, to rendering at 67%. Which isn't huge, DLSS genuinely is very good even when it's being made to do a lot.

Now you have me second-guessing myself about getting a 4070 Super. I’ll use it primarily for gaming, but I’m also interested in LLMs and might get into image generation at some point, and this is a card I’m planning to keep for the next 3-4 years. This would be a lot easier if Nvidia just made a 16GB 4070/Super.

 

[DavidS877]

Now you have me second-guessing myself about getting a 4070 Super. I’ll use it primarily for gaming, but I’m also interested in LLMs and might get into image generation at some point, and this is a card I’m planning to keep for the next 3-4 years. This would be a lot easier if Nvidia just made a 16GB 4070/Super.

You mean the 4070 Ti Super instead? Only $200 more... it's only money,,,,
Also someone in the Nvidia marketing department needs to work on their naming schemes.

 

[Two Dollars]

You mean the 4070 Ti Super instead? Only $200 more... it's only money,,,,
Also someone in the Nvidia marketing department needs to work on their naming schemes.

I’ve already spent more than planned on my build, I really don’t want to spend $800 on a GPU. $600 + tax is already pushing it.

 

[The Ugly One]

You mean the 4070 Ti Super instead? Only $200 more... it's only money,,,,
Also someone in the Nvidia marketing department needs to work on their naming schemes.

I tried to get them to rename it the Nigger Faggotizer, but I don't have that kind of clout.

 

[Leaded Gasoline]

I’m also interested in LLMs and might get into image generation

have you tried using your CPU before deciding on the card to get? LLMs run okay even on trash tier CPUs if you have enough RAM

image generation is a pain though, it takes my 18 core xeon a bit over 3 minutes per 512x512 image using easy diffusion

 

[The Mass Shooter Ron Soye]

Intel's Application Performance Optimization (APO) is no longer exclusive to 14th Gen desktop CPUs and 2 games (Metro Exodus and Rainbow Six: Siege):

https://wccftech.com/intel-to-bring-game-boosting-apo-optimizations-to-12th-13th-gen-desktop-cpus/

Intel also announced a host of new titles that would get APO support during CES 2024. These include Marvel's Guardians of the Galaxy, F1 22, Strange Brigade (Vulkan), World War Z, Dirt 5 & World of Warcraft, these are among the list of 14 games. In addition to the 12th and 13th Gen Desktop CPUs, Intel will also be adding its 14th Gen Raptor Lake-HX CPUs to the list of supported chips for APO. A BIOS update will be required to fully enable APO on existing 600 and 700-series motherboards with 12th & 13th Gen CPUs.

 

[Two Dollars]

have you tried using your CPU before deciding on the card to get? LLMs run okay even on trash tier CPUs if you have enough RAM

image generation is a pain though, it takes my 18 core xeon a bit over 3 minutes per 512x512 image using easy diffusion

If LLMs will run well on a 4070 Super (which is what I was going to get anyway for gaming), then that’s fine. Image generation would just be a perk.

 

[BPD Gunt Rider]

Also, for gaming, Nvidia's value actually isn't as bad as it seems, thanks to DLSS. Yeah they're not "real" pixels, but DLSS 2 & 3 are genuinely hard to tell apart from the real thing. Actively playing the game and not just looking at screenshots to find the flaws, you won't even notice.

DLSS2, sure. In most games, it looks fine, especially with a 4k monitor. It looks noticeably cleaner than just lowering resolutions.

DLSS3, it's impressive technology that I fail to see the purpose of. I easily noticed latency issues attempting to bump say ~30 fps to 60+.

 

[Leaded Gasoline]

If LLMs will run well on a 4070 Super (which is what I was going to get anyway for gaming), then that’s fine. Image generation would just be a perk.

not sure what you want to run exactly but 12GB will not be enough for larger LLMs, the smaller ones will work just fine and image generation typically uses way less RAM if you don't crank up the resolution

 

[Two Dollars]

not sure what you want to run exactly but 12GB will not be enough for larger LLMs, the smaller ones will work just fine and image generation typically uses way less RAM if you don't crank up the resolution

I’m thinking about 13B models specifically. I know 30B is a big step above that.