You're short 8GB with a 24 VRAM to run Illama 60B with 4Q guff. That's really the best the enthusiast consumer should be shooting for. Upping it to 12GB or 16GB on the second card really should be enough to maybe get to 5Q or an expanded context. Fuck running two 3 slot cards running 350w inside a single case. You need a PSU upgrade for that as well.
...not exclusively.
You are really going to see diminishing returns past 70B so slamming a third 3090 in there is kind of pointless. The 4th is just retarded.
This is basically what I did, except with older server GPUs. Many (older) Supermicro motherboards, even some of the non-standard form factor ones, will still use standard ATX PSU sockets. From there, you can either adapt an existing chassis to fit one of these boards, or simply build your own.
The trick is to use a standard atx motherboard with a shitload of pciex16 slots and mount it in a 6U case with a split bracket on top for the GPUs. Then just run ribbons up and/or split the 16x into 8s. Only issue is that for it to work you are gonna need two broadwell CPUs which only runs pciev3 or a threadripper.
Yep, that works. You can even use a separate PSU for all the GPUs if you get a PCIe slot power isolator like this one here: https://benchlab.io/products/elmorlabs-pmd-pci-e-slot-power-measurement-adapter
Where you getting those benchmarks from? just asking.
Funny how almost 5k sounds like a lot now but back in the 90's it only got you 2 average PCs. The problem with ex-mining cards besides the wear is that most were very poorly maintained, at least that's my experience after looking at tons of cryptotards who went all in during the last craze, they knew jack shit about basic PC hardware and just bought the rigs ready-made and plugged them, didn't even bother to dust the things once in a while so the fans were caked. Another problem is that the vast majority used radeon cards so its not great for AI, most of the rigs that used geforce have long been taken apart and sold piecemeal to gamers.
I knew a tard who burned his house down by doing both this and shoving way too many rigs into a small space.
There's a rumor that after releasing a 5080 16 GB, they will make a 24 GB version sometime after 3 GB GDDR7 modules become available. Initial cards come with 2 GB GDDR7 modules, 3 GB GDDR7 modules increase capacity without changing the bus width. We could end up seeing weird cards later in the generation like a 5070 refresh with 18 GB, or some 96-bit card with 9 GB instead of 6 GB.
I remember hearing the same about the 4070 Super having 16GB and it only got 12 like the regular 4070 and the Ti.
With the 5080 practically confirmed to be just 16GB I seriously doubt this is going to be 32, more likely 24.
I'm going to retire in my early 40's precisely because I don't do dumb shit like burning money on rapidly depreciating assets like this piece of future e-waste.
What I'm talking about would be like a refresh which could come out much later. The 5080 will definitely start out with 16 GB. Nvidia has the option to boost any card's VRAM capacity by 50% using 3 GB GDDR7 modules if they so choose. Maybe they would call it a 5080 Ti/Super.
It's practically confirmed that the 5090 will have 32 GB. It's such old news that people are talking about the die size (744mm^2) and 5090D now.
It's a tool, and the only people who ought to buy it are those professionals/"prosumers" who will use it to make money (accounting for the price and the power bill). Definitely don't buy it if you don't have a better reason than playing Cyberpunk or generating anime tiddies. The apparent ~600W TDP alone will cause problems, and it will be comically large.
Getting paid to stream video games or generate anime tiddies.
Don't tell ASCAS that.
