I finally had a chance to write up a simple test regarding NVIDIA's drivers on Linux vs. Windows that anyone can mess around with.
Steps to test
- Open a bunch of applications you'd normally use throughout the day (for later tests)
- Pick any three decently heavy video games (e.g. GORI, Death Stranding and Cyberpunk 2077)
- Run them and load into any level you wish, so that each game fills dedicated VRAM decently
- Suspend/pause the first two games, so that they're not consuming any GPU processing time
- Observe the subjective performance of your singular running game with the other two suspended
- Try switching between games by suspending/resuming their processes, observing the impact
- Observe the effect under load when switching between existing running applications
- Try opening other apps while the system is in this state to make sure everything works as usual
Relevant Commands
| Purpose | Linux | Windows |
Monitoring Resource Use
(for NVIDIA cards) | Use the CLI to get reliable stats:
Bash:
watch -n 100 'nvidia-smi -q |
grep -i "FB Mem" -A 3 && nvidia-smi -q |
grep -i BAR -A 3 && nvidia-smi |
grep -E "[0-9]{2,3}MiB \|" |
grep N/A && free -m'
| Just use Task Manager on Windows 11
nvidia-smi can't fetch usage stats because of WDDM and unfortunately Task Manager values aren't perfect because of the abstraction layers the GPU drivers are forced to utilise. |
| Suspending Processes |
Code:
pkill -STOP <name of process>
|
Code:
pssuspend <name of process>
|
| Resuming Processes |
Code:
pkill -CONT <name of process>
|
Code:
pssuspend -r <name of process>
|
| Terminating Processes |
Code:
pkill -9 <name of process>
|
Code:
taskkill /f /im <name of process>
|
Linux Results (NVIDIA)
VRAM usage:
Code:
FB Memory Usage
Total : 12282 MiB
Reserved : 163 MiB
Used : 11991 MiB
BAR1 Memory Usage
Total : 16384 MiB
Used : 4654 MiB
Free : 11730 MiB
The top consumers of VRAM were:
Code:
| 0 N/A N/A 2785 G /usr/bin/kwin_wayland 306MiB |
| 0 N/A N/A 2945 G /usr/bin/plasmashell 501MiB |
| 0 N/A N/A 3899 G /usr/lib64/firefox/firefox 315MiB |
| 0 N/A N/A 19884 G ./steamwebhelper 60MiB |
| 0 N/A N/A 42230 C+G ...dlyCarnage-Win64-Shipping.exe 3889MiB |
| 0 N/A N/A 44337 C+G ...TRANDING DIRECTORS CUT\ds.exe 4336MiB |
| 0 N/A N/A 46589 C+G ...077\bin\x64\Cyberpunk2077.exe 1620MiB |
| 0 N/A N/A 49283 G /usr/bin/systemsettings 111MiB |
| 0 N/A N/A 56675 G /usr/lib64/firefox/firefox 63MiB |
The NVIDIA Linux drivers are completely incapable of evicting anything from dedicated VRAM in this scenario, even when processes are suspended. The third game I launched (Cyberpunk 2077) ended up with insufficient VRAM from the GPU and ran like a slideshow.
Existing Wayland-native processes would mostly remain working (except for launching a second Firefox profile). New Wayland-native processes would either crash out, render transparent windows or somehow find a way to snake their way into what little framebuffer memory remained. XWayland processes would always end up rendering transparent windows when launched, even things with software rendering like the GTK2-powered Nero Linux. Upon closing one of the games, applications would begin to work, but some needed a manual resize before their contents would redraw.
One thing I haven't tried is sticking with purely open-source native Linux games designed to work natively on Wayland to see if the end result is the same. But that's mostly because I can't think of any which are heavy enough to challenge a 4070 Ti.
Windows Testing (NVIDIA)
VRAM usage:
The top consumers of VRAM were:
Cyberpunk 2077 was as playable as it would normally be (only the odd occasional hitch or two, usually along with pop-in when streaming in assets from other areas). Suspending/unsuspending processes to switch between games yielded similar general framerates to when they run standalone. The drivers also automatically reclaimed dedicated VRAM from any processes not visibly on-screen quite aggressively. The most notable issue under this level of load was a small amount of latency switching to/from applications which had been completely shunted out of dedicated VRAM, where the system would stutter for a split second (including the mouse pointer) but to be fair, this also happens from time to time when querying monitor/display info or loading some of the Task Scheduler UIs under normal load anyway.
Just to see what would happen under absolutely ridiculous VRAM pressure, I did try loading in a 12b model using koboldcpp with all 40 layers GPU accelerated, in addition to the above. As the rest of the games plus other software had already consumed a large amount of RAM, as well as both dedicated and shared VRAM, the NVIDIA drivers no longer had easy access to further system RAM to repurpose. This resulted in a very stuttery desktop, with no improvement after 3 minutes of waiting. Suspending the koboldcpp process also did nothing to improve this, as it would jump between 0 and 100% GPU usage irrespective of whether it was actively running or not, suggesting the NVIDIA driver was likely backlogged trying to scavenge for resources. Ending the koboldcpp process immediately resolved the problem, however, and even the games started working again.
A silly thing I haven't tried is using RAMMap to manually empty out working sets, standby pages etc. to free up further resources for the driver to claim, but that will have to wait for another day.