Replies to message #7,448
-
🔎 If you see an alert for analytics.usips.org, this is a self-hosted system I am running as a test. It is GDPR compliant and collects no invasive information.
The Linux Thread
They didn't say anything but it should manage a 7b model at 4 bit my guess somewhere in the area of 30 tokens/s. People got 7b and 13b LLMs...