- Joined
- Jul 22, 2017
iirc the malloc/free are meaningfully different, and everything mallocs.
The Linux Thread - The Autist's OS of Choice
- Thread starter tehpope
- Start date
-
🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.
naphtha
kiwifarms.net
- Joined
- Sep 7, 2024
- Joined
- Dec 19, 2022
I love ZFS. It has a tonne of superb tooling. For instance, if you also use ZFS on your computer, you can easily write a systemd service and timer to zfs send|zfs recv your datasets to the server.
- Joined
- Jan 3, 2024
Generally do zfs since it has actual crc validation so between the two copies the array knows which one to keep to uncorrupt the file
- Joined
- May 29, 2023
hardware raid has gone outta fashion due to software raid and raid-like solutions being much better. unless it's an old system with very little processing performance, don't bother, it's not worth the shitshow that'll happen if something goes wrong.
ZFS, mergerfs, btrfs and even mdraid all have their pros and cons, advantages and disadvantages.
it depends on what you're storing, the features that you want and how your storage communicates with your os.
ZFS is often chosen but it's half filesystem and half raid design means that it doesn't play nicely if something is in between it and the storage device. (i.e a raid card. I've heard stories that even if one in some kind of HBA mode, ZFS still doesn't play nicely)
I've also had performance issues with a veracrypt container stored on it where it gives extremely slow response times. it's a massive pain since the container stores sensitive documents that i only decrypt and access when i need to. (it's most likely an edge case though, i haven't found anyone else running into this issue)
anyways, i do genuinely recommend that you google each one, learn their pros and cons, and choose what works best for you and your requirements and setup.
- Joined
- Jan 3, 2024
- Joined
- Dec 19, 2022
ZFS has built-in encryption, you can apply it either to the entire pool, or to individual datasets. The intended solution for you would be an encrypted dataset in your home folder, you can use systemd-automount and a script to present a password prompt when you try to access it, and automatically unmount it again after a few minutes.
I just encrypt my entire pool. All halfway modern processors have hardware encryption, there's basically zero performance impact from turning it on.
- Joined
- Mar 30, 2023
Most of what's on the data pool will be movies and tv, where if I lose a drive it would suck but I can redownload the missing files as long as the rest of the files are complete. I do have nextcloud backed up to the pool in an encrypted backup, and my more sensitive files get synced to my desktop and laptop so I technically have two backups that way.
I was thinking of getting a m.2 PCIe adapter that holds 4 m.2 cards and putting them into a raid to quadruple the speed of the working directory (where torrents and nzb downloads to, and nzb files get extracted), but it turns out that it's much cheaper to buy a single large drive then several smaller ones
I was thinking of getting a m.2 PCIe adapter that holds 4 m.2 cards and putting them into a raid to quadruple the speed of the working directory (where torrents and nzb downloads to, and nzb files get extracted), but it turns out that it's much cheaper to buy a single large drive then several smaller ones
Meriasek
kiwifarms.net
- Joined
- May 16, 2020
Finally got around to putting a new OS on my machine. OpenSUSE Tumbleweed, let's see how it goes. KDE already annoys me by being not very user friendly in terms of just showing me somewhere what kind of graphics driver I'm running. I mean, I know got the Nvidia driver package installed at some point, but somehow all the window animation shit now runs slower than before, so I suspect it's running the open source driver. But while under KDE on Ubuntu Studio I could directly see what drivers were in use, I couldn't find it yet in the few minutes I spent fiddling around with it.
Just type glxinfo like a white man.
If it runs slow then that sounds like Nvidia on Wayland.
If it runs slow then that sounds like Nvidia on Wayland.
- Joined
- May 29, 2023
i had no idea ZFS could do that, it runs on my server so doing it in a way that allows me to decrypt it when i want to access it via a network share. however without storing the password on there in someway so i can access it will be an interesting challenge...
i may need to do some research and testing in order to figure out what works for my use case.
RAID is a volume manager that combines several storage devices into one in some redundant form. half filesystem, half raid.OpenZFS.org said:
i personally use ZFS for my media storage in a raidz5 (i think, i can't remember off the top if my head right now), due to the faster read times and better redundancy at the cost of write performance. depends on the ammount of drives you have, their capacity and what ZFS config you want.
- Joined
- Dec 12, 2022
If ZFS can see individual drives, it works... You don't understand the issues at hand. If you have a flaky HBA or cabling, ZFS is going to tell you all about it, while MDRAID will silently let errors accumulate while you think everything is fine. People have also historically used RAID cards to create an array native to the card and then present that complete array to ZFS. If it's actually in HBA mode, and simply presents the drives to the system as-is, there's nothing that's going to magically go wrong.
how will you truly know they are complete without regular scrubbing by zfs to verify file integrity? it's just not possible!
Last edited:
- Joined
- Mar 30, 2023
In that context I was referring to mergeRFS, in which case if a drive fails either the file is intact or it is completely missing, and so Sonarr/radarr/etc can see it's missing and I can redownload the lost files
Last edited:
- Joined
- Dec 12, 2022
There's more to data loss than drive failure. Bitrot is real! There's no getting past it without checksumming and redundancy so you can heal from parity.
- Joined
- Dec 19, 2022
Torrents specifically are bitrot resistant, when you tell the client to verify a file it will checksum each chunk and redownload any that don’t match the manifest.
- Joined
- Jan 3, 2024
Has nothing to do with it supposedly having issues with expansion cards
- Joined
- Sep 22, 2024
This matches my experience. musl's malloc implementation is really poor, which is funny because glibc's fork of ptmalloc is really only OK. They both have bad performance if you do a lot of small allocation (like CPython is wont to do). I don't understand why neither of them have incorporated tcmalloc or jemalloc as an option, other than maybe it's a licensing issue. Musl has a lot of other problems, like autistic adherence to the POSIX spec, especially around threads, when the POSIX spec is broken and very very stupid (see pthread_cancel for an example). Rich Felker was right that the glibc source is really nasty (some of the nastiest macros I've ever had to wade through are in glibc), but it turns out a lot of the tricks glibc does mean a lot in terms of performance. Glibc also is way more complete; musl may still lack support for TCP DNS resolution, which is a REALLY serious problem if you're doing something like pulling more than a handful of records at once via DNS. This is at least one reason Debian variants of containers are almost always double digit percentages faster than Alpine.
Squishie PP
kiwifarms.net
- Joined
- May 19, 2023
Ah yeah, I use XFCE instead of KDE on OpenSUSE, since XFCE is a lot more lightweight. It should have given you the option to choose it in the installation menu. You can install XFCE by following these instructions: https://forums.opensuse.org/t/how-to-install-xfce-in-tumbleweed/131938/2
As far as NVIDIA drivers go, here is the official documentation on how to install and uninstall them if needed: https://en.opensuse.org/SDB:NVIDIA
It may be possible that you have Secure Boot enabled in your UEFI/BIOS. NVIDIA drivers generally work better with Secure Boot disabled, so make sure you do that before doing anything else. NVIDIA is one of the few hardware vendors that still has proprietary kernel modules (drivers that aren't just built into the Linux kernel like LITERALLY EVERYTHING ELSE IS,) so getting them to work can sometimes be tricky for first-timers.
If you can't figure it out, let me know.
Last edited:
Meriasek
kiwifarms.net
- Joined
- May 16, 2020
Yeah, I know I could have used xfce or gnome, but decided to go for KDE since that was the one I used last.
I used the official driver documentation, except the one click installation of course didn't work (clicking the link opens the ytm or whatever file, saving it and trying to start it with YaST returns that it doesn't contain any instructions for yast. Other 1 click install links work at least, just the nvidia drivers are somehow broken). I installed the repository and so on and given that I have the whole Nvidia setting app going on now I'm sure *some* driver must be active, but hot damn this is convoluted to find out. Previously there was a menu point in the system settings where one could simply check in an out third party drivers.
Or just a simple device manager would be nice.
Squishie PP
kiwifarms.net
- Joined
- May 19, 2023
Damn
If you run nvidia-smi in the terminal, does it output anything?
Honestly, I would have recommended Mint if I knew you were going to be stuck in NVIDIA driver hell; it has a similar driver manager utility to that of Ubuntu's.
Last edited: