The Linux Thread - The Autist's OS of Choice

  • 🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.
Hey is there a way to prioritize reads over writes? Like I have 4 14tb drives in a mergerfs pool that all my media is on, but if I'm transferring new content to the same drive that jellyfin is trying to read from then Jellyfin gets really slow and unresponsive. Would something like an xfs filesystem help, or some setting I can change?
 
Belisarius Cawl said:
Click to expand...
sudo is "owo" isn't it?


edit:
1733635404011.png
 
  • Horrifying
  • Islamic Content
  • Like
Reactions: Foghotten, TORtlEMUStRaNSUS, Moonglade Feral Druid and 9 others
Betonhaus said:
Hey is there a way to prioritize reads over writes? Like I have 4 14tb drives in a mergerfs pool that all my media is on, but if I'm transferring new content to the same drive that jellyfin is trying to read from then Jellyfin gets really slow and unresponsive. Would something like an xfs filesystem help, or some setting I can change?
Click to expand...

I don't know how mergerfs plays into everything, but the first thing to try (presuming you're using a relatively recent kernel) would be to set the IO scheduler for those block devices to BFQ and then use ionice to deprioritize whatever process is copying and/or prioritizing your Jellyfin instance. That makes more sense to me than trying to prioritize reads/writes as it's more surgical and probably better accomplishes what you're after. Just running BFQ without even resorting to ionice may improve things, but it's got some overhead (really for spinning disks though on hardware that isn't really anemic, you should be OK); I don't recall what tuning is possible with directly with BFQ (or any of the other exotic IO schedulers), but that's where I would start, if just turning it on and maybe trying ionice doesn't work. Be warned that there's some gotchas also around exotic IO, l think if you're doing certain kinds of async IO on your block devices.

Barring the above, the next thing that I know can control how much certain things do I would be cgroups. That's likely somewhat involved, but I think systemd puts everything in their own cgroup now? It might not be a huge pain, if systemd provides a way to get at the cgroups and you're willing to experiment.

You got me thinking (and maybe bcache can do this and I've never looked hard enough), but it would be nice to have a caching overlay filesystem like bcache where you could just drop an SSD in and mount it on top of an existing filesystem like unionfs or overlayfs, and for access via the new mount point, reads would be cached and writes would pass through without disturbing or needing to modify anything else (like the base filesystem/block device), and which can be installed in place and online for an existing setup. I don't think it's possible to migrate in place to a bcache layer, but that's also a usually good way to accelerate reads over something that looks like a block device.
 
  • Like
Reactions: Vecr
is it possible to configure dnsmasq to run alongside systemd-resolved so that resolved handles all internal dns resolution and server appliances and docker images, while dnsmasq can be set up as the primary dns for my network so that clients that want to access something from my server using it's external url will talk to it directly instead of bouncing it through my router/modem first, without having to resort to a hosts file which is a pain for guest devices and portables used outside the network.

right now i have systemd-resolved disabled and using dnsmasq, but it keeps crashing when i get a network interruption (possibly because sonarr and radarr is pounding it so i did raise what is hopefully the limit it's hitting), and my management interface cockpit has a hard dependency on systemd-resolved which is difficult to work around.

though it's problably moot as i don't know exactly what steps I did to get this configuration, and may no be able to undo it.
 
Betonhaus said:
is it possible to configure dnsmasq to run alongside systemd-resolved so that resolved handles all internal dns resolution and server appliances and docker images, while dnsmasq can be set up as the primary dns for my network so that clients that want to access something from my server using it's external url will talk to it directly instead of bouncing it through my router/modem first, without having to resort to a hosts file which is a pain for guest devices and portables used outside the network.

right now i have systemd-resolved disabled and using dnsmasq, but it keeps crashing when i get a network interruption (possibly because sonarr and radarr is pounding it so i did raise what is hopefully the limit it's hitting), and my management interface cockpit has a hard dependency on systemd-resolved which is difficult to work around.

though it's problably moot as i don't know exactly what steps I did to get this configuration, and may no be able to undo it.
Click to expand...
systemd-resolved is a total pile of unmitigated bullshit, and I there's no way I would ever use it for anything. dnsmasq is capable of doing everything you want, and is probably one of the most stable pieces of software I've used over the last decade or so, and I would probably just stick to using it unless I had a really good reason to do otherwise.

Can you post logs and what version you're using? It might be easier to fix it (and I've spent time looking at the internals of dnsmasq), so I might be able to help. What exactly happens?
 
  • Agree
Reactions: teriyakiburns and Vecr
Weapons of Mass Tardation said:
systemd-resolved is a total pile of unmitigated bullshit, and I there's no way I would ever use it for anything. dnsmasq is capable of doing everything you want, and is probably one of the most stable pieces of software I've used over the last decade or so, and I would probably just stick to using it unless I had a really good reason to do otherwise.

Can you post logs and what version you're using? It might be easier to fix it (and I've spent time looking at the internals of dnsmasq), so I might be able to help. What exactly happens?
Click to expand...
it may be unmitigated bullshit, but it's unmitigated bullshit that is tightly integrated into Ubuntu and Fedora and it's more pain then it's worth to disable core system functions just because you want to install an app solely for use by other clients on the network.

As it is, everything works except Cockpit, which is convinced the server is offline when it's clearly not
1733698027975.png

Also I don't know why, but trying to launch this Workhorse vm crashes libvirtd and it cannot be restarted until you reboot the server. happened after i deleted the drive it boots from, and added the install disk, virtio driver disk, and a raw disk it was to be directly installed onto
After further investigation it is due to giving the windows vm direct access to the secondary GPU, which was literally the whole point of this server upgrade. I can create a new Windows VM and set it up the way I want, but as soon as i added the Nvidia gpu and try to start it libvirtd crashes and is unrecoverable until system reboot.

...Oh i see, the difference was that before when it worked, i didn't have the nvidia drivers installed on the host so it wasn't preventing the vm from grabbing it
I found a much better solution for a local dns server: https://github.com/TechnitiumSoftware/DnsServer
it uses unbound and is easy to set up to ignore systemd-resolved, and worked perfectly in my testing vm without messing with a bunch of system configurations. I have a prepared docker config file in my server to do the switch over once i figure out how to re enable systemd-resolved. Or maybe i'll deal with that the next time i do a server upgrade or reinstall.
 
Last edited:
Now, it only took me about ten seconds to figure out what I did wrong here, but I'm sorry, you can't justify this logic.
Code: 
amuhammed@t480:~$ sudo cryptsetup luksFormat /dev/sdd
WARNING: Device /dev/sdd already contains a 'isw_raid_member' superblock signature.
WARNING: Device /dev/sdd already contains a 'PMBR' partition signature.

WARNING!
========
This will overwrite data on /dev/sdd irrevocably.

Are you sure? (Type 'yes' in capital letters): yes
Operation aborted.
It would have been trivial for the guy who wrote this code to instead print 'Operation aborted- you typed '$input' not 'YES'. But then you wouldn't be able to jerk off to the thought of all the people copy and pasting this output into forums for decades into the future.
 
  • Horrifying
  • Lunacy
Reactions: Vecr and Jang Joo
⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛ said:
Now, it only took me about ten seconds to figure out what I did wrong here, but I'm sorry, you can't justify this logic.
Code: 
amuhammed@t480:~$ sudo cryptsetup luksFormat /dev/sdd
WARNING: Device /dev/sdd already contains a 'isw_raid_member' superblock signature.
WARNING: Device /dev/sdd already contains a 'PMBR' partition signature.

WARNING!
========
This will overwrite data on /dev/sdd irrevocably.

Are you sure? (Type 'yes' in capital letters): yes
Operation aborted.
It would have been trivial for the guy who wrote this code to instead print 'Operation aborted- you typed '$input' not 'YES'. But then you wouldn't be able to jerk off to the thought of all the people copy and pasting this output into forums for decades into the future.
Click to expand...
These are the same people that made rm -Rf / require a --i-am-not-a-moron flag.
 
⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛ said:
Now, it only took me about ten seconds to figure out what I did wrong here, but I'm sorry, you can't justify this logic.
Code: 
amuhammed@t480:~$ sudo cryptsetup luksFormat /dev/sdd
WARNING: Device /dev/sdd already contains a 'isw_raid_member' superblock signature.
WARNING: Device /dev/sdd already contains a 'PMBR' partition signature.

WARNING!
========
This will overwrite data on /dev/sdd irrevocably.

Are you sure? (Type 'yes' in capital letters): yes
Operation aborted.
It would have been trivial for the guy who wrote this code to instead print 'Operation aborted- you typed '$input' not 'YES'. But then you wouldn't be able to jerk off to the thought of all the people copy and pasting this output into forums for decades into the future.
Click to expand...
to be fair it was pretty clear it was expecting YES
 
  • Like
Reactions: Vecr
Betonhaus said:
to be fair it was pretty clear it was expecting YES
Click to expand...
NO because I already typed the original command and shouldn't have to waste time re-re-reading stupid bullshit prompting me to type 'yEs I rEaLyDO!!!1'. An extra step which would not under any circumstances prompt me to go back to the man page and re-check the original command I deliberately typed because I intended to run it. It should just accept literally anything that begins with 'y' or 'Y'. It's like the bullshit in the AWS web console that makes you copy and paste random shit like 'permanently delete' into the confirmation box, rather than just having 'yes' and 'no' buttons for dangerous operations.
 
⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛ said:
NO because I already typed the original command and shouldn't have to waste time re-re-reading stupid bullshit prompting me to type 'yEs I rEaLyDO!!!1'. An extra step which would not under any circumstances prompt me to go back to the man page and re-check the original command I deliberately typed because I intended to run it. It should just accept literally anything that begins with 'y' or 'Y'. It's like the bullshit in the AWS web console that makes you copy and paste random shit like 'permanently delete' into the confirmation box, rather than just having 'yes' and 'no' buttons for dangerous operations.
Click to expand...
The command appears to format you drive. It's probably the only reason why you haven't suffered massive data loss for not reading the warnings on commands that you mistyped or did something you weren't expecting.
 
  • Like
Reactions: Rusty Metal Skull Gun
Betonhaus said:
The command appears to format you drive. It's probably the only reason why you haven't suffered massive data loss for not reading the warnings on commands that you mistyped or did something you weren't expecting.
Click to expand...
If only there were some way to tell the computer "Hey, I know this is a dangerous command, I want to run it as an admin."

Sure, prompt if there's something off, but don't make the prompt retarded.

Wonder how long until dd tells me I can't write /dev/zero to a device.

I briefly tried to check the commit history but that dumbness has been in cryptsetup for many years. Probably came from RedHat who is the same company that decided 'alias rm="rm -i"' was a good default for all users.
 
  • Like
Reactions: Rusty Metal Skull Gun
Betonhaus said:
The command appears to format you drive. It's probably the only reason why you haven't suffered massive data loss for not reading the warnings on commands that you mistyped or did something you weren't expecting.
Click to expand...
Entirely disagree. Have you ever run a command that formats a partition like mkfs.ext2 or mkfs.msdos? They don't provide any warnings and tell you to cross your toes upside down and rewrite the command in mixed case with Oxford commas, because, when you're running them, you check what you were typing before hitting enter. Of course, the original 'mkfs' commands on Unix v1 didn't prompt you to confirm that you really meant what you typed, so it's traditional (and thus inherently good) not to ask the user again.

Now, for a newfangled command like cryptsetup, I don't object to having a prompt displayed if it is detected that the drive you're writing a LUKS container to isn't completely blank but has partition signatures. That prompt, properly designed in a essential executable for any computing machine, looks like this:
Code: 
Really quit? [yn] (n)
y and Y quit, n and N and return keep you in nethack, anything else does nothing until you send one of the keys it responds t.

I'm not saying it has to be a finely honed instrument of perfection. I'm just saying if it's something that is totally irrelevant to a user who would actually run the command (because they already ran 'lsblk' to check they really had the device id's right and 'cfdisk /dev/sdd' just to full, fully confirm that they weren't operating on a system disk), maybe at least provide a USEFUL error message like 'Operation aborted- you typed 'YES I FUCKING MEANT WHAT I SAID' not 'YES'. rather than just 'Operation aborted'. Even 'Operation aborted- read the instructions @g@1n y0u fu<k1ng n00b l0lz' would be more helpful than 'Operation aborted' which suggests my external drive dock disconnnected part way through writing the LUKS headers or something.
 
  • Like
Reactions: Weapons of Mass Tardation and Jang Joo
⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛ said:
NO because I already typed the original command and shouldn't have to waste time re-re-reading stupid bullshit prompting me to type 'yEs I rEaLyDO!!!1'. An extra step which would not under any circumstances prompt me to go back to the man page and re-check the original command I deliberately typed because I intended to run it. It should just accept literally anything that begins with 'y' or 'Y'. It's like the bullshit in the AWS web console that makes you copy and paste random shit like 'permanently delete' into the confirmation box, rather than just having 'yes' and 'no' buttons for dangerous operations.
Click to expand...
Run wipefs -af on the device first then. The warning is reasonable.
 
You must log in or register to reply here.
Back