Home Server and Self Hosting General - Technological Self-Sufficiency

[Civillian 18]

ve considered hosting a Mineclone or Veloran server for online aquaintances to get on together but I don't have a reverse proxy to put my dynamic IP behind.

If you just wanted to solve this, you can buy those weird 8 digit '.xyz' TLDs for like $2 and then set up Caddy2 or Nginx with a bash script or ddclient or something to update your DNS records to whoever you pick to manage that.

 

[tehpope]

So i'm an insane autist with a digital hording problem. I'm planing an upgrade to my current nas. Biting the bullet, eventually, and using zfs. I plan on using a Fractal Design Define 7 XL. Which, if you don't know, is a case capable of using 20 3.5'' drives. Mostly like will use like 12 24tb drives, at first. In separate RAID z2 6 drive vdevs. Just to lessen damage done by resilvering.

Its probably going to be two servers. one for proxmox in a cooler master qube 500 case. another one for truenas scale. both will run off of a lga 2011-3 platform. I'm toying with the idea of using am4 for proxmox. Faster cpus and rebar for a possible gaming vm. But I'm concerned about pcie lanes. And lga 2011-3 has rdimm ecc ram support. You don't need ECC for ZFS, but the ram for 128gb is cheap.

I started the journey the other day. Found a deal on ebay for a LSI 9405w-16i for $100. Mini SAS to Sata cables included. I also got a LSI 9300-16i for $25 cause some point down the line, whenever I can find and purchase the rare as shit nvme u.2 cables, I might add some u.2 drives for slog and metadata. At this point, I'm most likely going to use SAS or SATA enterprise SSDs. And even if I don't, I will need another HBA to use 20 drives.

I'm concerned with powering all those drives in my truenas server. But apparently just using some SATA splitters works fine. Something like this: https://www.amazon.com/dp/B012BPLW08. And also to make sure that the power supply can handle them. I plan on getting a 80 plus gold one.

edit: I didn't explain why I'm building a proxmox server. Mainly for home lab self hosted shit. Mainly home assistant, openwrt, paperless, jellyfin. Shit like that. A possible gaming vm down the line.

 

[DavidS877]

So i'm an insane autist with a digital hording problem. I'm planing an upgrade to my current nas. Biting the bullet, eventually, and using zfs. I plan on using a Fractal Design Define 7 XL.

Just get a rack case like a normal person. Which also makes the cabling far simpler. Not sure what the options for U.2 are in decent priced rack mounts though.

 

[tehpope]

Just get a rack case like a normal person. Which also makes the cabling far simpler. Not sure what the options for U.2 are in decent priced rack mounts though.

I'm trying to do this on a budget. And I don't have the space for a server rack. Thus why I'm going with lga 2011-3 and not threadripper or epyc.

 

[DavidS877]

I'm trying to do this on a budget. And I don't have the space for a server rack. Thus why I'm going with lga 2011-3 and not threadripper or epyc.

True, they would be about $250 more, but they don't need a rack, they sit on a desk just fine and looking at my Define 7 XL, I think my 4u rack cases are smaller. Actually, I don't know how much all the drive caddies for the Define would cost, that could shrink the price gap. Depending on the model some rack cases you could take off the handle and put it on its side.
I ended up with a Define just so I could have 2 multi-slot GPUs without a riser. It was the easiest one to find that I knew would work, looks like I could probably have gotten something smaller though.

 

[tehpope]

I'm open to suggestions. This is in flux. I've only bought a few parts. If I see a good deal pop up locally, I might snatch it.

 

[teriyakiburns]

Just managed to score myself a d-link dgs-1210-16 managed switch for an absolute pittance in an auction. I don't think I'll use the management all that much, but I can see some potential fun to be had with vlans and the like. Finally saying goodbye to the stack of cheap switches everything was talking through before.

 

[Linksys WRT]

Just managed to score myself a d-link dgs-1210-16 managed switch for an absolute pittance in an auction. I don't think I'll use the management all that much, but I can see some potential fun to be had with vlans and the like. Finally saying goodbye to the stack of cheap switches everything was talking through before.

First managed switch? The possibilities open up like never before.

If you play with it, post how you like it. I don't hear too much about D-Link.

 

[Barry McKockner]

Just managed to score myself a d-link dgs-1210-16 managed switch for an absolute pittance in an auction. I don't think I'll use the management all that much, but I can see some potential fun to be had with vlans and the like. Finally saying goodbye to the stack of cheap switches everything was talking through before.

My house would become the home of VLANs. Too much shit runs on IoT nowadays.

 

[Linksys WRT]

I am a bit disappointed to see that this thread doesn't get more activity, so I will sperg a bit about my setup and my stupid problems.

Recently, OPNSense added support for Dnsmasq DHCP/DNS which I've used in small "I need this shit to work and I need it now" setups before. But my OPNSense experience has been a bit troublesome. I'm trying to forward DNS requests through Dnsmasq and on to Unbound for recursive name resolution, but it seems like sometimes DNS just doesn't move quickly enough. Sites start telling me that there's certificate problems (there aren't) and things can take a moment to load before DNS is cached. As soon as it's cached, it works just fine. It's happening on just about every device, even my recently-acquired Switch 2. I can just wait a moment and try again after the record is cached, but that just sucks and I don't want to deal with that.

To this end, I was thinking that I might completely ditch Dnsmasq DNS and just have it all in Unbound, but this comes with a caveat - DHCP leases won't have their names registered in Unbound, so local name resolution won't work. I don't really do much name resolution on my home network, but that's a crappy trade off.

I need to do some more testing to ensure I know where the problem lies, but fucking with DHCP/DNS can be a dangerous game. I don't want to lock myself out of my router as I am sometimes prone to doing, but I did make at least one smart move which was configuring an emergency management port. It runs a DHCP server on the interface so I can just plug in and get an IP. Highly recommended for retards like myself.

 

[DavidS877]

DHCP leases won't have their names registered in Unbound

I assume that's because you want to use Dnsmasq DHCP or Kea and not ISC.

On the other issue I usually see this sort of behavior(generally,not Dnsmasq specific) when there's a bad resolver tried first before the proper one. So make sure that unbound is responding correctly and quickly and then Dnsmasq is querying correctly. Querying each directly, presumably from the OPNSense command line should give you an idea of which one is broken.

 

[Slav Power]

Today I picked up a brand new Netgear GS108Ev4 switch. Cheapest I could find that would have 8 ports and management. Did a rudimentary VLAN config and it pairs with my new Mikrotik hAP ax3 rather nicely for a router on a stick setup. About time I upgraded my home network to something slightly nicer, especially after upping the Internet speed. I'll probably try and mess around with separate VLANs and isolation later, like for plugging in machines that I don't want to see the rest of my network and such. Definitely good to actually learn how the fuck do VLANs work since I always had issues with them.

As for the WiFi in the hAP ax3, it's good enough for me. Can't really say much as I only have one device with WiFi 6E and the rest is WiFi 5 max, but it does pull ~700Mb/s on Librespeed next to it whereas previously my hAP ac2 would cap out at 300Mbps so it's an improvement. And obviously the range is better as well. I guess if you really care about WiFi you'd just get something like an RB5009, if not a CCR, and then get Unifi AP's to fill in the blanks, Mikrotik always sucked at WiFi and the MT+UB combo is a very common one.

 

[Linksys WRT]

I assume that's because you want to use Dnsmasq DHCP or Kea and not ISC.

On the other issue I usually see this sort of behavior(generally,not Dnsmasq specific) when there's a bad resolver tried first before the proper one. So make sure that unbound is responding correctly and quickly and then Dnsmasq is querying correctly. Querying each directly, presumably from the OPNSense command line should give you an idea of which one is broken.

Yeah that's exactly it, but I specifically want Unbound DNS and Dnsmasq DHCP. Getting them to play nice is the goal.

If this behavior isn't Dnsmasq specific, then that's a good sign. It's got to be a me problem. I'll try some troubleshooting when I get a bit of acceptable downtime-time, querying each individually like you said. Probably is the path of least resistance to nail down the problem.

Mikrotik hAP ax3

Love that thing. I have one and it's just my WiFi access point right now, but it works great. I was debating making it my main router but I really like OPNSense because of open source autism.

I'm able to cap my bandwidth over WiFi, but I've only got 300Mb. I have gigabit on everything locally, but I don't think I've ever even tested it to see how fast it is. I only really stream some shows and play the odd game or two, so for that it's more then enough.

I don't know anything about the Netgear stuff because I only ever had a dumb switch from them, but if you're looking for a bit more down the road, Mikrotik makes some really great switches. I have a CRS326 from them and I love it. I really can't suck Mikrotik off enough. I just love how they let you do anything you want. DHCP on a switch? Sure. DNS? Of course. BGP? Why not?

 

[Slav Power]

I don't know anything about the Netgear stuff because I only ever had a dumb switch from them, but if you're looking for a bit more down the road, Mikrotik makes some really great switches.

I'm sure they do, it's just that I wasn't looking for too much and that Netgear was the best bang for the buck that I could find on wisp.pl given what I was looking for. Something with a few extra Gigabit Ethernet ports and something with VLANs so that I can do a RoaS setup.

I really can't suck Mikrotik off enough.

Same, instantly became a fanboy of them after getting the hAP ac2. Though at the same time I can't shit on MikroTik enough for everything that they do wrong. Releasing sub-par ax hardware once Unifi and the rest is pumping out be hardware, always being lackluster in the 2.5G port availability, overly relying on their passive PoE instead of standardized solutions, releasing the L009 as a replacement for the RB2011 that is on the same level of performance as the hAP ax lite but at 2x the price, using their own WiFi drivers that were trash until moving to Qualcomm ones with the ax models and then redoing the wireless packages to allow for the Qualcomm drivers to be used on older ac devices in ROS v7.13 and other sins like that.

hAP ax3 is definitely the best bang for the buck they have on offer though. Basically the same super powerful guts as the cAP ax but in a SoHo form factor so you get 4 1G ports and 1 2.5G port, you could put it in your living room and it would add to decor unlike the cAP ax, and you get RouterOS. I really don't care about open source everything so much if there is not a single open source router firmware that can fit in 16MB of FLASH memory and offer the same amount of functionality. Also my hAP ax3 goes from cold to fully operational in like 40 seconds, try to beat that.

 

[Linksys WRT]

I'm sure they do, it's just that I wasn't looking for too much and that Netgear was the best bang for the buck that I could find on wisp.pl given what I was looking for. Something with a few extra Gigabit Ethernet ports and something with VLANs so that I can do a RoaS setup.

Same, instantly became a fanboy of them after getting the hAP ac2. Though at the same time I can't shit on MikroTik enough for everything that they do wrong. Releasing sub-par ax hardware once Unifi and the rest is pumping out be hardware, always being lackluster in the 2.5G port availability, overly relying on their passive PoE instead of standardized solutions, releasing the L009 as a replacement for the RB2011 that is on the same level of performance as the hAP ax lite but at 2x the price, using their own WiFi drivers that were trash until moving to Qualcomm ones with the ax models and then redoing the wireless packages to allow for the Qualcomm drivers to be used on older ac devices in ROS v7.13 and other sins like that.

hAP ax3 is definitely the best bang for the buck they have on offer though. Basically the same super powerful guts as the cAP ax but in a SoHo form factor so you get 4 1G ports and 1 2.5G port, you could put it in your living room and it would add to decor unlike the cAP ax, and you get RouterOS. I really don't care about open source everything so much if there is not a single open source router firmware that can fit in 16MB of FLASH memory and offer the same amount of functionality. Also my hAP ax3 goes from cold to fully operational in like 40 seconds, try to beat that.

Oh yeah, the WiFi has always been a bit of a weak point for them, but I've always used the Qualcomm drivers, so I can't really complain. As you said, gotta pull out the Ubiquiti devices for the WiFi coverage. I'm pretty sure everyone I know that uses Mikrotik does exactly that, but I figured I'd just spring for the hAP ax3 because I was so happy with the switch and just wanted to play with their stuff more. It suits my needs and hell, it's better than trying to do WiFi off of my Protectli with OPNSense. That runs on top of FreeBSD and good god is it shit.

And you know, I'm not sure how to feel about 2.5G. My ISP will only go up to 2Gig but it's pricey, so I probably wouldn't care to go over 1G for Internet, and by extension, most other things. At that point I might as well just shoot for some 10G SFP+ over fiber locally. It's good to have the 2.5G option, but I don't know if it's even worth buying a new switch for to get more 2.5G ports to make it work. I might as well just spring for the CRS328 so I can get PoE out and some SFP+ for my daily driver, NAS, and a couple other things that are yet to be decided.

 

[Slav Power]

And you know, I'm not sure how to feel about 2.5G. My ISP will only go up to 2Gig but it's pricey, so I probably wouldn't care to go over 1G for Internet, and by extension, most other things. At that point I might as well just shoot for some 10G SFP+ over fiber locally. It's good to have the 2.5G option, but I don't know if it's even worth buying a new switch for to get more 2.5G ports to make it work. I might as well just spring for the CRS328 so I can get PoE out and some SFP+ for my daily driver, NAS, and a couple other things that are yet to be decided.

Honestly, yeah I've been thinking about that when I couldn't sleep and realized it doesn't matter too much in most cases. The smart thing to do is to have the router on a stick topology, where suddenly your router's max transfer speed is only there for the Internet, and for your local machines you just get 10G NIC's and fiber it all up. Usually the best that you can get is 1Gbps, maybe sometimes 2Gbps. The RB5009 has an SFP+ case, and with a LEOX LXT-010S-H you can clone your ISP's ONT and have those 2.5Gbps on SFP+, then use the 2.5G eth1 port to hook it up to your switch. Anything higher than that and you're in the really fucking expensive territory where Mikrotik ceases to matter, especially when ROS still can't multithread PPPoE. In my case, my PC's mobo NIC is 2.5G. I could plug it into the ax3, but it wouldn't matter. Everything else is strictly 1G and my net speeds are 2/3rds of it.

 

[Linksys WRT]

with a LEOX LXT-010S-H you can clone your ISP's ONT

This is the first I'm hearing of this, and it sounds really, really fun. I wonder if my ISP would be pissed off if I did something like that.

So you're telling me that I could theoretically negotiate 2.5G uplink as long as my ONT supports it? I have never played around with alternative ONTs.

Edit: or rather, maybe not the ONT because technically the LEOX LXT-010S-H would be the ONT, but the FTTH infrastructure?

 

[Slav Power]

This is the first I'm hearing of this, and it sounds really, really fun. I wonder if my ISP would be pissed off if I did something like that.

So you're telling me that I could theoretically negotiate 2.5G uplink as long as my ONT supports it? I have never played around with alternative ONTs.

Edit: or rather, maybe not the ONT because technically the LEOX LXT-010S-H would be the ONT, but the FTTH infrastructure?

Technically yes, if your ISP gives you a >1Gbps connection and you clone their ONT with the LEOX SFP+ insert, you can reach those >1Gbps speeds with low latency. Sometimes ISP's will give you some BS ONT that has 1Gbps Ethernet port even though you're paying for more than that, or some retarded dual 1Gbps Ethernet port link aggregation hack. Obviously ISP's tend to be extremely pissy about you being able to use your own hardware, even though back when we relied on telephone lines you had zero issues hooking up your dial-up/ADSL/VDSL modem.

The story of Leox is that of a regional Polish ISP getting so fed up with existing ONT solutions that their CTO, Marcin Kuczera, took it upon himself to develop better ONT solutions so that people can simply buy and use their own hardware regardless of what ISP's have to say. He had some good talks about it, but unfortunately they're all in Polish with no subtitles.

There's an entire community in Poland looking for ways to use their own hardware and sharing all the ways to do so. Of course, depending on the ISP they either don't care about you using a Leox ONT, or they outright block your access. Even though there are EU directives that prohibit it that were either unimplemented or reworded in such a way where they benefit ISP's instead of the consumers as was the initial intention in the English draft. It's basically an on-going topic on how to stop ISP's fucking you over with an underpowered ONT despite the money that you're paying them, and also the ability to only get fiber net without having to rent ISP's hardware.

 

[Quandale Octavius Dingle]

I got another computer about two weeks ago from a guy getting rid of a lot of junk that turned out to be a Dell Precision T5500. Managed to snag 64gb of ECC DDR3-1066 for $20 off ebay for it, but it can only use 48gb of it since it's an LGA1366 socket with 6 ram slots. I'm not sure if it'd be worth upgrading it from the 95 watt quad core Gainestown xeon that's in it now to a 60 watt hexa core Westmere-EP xeon with a better cooler for $30. I'm hesitant to migrate my current NAS setup to it since it would technically be a downgrade from the Ivy Bridge system running it now since the X5570 xeon in it now lacks important instruction sets like AES-NI. I could also buy a daughterboard and throw in a second 60 watt CPU for about $100 but at that point I'm sure I'd be better off getting something like an HP Z420 board with a much newer Haswel CPU.

 

[Linksys WRT]

It's basically an on-going topic on how to stop ISP's fucking you over with an underpowered ONT despite the money that you're paying them

Damn I wish I spoke Polish. Those seem super interesting. And why do Polacks seem so based all the time?

I am in the US so I can bitch to my ISP and they'll come fix it, but I haven't been screwed over yet, thankfully. Even so, I'd love to play around with one of those things because my knowledge starts to end past the ONT. I wonder if those guys don't mind inquiries from private English-speaking individuals...

I wouldn't know how to even figure out which OLT vendor my ISP uses. That ought to be the first order of business.

I got another computer about two weeks ago from a guy getting rid of a lot of junk that turned out to be a Dell Precision T5500. Managed to snag 64gb of ECC DDR3-1066 for $20 off ebay for it, but it can only use 48gb of it since it's an LGA1366 socket with 6 ram slots. I'm not sure if it'd be worth upgrading it from the 95 watt quad core Gainestown xeon that's in it now to a 60 watt hexa core Westmere-EP xeon with a better cooler for $30. I'm hesitant to migrate my current NAS setup to it since it would technically be a downgrade from the Ivy Bridge system running it now since the X5570 xeon in it now lacks important instruction sets like AES-NI. I could also buy a daughterboard and throw in a second 60 watt CPU for about $100 but at that point I'm sure I'd be better off getting something like an HP Z420 board with a much newer Haswel CPU.

For NAS stuff, as long as I have enough RAM to keep ZFS happy and enough compute to run OMV with Jellyfin in a container, I'm satisfied. And I've only got 16 gigs with 80TB of storage. People say not to do that but eh, it's worked fine thus far.

Do you have any use cases for it other than just serving and saving data? I do NFS shares and then I run VMs off my Proxmox cluster and just mount the NFS shares to host whatever I need to, other than Jellyfin. I really try to touch my NAS as little as possible. All the spinning rust in there makes me nervous, but those shitty disks have been happily spinning for years now.