Offline Long-Term Digital Archival - Archiving data for when the Internet cannot be depended on.

[N Space]

I think people should be archiving things the the highest quality that their storage space allows for. I also think considering what windows media player can do at all is foolish.

Yet you refused to answer my question. Compatibility is a personal and arbitrary decision that depends on the devices you plan to stream on.

Who said anything about lossless compression? I am assuming you already have something that is using lossy encoding (like a blu-ray as you mention) and saying that re-encoding this with lossy compression again is a bad idea. You should be keeping it in the original format if possible.

Ideally you wouldn't be re-encoding things with lossy compression.

Au contraire this shows that it's you who doesn't know what they're talking about. First off nothing gets streamed as uncompressed video because of the enormous amount of bandwidth that would take.

Oh really? Is that why HDMI and DisplayPort both support raw video streaming without even chroma subsampling?

 

[SCV]

Yet you refused to answer my question. Compatibility is a personal and arbitrary decision that depends on the devices you plan to stream on.

Compatibility for old devices is easily solved with transcoding. I also think you're trying to muddle the argument here between old devices with limited codec support (like a roku or something) and "windows media player" knowing full well that anything using windows media player will have other better options that support basically any codec.

Oh really? Is that why HDMI and DisplayPort both support raw video streaming without even chroma subsampling?

If you're going to be a fucking dickhead about it why even reply? This whole discussion is clearly about storing media and streaming it over a network which is not going to be uncompressed video, not how your monitor/tv is connected. Or do you have 40Gbit/s home network that you use to stream a single video to one device? I also notice you didn't respond re:transcoding.

Anyway if you're going to argue in bad faith because you're upset I'm not going to bother replying. Goodnight.

 

[Jotch]

based on windows media player is an utter fucking retard

are you ChatGPT? everything you said is just entirely made up bullshit.

If you're going to be a fucking dickhead about it why even reply?

The post was kinda rude to begin with.

 

[SCV]

The post was kinda rude to begin with.

I guess I could've been more clear. By dickhead I meant how dumb it is to try to conflate IP network streaming and displayport/HDMI as some kinda gotcha (rather than a complaint about tone). It's so obviously flailing to try an "win" an internet argument and so nonsensical that it's not going to fool anyone. Why bother?

I stand by what I said regarding making decisions based on windows media player.

Edit:
A bit more on topic - An old computer + linux + whatever case will fit your HDDs + a checksumming filesystem* + samba/jellyfin/nfs/plex/whatever program is the gold standard for online storage. Having 14 externals that slowly go out like lightbulbs or getting suckered into paying 300$ for an synology appliance that won't be supported in 3 years or opening a bunch of files on your exFAT drive that are now blank are tragedies.

*zfs or btrfs. I prefer btrfs for it's flexibility of drive sizes and because it's in the kernel. Compression is super nice on both.

 

[Casshern]

I stand by what I said regarding making decisions based on windows media player.

I feel part of the problem here is you’re hyperfocused on consooming content via network streaming, but a major part of this thread is about general archiving without the Internet, and the entire subforum is dedicated to SHTF. There are absolutely files I want to be quickly and natively available to every device I have, that I want to have backup copies of on every device and NOT via LAN. First aid videos, home movies, MATI, etc.

What do you do if you’re out in the field, Internet is down, and thus you can’t access your network and get the video you need in that moment?

 

[Mr. Pestilence]

If you wanna be a real data hoarder, look into a IBM TS1150 tape reader/writer, they're a decade old at this point so you can find them cheap if you know where to look. They use 3rd and 4th gen IBM 3592 tape cartridges, Ive seen 20 packs of 10TB tapes go stupid cheap, like $20-40 cheap so look for that if you can. Don't be an idiot that spends 100+ for a single tape off ebay.

 

[SCV]

I feel part of the problem here is you’re hyperfocused on consooming content via network streaming, but a major part of this thread is about general archiving without the Internet, and the entire subforum is dedicated to SHTF.

I think that perception is due to intentional muddying the waters between transcoding and specifically windows media player capabilities. I said as much:

I also think you're trying to muddle the argument here between old devices with limited codec support (like a roku or something) and "windows media player" knowing full well that anything using windows media player will have other better options that support basically any codec.

There are absolutely files I want to be quickly and natively available to every device I have, that I want to have backup copies of on every device and NOT via LAN. First aid videos, home movies, MATI, etc.

What do you do if you’re out in the field, Internet is down, and thus you can’t access your network and get the video you need in that moment?

If you want a copy of whatever file on multiple devices and to be able to play those files locally you will need to ensure each device has a player capable of playing that file (big insight, I know, but bare with me). For any device that isn't a windows PC what windows media player can or can't do is entirely irrelevant and any concessions made for that ability (ex. file size on your "in the field" device which is probably a phone or table where space is at a premium) are worthless. On top of this finding a good player on other platforms is harder than on windows so again what you're getting isn't really worth much.

This whole conversation has been kinda bizzare because even using windows media player is IMO basically a meme. It's so laughably bad (see: pay for h265 codecs) and the alternatives are free, well known, significantly better, and easily available. I don't understand how you could save a significant amount of video while worrying about things like re-encoding with lossy compression, what formats are supported, etc. and still be using windows media player. It'd be like designing a nuclear power plant and doing all the calulations an an abacus.

MPV, VLC etc. also aren't tied to windows. That seems like another obvious point to make but literally the only time ensuring windows media player can play your files is useful is if you had a windows PC with no other media player installed.
Phone/Tablet/other "in the field" device? Worthless.
Linux or Mac PC? Worthless.
Windows PC with literally any alternative? Worthless.
And what do you give up for this wonderous ability? If we consider the aforementioned lack of x265 and say we're going to do h264 to retain compatability you're looking at 20-30% larger files. Is this a worthy trade off? A copy of the k-lite codec pack is a couple hundred Megabytes to potentially save you Terabytes.

To sum up my horribly long winded rant:
If you're using windows media player you're a fucking retard. If you're making archival decisions baed on windows media player you're an utter fucking retard.

 

[cock gobbler]

WRT physical media for offline storage, I'm surprised no one has mentioned LTO tapes/cartridges yet. NVM, @Mr. Pestilence and I had similar thoughts at the same time, though I'm talking about the $100+ tapes. I already wrote this up, so I'll post anyway...

Spoiler: LTO Info

Current-gen (LTO-9) gets 18TB uncompressed/45TB compressed, per $100 tape. Upfront cost for a current-gen tape drive is expensive both new and used, but the media is designed for long-term, business-grade archival. You can expect tapes to last at least 2 decades, if not longer. Also does ECC. Transfer speeds are somewhat slow at <500MB/s, but this is for archival, not instant retrieval or running a server. Power consumption is somewhat low at <40W continuous, meaning drives can easily run off of a UPS in no-power situations.

If drive cost is a concern, try going back a generation or two (check the Wikipedia article to find storage capacities). Used LTO-8 drives go for about 1/3 the price of LTO-9 drives. LTO-8 tapes sell for about 1/2 the price of LTO-9 tapes at 12TB uncompressed/30TB compressed.

As far as backwards compatibility goes, newer tape drives will read tapes from previous generations. LTO cartridge form factor hasn't changed since its inception in 2000.

To be clear, LTO's application lies in keeping an offline backup of all of your important files. I.e. it is for parts 3 and 2 of your 3-2-1 backup strategy.

I can't list distinct differences between IBM 3592 and LTO/"Ultrium", since I'm only familiar with HPE's terminology. This page may be a good starting point (check the "Storage" column under "Single tape devices"). Ultimately, the decision would be yours to make between your money, sense, and storage capacity needs. In general, adding any tape carts to your backup strategy is less expensive and more reliable than filling the same amount of storage with spinning/solid-state media.

When I move back I'm going to start collecting my favorite albums on vinyl like a fag

I do this lmao. I also keep a couple handfuls of albums I've never listened to, for the express purpose of having 'new' music in the event of a prolonged outage.

 

[SCV]

@cock gobbler
You probably already know this but the LTO "compressed" values are basically fairy tales that assume better than 50% compression. That's really only achievable with text and for anything already encoded (video, audio, images, literally anything that isn't uncompressed text, etc.) you'd be lucky to see 5%. I wouldn't even mention them or include a disclaimer about it.

 

[N Space]

If you want a copy of whatever file on multiple devices and to be able to play those files locally you will need to ensure each device has a player capable of playing that file (big insight, I know, but bare with me). For any device that isn't a windows PC what windows media player can or can't do is entirely irrelevant and any concessions made for that ability (ex. file size on your "in the field" device which is probably a phone or table where space is at a premium) are worthless. On top of this finding a good player on other platforms is harder than on windows so again what you're getting isn't really worth much.

This whole conversation has been kinda bizzare because even using windows media player is IMO basically a meme. It's so laughably bad (see: pay for h265 codecs) and the alternatives are free, well known, significantly better, and easily available. I don't understand how you could save a significant amount of video while worrying about things like re-encoding with lossy compression, what formats are supported, etc. and still be using windows media player. It'd be like designing a nuclear power plant and doing all the calulations an an abacus.

The reason why I care about WMP is precisely because it's bad. Are you even thinking about compatibility if you'll choose your codec based on highly capable media players? I already have a FOSS media player with virtually universal support for video, audio and subtitle codecs (FFplay) yet it'd be foolish to think compatibility is a solved issue because it can play my media. As I said, compatibility is an arbitrary and personal decision, not an objective one.

And what do you give up for this wonderous ability? If we consider the aforementioned lack of x265 and say we're going to do h264 to retain compatability you're looking at 20-30% larger files. Is this a worthy trade off? A copy of the k-lite codec pack is a couple hundred Megabytes to potentially save you Terabytes.

WMP reproduces AV1 encoded video which is my preference, in that regard compatibility is barely a compromise since only VVC beats it in terms of efficiency, so answering your question, I give up nothing.

If you're going to be a fucking dickhead about it why even reply?

Were you expecting me to be polite after calling me a retard and bitching at me for the most autistic of reasons?

 

[SCV]

Dude that post didn't quote you, tag you, or even mention you. If anything it made me look stupid because it was way too long for the idea I was trying to get across.

WMP reproduces AV1 encoded video which is my preference, in that regard compatibility is barely a compromise since only VVC beats it in terms of efficiency, so answering your question, I give up nothing.

That was just a general example because h264 is widely supported. But if we must:

Opus really is the endgame audio codec beating any other codec over a wide range of bitrates and having very low latency, however, I've chosen to use AAC instead since it's a lot older and enjoys more support, for example, Windows media player can't play MP4 files with opus audio.

You, yourself, admit you are using inferior encoding specifically for compatibility with windows media player.

The reason why I care about WMP is precisely because it's bad. Are you even thinking about compatibility if you'll choose your codec based on highly capable media players? I already have a FOSS media player with virtually universal support for video, audio and subtitle codecs (FFplay) yet it'd be foolish to think compatibility is a solved issue because it can play my media. As I said, compatibility is an arbitrary and personal decision, not an objective one.

This is a bizarre statement. Of course I make decisions on compatibility based on "highly capable media players". I have MPV and it can play everything I have currently. Unless I somehow lose MPV and then only have access to a stock windows machine then it's a solved problem. Do you foresee a situation where you somehow keep your archive but only have stock windows? If that is something you are worried about, are you so brain damaged you can't simply include a copy of the k-lite codec pack or whatever you prefer in your archive? Honestly baffling.

Were you expecting me to be polite after calling me a retard and bitching at me for the most autistic of reasons?

I know I didn't tag you in this other post either but I'm weirdly certain you would've seen it:

I guess I could've been more clear. By dickhead I meant how dumb it is to try to conflate IP network streaming and displayport/HDMI as some kinda gotcha (rather than a complaint about tone). It's so obviously flailing to try an "win" an internet argument and so nonsensical that it's not going to fool anyone. Why bother?

I think you've been pretty soundly btfo regarding transcoding and uncompressed video. And while you're right that compatibility (along with anything else you do with your data and hardware) is a personal decision, it is not an arbitrary decision because there are easily quantifiable benefits and drawbacks.

I can honestly say I when I started reading this thread I didn't expect anyone to die on the hill of "my archives should be compatible with windows media player! nooooo it's an arbitrary decision!!" or say "uh, Displayport and HDMI are uncompressed, chud. Bet you feel stupid now huh ". I also wasn't expecting anyone to get so asshurt over being called a retard on the farms.


P.S. This is a pedantic tangent with no value but modern DP and HDMI connections will use what's called Display Stream Compression (DSC). Obviously we were discussing streaming over IP and file storage formats not monitor signals but, well, you're technically wrong about those being uncompressed too.

 

[Jotch]

@SCV, @N Space,
I would like to politely ask that you both please stop filling the thread with (non-technical) arguments.

 

[IdiotPlusPlus]

If you wanna be a real data hoarder, look into a IBM TS1150 tape reader/writer, they're a decade old at this point so you can find them cheap if you know where to look.

That might be a region specific thing, where I am tape drives almost never go cheap, that TS1150 for example are all 2-3k used, I've only seen LTO1/2 drives sub $1k last time I looked and may as well use Blurays at that point lol.

Annoys the fuck out of me because I want decent offline non-hdd backups without having to burn a million discs or rely on hdd's that could die in storage rip

 

[Mr. Pestilence]

That might be a region specific thing, where I am tape drives almost never go cheap, that TS1150 for example are all 2-3k used, I've only seen LTO1/2 drives sub $1k last time I looked and may as well use Blurays at that point lol.

Annoys the fuck out of me because I want decent offline non-hdd backups without having to burn a million discs or rely on hdd's that could die in storage rip

If you go to a lot of corporate auctions, you can find tape readers like that way cheaper than 2-3k

 

[Dixieland Buckaroo]

You faggots talking about codecs is laughable! If civilization collapses, this is all worthless unless you record in this shit (which, from the title, is what the thread is about):

That's right, fucking tablets. Clay tablets.

You want to know a prayer song 3700 years ago? Or see a merchant's books from the same time? Want to prove you solved trig 1000 years before the Greeks? Want to extol the virtues of Sargon of Akkad? (Not that one!) Only way it's going to last is putting it in something like clay tablets, and hoping like hell there's not a nuclear blast.

 

[Duiker]

You faggots talking about codecs is laughable! If civilization collapses, this is all worthless unless you record in this shit (which, from the title, is what the thread is about):
View attachment 6170229
That's right, fucking tablets. Clay tablets.

You want to know a prayer song 3700 years ago? Or see a merchant's books from the same time? Want to prove you solved trig 1000 years before the Greeks? Want to extol the virtues of Sargon of Akkad? (Not that one!) Only way it's going to last is putting it in something like clay tablets, and hoping like hell there's not a nuclear blast.

Dont forget that even clay tablets have packet loss. They still don't have all of the Saga of Gilgamesh. But, unironically, this is peak analog and all our internet memes should be preserved on clay tablets in an abandoned salt mine.

 

[geckogoy]

Transcoding is still the answer.

You should keep the originals however they came out of the source (ie yt-dlp -f bestaudio+bestvideo/best) and then maybe toss on the thumbnail and metadata flags if you are feeling fancy. Then transcode off variations on the original source material based on whatever your specific needs are. For instance, I made a script to turn a few video series into thumbnailed MP3s so I can put them on my airsonic server and then cache them from onto my phone for offline listening. or log onto the airsonic from work if I want to. I say don't bother messing with the source format until you know what you actually need.

There isn't currently anything on youtube or anywhere else I can think of that isn't supported by ffmpeg, which makes sense since it just wraps libav which in turn drives almost everything.

 

[Casshern]

Plexniggers get out.

This thread is specifically about archiving media for the Death of the Internet, not how to play back your goyslop tv shows. What codec you think is best or how X media player is the worst belongs elsewhere. This thread has derailed enough.

Dont forget that even clay tablets have packet loss. They still don't have all of the Saga of Gilgamesh. But, unironically, this is peak analog and all our internet memes should be preserved on clay tablets in an abandoned salt mine.

Honestly, when I get a CNC set up I will make clay meme tablets and hide them away somewhere they’ll be safe. Perhaps the Epic of CWC’s Mess should be constructed.

 

[geckogoy]

This thread is specifically about archiving media for the Death of the Internet, not how to play back your goyslop tv shows. What codec you think is best or how X media player is the worst belongs elsewhere.

Didn't even make it to the second sentence of the OP:

This includes every MATI stream recording I could find. As well as youtube channels, movies and TV seasons which I find entertaining/nostalgic.

 

[Casshern]

Continuing on from the clay tablet meme, there is an alchemical method to improving clay deposits that seems rather interesting. Although this video cut out the original recipe‘s goat hair, which I assume worked as both parts filler and support structure, and could be replaced with twine or other similar fibrous materials.

And back to the actual topic of archiving, how many backups does everyone make? Strictly 3-2-1, or do you go even further? Do you have media in different “levels” of backups? Personally most of my media that isn’t crucial just gets a simple mirror, if that. Especially if I own it on some form of physical media. What I really focus on is the irreplaceable things like Grandma’s recipes or family memories and survivalist info like first aid videos, which I have accessible across multiple backups and devices, even a mix of low and high tech.