Archival Tools - How to archive anything.

  • 🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.
btw if you are using ffmpeg nvenc, you need to use cq option instead of crf for constant quality flag. watch the reported bitrate. https://trac.ffmpeg.org/wiki/HWAccelIntro says you get much faster encoding (ymmv) at the cost of worse quality.

if you have lots of time to run encodings and want to compress as much as possible to get under 200 MB, I would guess h265/AV1/VP9 as best video codec and Opus as best audio codec. https://techoverflow.net/2021/12/31/how-to-re-encode-videos-as-h-265-opus-using-ffmpeg-for-archival/
my ffmpeg from ubuntu 20.04 is incredibly slow with av1 experimental mode at 0.004x speed. vp9 encodes at 0.5x which is ok, h265 encodes at 2.5x, and h264 as a baseline encodes at 7x.
there is also 2-pass mode which probably should be used in vp9 and other modes to squeeze the most data out.
 
Last edited:
awoo said:
btw if you are using ffmpeg nvenc, you need to use cq option instead of crf for constant quality flag. watch the reported bitrate. https://trac.ffmpeg.org/wiki/HWAccelIntro says you get much faster encoding (ymmv) at the cost of worse quality.

if you have lots of time to run encodings and want to compress as much as possible to get under 200 MB, I would guess h265/AV1/VP9 as best video codec and Opus as best audio codec. https://techoverflow.net/2021/12/31/how-to-re-encode-videos-as-h-265-opus-using-ffmpeg-for-archival/
my ffmpeg from ubuntu 20.04 is incredibly slow with av1 experimental mode at 0.004x speed. vp9 encodes at 0.5x which is ok, h265 encodes at 2.5x, and h264 as a baseline encodes at 7x.
there is also 2-pass mode which probably should be used in vp9 and other modes to squeeze the most data out.
Click to expand...
Guide:
1. Download videos.
2. Handbrake(it's a program for Windows). Audio and frame rate can be dumped a bit just like resolution to reduce video size.
3. Still can't get under the upload limit? Avidemux will do a lossless split of the original video for you, or the resulting video after re-encoding.
4, Encoding something to a lower resolution is usually very fast.
 
Smaug's Smokey Hole said:
Guide:
1. Download videos.
2. Handbrake(it's a program for Windows). Audio and frame rate can be dumped a bit just like resolution to reduce video size.
3. Still can't get under the upload limit? Avidemux will do a lossless split of the original video for you, or the resulting video after re-encoding.
4, Encoding something to a lower resolution is usually very fast.
Click to expand...
But why did you quote me. I know what handbrake is, just a silly GUI for ffmpeg
 
  • Horrifying
Reactions: Mr. 0
Alright so I'm too lazy to do all that coding shit so I just use youtube.dlg and from what I've seen in this thread, I've yet to see this (at least from what I've seen).
In order to archive longer videos, here are some simple steps.
>Gear on the top right >Options >Formats >Video Formats
From there, you will see this massive list of options. For the love of God, choose MP4 [360p]
 
  • Like
Reactions: ⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛ and NOT Sword Fighter Super
Thanks for Kdenlive. I'm a total boomer with editing software and most of my self-made clips (excluding Movie Maker ones lol) are always ridiculous sizes, even short ones, so this is a great thing to learn. Will use in the future here too.
 
  • Agree
Reactions: BoJack Horseman
2 Mbps for a 1080p60 AVC video?! Nigger, are you shitting us? I mean, with HEVC or VP9 it wouldn't be that bad, but this is AVC! That's gonna look like absolute shit for anything you'd want 60 FPS for! Unless you're using some super slow quality preset for encoding.
You're way better off just admitting defeat and downscaling it to 1024x576. Trust me, in my experience lower resolution videos are of better quality than higher resolution videos at the same bitrate.
BTW, why is there no HEVC support? Lack of HEVC support is quite gay. One might say "B-but, HEVC isn't royalty free, unlike VP9!", but neither is AVC, and yet it's fair game! And HEVC is more efficient as well.

Seriously, I fucking can't stand people trying to cram 1080p video into bitrates lower than VCD bitrates! It's gonna be filled with compression artifacts and will look like Canale Italia 83 and similar, mostly regional, Italian trash televendite channels with bitrates so low, a VHS recording would still make them visible!

TL;DR downscaling a video and proportionally reducing bitrates should be prioritized over bitrate crushing the video while leaving the resolution intact.
 
  • Like
Reactions: duckbutter&toejamsandwich, BoJack Horseman, ⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛ and 1 other person
Cavalier Cipolla said:
2 Mbps for a 1080p60 AVC video?! Nigger, are you shitting us? I mean, with HEVC or VP9 it wouldn't be that bad, but this is AVC! That's gonna look like absolute shit for anything you'd want 60 FPS for! Unless you're using some super slow quality preset for encoding.
You're way better off just admitting defeat and downscaling it to 1024x576. Trust me, in my experience lower resolution videos are of better quality than higher resolution videos at the same bitrate.
Click to expand...
2mbp 1080p60 CAN work if there is no movement, like youtube talking head videos where someone zonked out on anti-psychotics drone about whatever. If there's movement then ~1.5-2 mbps at 960x540 is by far the better choice(dumping the resolution like you say is a far the better choice and I think 540p looks fine). Another thing to consider is what would be lost if the hypothetical video wasn't 60fps? Are any important details or anything of value lost if it runs at 30?
 
Recently learned two things that readers of this thread might find helpful:

Email savepagenow@archive.org, it'll archive any links you send it. Tried it with a list of about 20 and it took a little while to come back but eventually did and works fine. Due to the delay I wouldn't use it for something that's at high risk of being imminently altered/removed, but for other uses it's a comfortable way of batch archiving without having to install any specialized tools.

You can use archive.today to access -and archive- Twitter /following, /followers, /likes pages without having an account. Site has been working like shit but if you manage to make it work it's a pretty cool usecase.
 
  • Thunk-Provoking
  • Informative
  • Like
Reactions: notafederalagent, BoJack Horseman and (((Obar Lazcano Kamz)))
8
Smaug's Smokey Hole said:
2mbp 1080p60 CAN work if there is no movement, like youtube talking head videos where someone zonked out on anti-psychotics drone about whatever. If there's movement then ~1.5-2 mbps at 960x540 is by far the better choice(dumping the resolution like you say is a far the better choice and I think 540p looks fine). Another thing to consider is what would be lost if the hypothetical video wasn't 60fps? Are any important details or anything of value lost if it runs at 30?
Click to expand...
Well, let's say that the video in question is some news report from a TV channel, those being almost always 50 or 60 FPS. Or perhaps some sort of VHS or camcorder rip that was also in 50 or 60 FPS. Then it would be more accurate to retain the smooth framerate.
But in those cases you might as well retain 480p60 or 576p50, because it's not like JewTube that requires at least 720p for higher framerates.
In those cases 2 Mbps with VP9 will be plentiful for a video that isn't a macroblocking mess.
My rule of thumb is to use bitrates similar to those our digital terrestrial TV uses or used. For 576i50 4-5 Mbps for MPEG-2, 2 Mbps AVC, and 1.5 Mbps for HEVC or VP9.
 
  • Like
Reactions: Smaug's Smokey Hole
I have a question i want archive a youtube channel i know how to do it but how should i store it for others to potentially download ?
 
As someone who has kept up to date with the status of youtube-dl, they found new leadership but it seems yt-dlp is adding features quicker. So both are viable but I use yt-dlp for now.

AutistNofrens said:
I have a question i want archive a youtube channel i know how to do it but how should i store it for others to potentially download ?
Click to expand...
archive.org is a good first place. If it's an important enough lolcow, you can ask Null to host it but I would only do that for important lolcows that have stuff archive.org doesn't like (ex. copyrighted stuff and stuff that the video creators could ask to be taken down)
 
  • Informative
Reactions: fartsnstuf
You must log in or register to reply here.
Back