Archival Tools - How to archive anything.

  • 🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.
Does anyone know how to record Dlive.tv streams with yt-dlp? Whenever I try it it just gives me short videos that are just a static black image or rarely a video of a static image with the Dlive logo and text saying "Watch on Dlive!"

Passing cookies from a browser that has an account logged into Dlive didn't help either. I can't find anything about this issue online. It's driving me mental, I have to resort to using OBS to record my screen like a neanderthal.
 
The Deep State said:
The "true" value in the archived URL has to be changed to "false" for anything to show up, and even if you copy/paste the "false" link, it just defaults back to "true" when clicked on.
Click to expand...
You're supposed to click on Main menu then Archived page not displaying properly? Click here instead of doing this.
 
I wonder if YT-DLP will be able to get around the interference that server side ad injection is apparently causing for the downloading of videos?

If not, a core archival tool will have been lost.
 
Prokhor Zakharov said:
I wonder if YT-DLP will be able to get around the interference that server side ad injection is apparently causing for the downloading of videos?

If not, a core archival tool will have been lost.
Click to expand...
Worst case, if you can even watch something you can capture it. Maybe YouTube can disable viewing videos entirely.
 
  • Like
  • Agree
Reactions: The Mass Shooter Ron Soye and ⠠⠠⠅⠑⠋⠋⠁⠇⠎ ⠠⠠⠊⠎ ⠠⠠⠁ ⠠⠠⠋⠁⠛
Prokhor Zakharov said:
I wonder if YT-DLP will be able to get around the interference that server side ad injection is apparently causing for the downloading of videos?

If not, a core archival tool will have been lost.
Click to expand...
Server side ad injection is not by any means an insolvable problem.

If you do one download pass, you can download a video plus ads that you could manually skip or remove to give a full video.

If you do two or three download passes with yt-dlp, a tool could then analyze the files it generated to produce a clean file:
  1. if the ads are exactly the same length and at exactly the same places, you can download two or three videos which can then be trivially automatically analyzed by another tool to find the differences, and those differences automatically cut out
  2. if the ads are a slightly different length but start in the same places, it's slightly harder
  3. if the ads are randomly distributed through the video, then there'd need to be a tool to checksum segments in the video and automatically analyze what bits are held in common
The only way it becomes a difficult problem would be if YT started deliberately corrupting or reencoding video for every single user downloading a video. Which I suspect is a LOT more expensive than just accepting that video downloaders will still work.

Various other bits of Youtube metadata- like subtitles, which presumably have to be automatically shifted if the server side video is being changed- could be used to aid in this.

Indeed, assuming that Youtube hasn't completely broken subtitles, they could be used by adblock tools to automatically skip the vast majority of server inserted ads even in your web browser. So your adblocker downloads the 'expected subtitle' with the text and the time offsets from a Sponsorblock server. If there's a thirty second gap in the expected subtitles with different text, then a second or so into the 'unexpected text', your adblocker skips to a second before the 'expected text' starts again. Ad blocked.
 
  • Informative
  • Agree
  • Like
Reactions: Edict of Expulsion, The Mass Shooter Ron Soye, i440BX and 1 other person
Regarding archival of Youtube Channels, I have certain questions:
1. What is the best tool to automate youtube channel downloads? It gets old using yt-dlp manually.
Also it would be cool to be able to add already downloaded videos in such a tool.
2. I realized I should have downloaded description of the videos and chapters. Is it possible to embed them in an already downloaded videos or download to a separate file (JSON, etc.)?
 
  • Thunk-Provoking
Reactions: Fag Alert Major Fag Alert
Is there a way to archive a page with a region lock on the rendered content? Sometimes archive.today just renders a splash screen with: "this content is only available in X country", rather than the actual webpage.
 
QE65 said:
Is there a way to archive a page with a region lock on the rendered content? Sometimes archive.today just renders a splash screen with: "this content is only available in X country", rather than the actual webpage.
Click to expand...
If I recall, archive.today uses a random server for most sites so you might just have to keep retrying
 
  • Like
Reactions: jenna raiding, Dunkin DoNUT and QE65
Musashi's Son said:
Use archive.is or ghostarchive.org with nitter.poast.org when archiving Twitter. Most of the time it works. In case it doesn't work change the end of the URL from "#m" to "?a" or other single letters.
Click to expand...
Are there any other workarounds to archiving it? I tried it with this method, but sadly, when you go to the bottom, the strips end with an error. I wonder if it could be my browser choice..

scif.png
 
Last edited:
i440BX said:
2. I realized I should have downloaded description of the videos and chapters. Is it possible to embed them in an already downloaded videos or download to a separate file (JSON, etc.)?
Click to expand...
(I'm using youtube-dl-gui for this).
You can download the separate JSON file by doing this:
howto1.png
extra.png
 
  • Informative
  • Like
Reactions: Caution Trapdick, i440BX, The Mass Shooter Ron Soye and 1 other person
Ghostarchive seems to be down right now. Given that the CrowdStrike incident happened today I'm willing to bet it's related to that.

Edit: Thread here.
 
Last edited:
  • Informative
Reactions: eval()
The Mass Shooter Ron Soye said:
I just tried that and liked it, but the file size estimates seem off. Downloaded this at 360p, it says total queried size is 77 MB, in reality it's about 93.43 MiB / 97.97 MB (I thought the KF limit was 100 MB, apparently it's 200 MiB).
Click to expand...
Yeah, I noticed the file estimation is pretty bad on it. I think it might be because of the variable bitrate re-encoding, as I've seen the same issue pop up in other video downloaders as well.
 
  • Thunk-Provoking
  • Like
Reactions: The Mass Shooter Ron Soye and Markass the Worst
You must log in or register to reply here.
Back