need software recommendation to download entire twitter / mastodon

  • 🐕 I am attempting to get the site runnning as fast as possible. If you are experiencing slow page load times, please report it.

revenge of rebecca

kiwifarms.net
Joined
Feb 14, 2024
Before elon bought twitter, I irregularly used an application called TumblThree (https://github.com/TumblThreeApp/TumblThree) to backup twitter profiles to my computer. I have not used it in a long time, but it was a useful tool, allowing me to save hector marcan's entire twitter (see https://kiwifarms.net/threads/hecto...s-asahi-lina-asahi-linux.169453/post-17773666). At some point after Elon took over, the tool stopped working. I do not know if they fixed it / if it works now, but you probably need an account in order to use it to backup entire twitter profiles now, unfortunately.

But since the tool proved somewhat useful for kiwifarms, I am looking for new tools with similar capabilities. This time, I am also interested in downloading for mastodon, not just twitter. A site like archive.ph, while useful, is only really good at archiving specfic tweets at specific times. I am looking to download the entire contents of someone's mastodon or twitter through the application, including images, videos, raw text for easy search purposes, maybe .jsons, etc. I am also looking for a tool that I can proxy through the tor network for security/privacy/anonymity. In theory I could run a tool through Tails/Whonix, but even if TumblThree works now, it is a Windows-only application, and Tails is based on Linux. So I would need a different tool. Besides, it is hard to scrape twitter without an account now, and TumblThree does not allow me to archive mastodon.

Does anyone have any suggestions on what FOSS tools I can use to accomplish this goal?
 
Back