Lightweight Anonymous Distributed Shitposting - Distributed forum possiblities sperg zone

[TurdFondler]

Instead of shitting up the No Stupid Questions thread any further, let's discuss the L.A.D.S system here.

Secure scuttlebutt is interesting, but has the usual limitations of blockhain along with not actually being a forums format.

For a distributed message board architecture I'm thinking a fluid peer to peer network with nodes going on and off throughout the day, as users log on. The client would determine your nearest peers and ask for updates. The clients would be synchronized on UTC.

For now I'm imagining each post as its own distinct unit but it doesn't have to be implemented that way. So, for example:

Your client would ask the network for newest 10 posts. The other clients would check the timestamp of the request and flag the appropriate post(s). Your nearest peer would send the checksum, which would be verified against other nodes. If the checksum matches something like 80% of the network, the data is transferred.


I'm not sure what the backbone of this would be but there are a number of off the shelf protocols that might work. Even having an IRC server or possibly usenet to shuttle the data back and forth between clients is feasible, ham fisted but feasible I think.

 

[George Orson Welles]

IRC would probably be the cleanest.

 

[Kosher Dill]

How about starting with a spec? I'd say a forum needs:

- Identities. Each user is distinguishable (but not doxable) and can't be impersonated.
- Membership. Users can be assigned permission to read and/or post in various places. Permissions are assigned top-down - this is not a direct democracy.
- Posts. The basic unit of communication is a single post, which has a unique identifier so you link back to it, quote it, and so on. A post should be attributable to the user that created it, and ideally it should have an audit trail of who edited it after creation.
- Threads. The way you read the forum is linear, a thread consists of a list of posts. Not a tree like in those old Web 1.0 bulletin boards, or an undifferentiated firehose like Twitter. Threads should be identifiable and listable.
- Moderation. It should be possible in general for properly permissioned users to edit or delete content (such as illegal content) and remove it from the network. Problem/malicious users should be bannable.
- Integrity, The forum data needs to be tamperproof without being immutable. A distributed user should be able to verify that they have a "good" copy of the data.

The forum should be able to establish some sort of global consensus about:
- What content is part of the forum (no unauthorized additions from outsiders, no deleted data still hanging around)
- What users have what permissions (I can't just publish my own "permissions file" and do a 51% attack)
- Whether content is up-to-date (what happens if I try to reply to a deleted or edited post while my copy of the DB is out-of-date?)

I'm sure people could add more to it, but that's a start. I think much of this would be tricky without any centralized database and server, but it's fun to bat some ideas around.

 

[Kosher Dill]

Double-posting, but whatever. Here's an idea: what if a forum was implemented as an Ethereum distributed application?
Yeah, I'm going full blockchain retard here.
We use some sort of P2P system (IPFS, or whatever) as the backing store. All posts and metadata are nodes in this store.
There would be a "master node" that contains pointers to the current permissions list, threads, and so on. Accounts would be Ethereum addresses.

The Ethereum smart contract running this thing will store a pointer to the current master node in the store and a pointer to a queue of incoming actions.

The smart contract would have a few APIs:
Post: Adds a new action to the incoming actions queue. (which is to say, updates the queue pointer to a new node containing your new post as well as all the previous ones in the queue) Actions are anything that changes the state of the forum, such as posting, creating a thread, or using moderator tools.
Integrate: Run by the server operator. This clears the input queue and updates the master node with a new pointer, which is assumed to represent a new, valid master node reflecting the state of the forum after all these input actions have been done.

In terms of software, you'd have:
- The "add to queue" scripts for various actions
- The "integrate" program which does the real work of maintaining the forum's state and publishes the corresponding new nodes.
- Some sort of viewer to actually browse the forum and interact with it.

You'd have to have a server operator calling the "integrate" API whenever there are pending actions, but they could now do this from anywhere as long as they have access to Ethereum, the data store, and the software.

There are a number of issues and difficulties with this:
- How do we handle concurrency? We have to make sure that these queues behave reasonably when multiple people are trying to update them simultaneously
- Ethereum contracts can't read off-chain storage. How can we validate that the nodes being saved onto the contract are really following "forum logic" correctly? The server operator's "integration" program will be complex. What happens if a rogue user just uploads an empty input queue, or a rogue server operator uploads a bad master node?
- How do we stop randoms from spamming the input queue if Ethereum can't weed them out with the permissions list?
- There's no way to do anything privately in this system, it's all out in the open.
- Is there a way to let anyone run integration if they're using an approved, signed binary to do it? Or do we need to lock this behind permissions?
- Even the tiniest operation on the main Ethereum blockchain costs about USD $0.20 worth of ETH, which would be prohibitively expensive at scale. You could run it on a less popular ERC-20 blockchain or start your own, but those have disadvantages of their own.
- I have zero idea what the performance would look like.

EDIT for even more autistic thoughts:
The "queue" doesn't have to be a document. As long as you're updating the blockchain every time you update the queue, just have a "new item" pointer instead of a queue pointer. Then have the integration program scan the blockchain from the last integration point (which you'd also save each time) up to the present and process all the items linked in any revision since then. That would solve some of the concurrency issues, I think.

Feel free to tear this apart.

 

[robobobo]

I think a forum, with its large amount of static content, wouldn't be a good fit for a decentralized concept. The amount of storage KF is presently using must be significant. The closest that could be feasibly achieved in my opinion is something like a Matrix/Element server that permits threaded conversations in a room. Encrypt everything at rest, spread the data over as many nodes as possible, then none of the node owners can be identified as the person holding the offending piece of data if someone sends lawyers sniffing.

 

[Kosher Dill]

I think a forum, with its large amount of static content, wouldn't be a good fit for a decentralized concept. The amount of storage KF is presently using must be significant.

As I write this, the current total posts on KF is 7,583,872. So just the text wouldn't be an outrageous amount in this internet age, even if we assume they're a few KB apiece. Images and attachments, I have no clue, but I think that's going to be an issue with any decentralized system.

 

[twerjlh]

Regarding permissions and banning, you could leave it up to individual thread owners (if you make it, you obviously own it)

Regarding storage, even my phone has 128GB and I'm iShilling, my burner droid has a 512GB SD sitting around I'm pretty sure

 

[cecograph]

If it's decentralised, why are we worried about storage and mutability? Are we imagining that every peer downloads the entire forum? I'd say peers should only download the posts they ask for, and we go with something like a DHT. Generous forum donators could be replaced by people willing to crawl the entire network and donate their bandwidth to seeding all of it.

Regarding permissions and banning, you could leave it up to individual thread owners (if you make it, you obviously own it)

I'd much prefer a model like this. I'd rather the basic network contains absolutely everything, illegal content included, and then curation and filters could be overlaid in a composable way. You'd want everyone to naturally use the "no seriously illegal shit" overlay, in the hope that this leads to such content dropping off the network forever because no peers ask for it or distribute it. A slightly weaker standard overlay might be "no blatant copyright violations; I don't want letters from Warner Bros."

A thread OP might have a standard overlay, deleting posts that they deem fuck up their thread. Maybe it'd be automatic that clients would subscribe to this by default. Everyone might have a personal overlay for stuff, and maybe they'd be willing to have their tastes combined with readers of that thread that anyone could opt in to use.

Basically, I want all moderation and content management to be opt-in and decentralised. If the content is decentralised, why isn't the moderation? Again, we shouldn't have peers downloading the whole forum, so we're not worried about the whole thing getting taken down because someone's posted a rip of Tenet in a reply to me.

Regarding storage, even my phone has 128GB and I'm iShilling, my burner droid has a 512GB SD sitting around I'm pretty sure

With a decent decentralised filesystem backing it, we don't need to upload images multiple times, worrying that hosts are going to go down. We just aggressively hyperlink.

 

[Account]

To those interested, E-hentai actually has a peer to peer distribution system that you could look at for inspiration.

The most basic thing I could think of is a central clearing house server that issues bans (in terms of, say, GPG public keys of the users) and deletion of posts and content (in terms of name and hash) along with maintaining account data (which should be a relatively small thing to store). The clearing house can then sign and publish cleared posts (without having the entire post database saved) to peers it knows, who can then spread it around. You might be able to use something like IPFS to enable people to seek out content not available from their immediate peers.

 

[twerjlh]

If it's decentralised, why are we worried about storage and mutability? Are we imagining that every peer downloads the entire forum? I'd say peers should only download the posts they ask for, and we go with something like a DHT. Generous forum donators could be replaced by people willing to crawl the entire network and donate their bandwidth to seeding all of it.

I have a colocation I'd be willing to dedicate to archival, especially if encrypted. I already use it to seed shit people ask me to.

 

[TurdFondler]

I agree with the point about decentralized moderation. I'm imagining this to be a lot more like a chan, where every post is anonymous. Why you'd ever want persistant user data on a forum you're making to specifically avoid lawsuits is beyond me.

I know it's a big leap, but let's assume the very core of this system can be trusted. We start off with a clean master index list.

You only have to worry about the integrity of the newest post.

As for the mechanics of it I think blockhain and torrents are slow. There's some utility to them, perhaps as a means of distributing daily backups, but for more real time posting I'm thinking encrypted irc?

Yeah it's a little convuluted a dumb but it would work like this:
Distribute (randomized?) sever address through torrent side of system or via trusted p2p connections, distribute keys via similar manner.

Clients join irc server, clients scrape irc data. Client decrypts and processes IRC data. Posts are essentially added and read in real time. Set a time limit for each server instance and a lifespan for each session key, similar I guess to radio way frequency hopping. Again, maybe use UTC to synchronize clients so this could happen hands off. Changing servers and keys ensures no one parks a malicious scraper in the server and takes everything.

You could even use this kind of system as the backbone. Using UTC sync you could sort out concurrency. Once in the server the clients could ping each other, figure out the current backlogs of each client and then update the master log, assuming each post is signed with a time.

The irc server would be essentially a pass-through, not keeping any logs. The reason you'd want to do it this way instead of a p2p connection is to prevent any client from knowing any other client's IP.

 

[Kosher Dill]

If it's decentralised, why are we worried about storage and mutability?

Mainly because a lot of the solutions currently out there are lazy and assume immutability because it's easier.

Are we imagining that every peer downloads the entire forum? I'd say peers should only download the posts they ask for, and we go with something like a DHT. Generous forum donators could be replaced by people willing to crawl the entire network and donate their bandwidth to seeding all of it.

This is what I was thinking too. You wouldn't download any more of the forum than you currently do, unless you want to. "Full mirrors" will still be important however. To read a thread you just take the master node, then the thread node, then all the post nodes it links to.
(Side thought: under my scheme you couldn't bookmark a thread...)

Basically, I want all moderation and content management to be opt-in and decentralised. If the content is decentralised, why isn't the moderation?

Who can moderate where is something we can choose. But I think you do need some central administration for things like who's allowed to create threads, unless you want the "main page" spammed to hell. If there are any "sysop" roles, those need to be centrally managed too.
User accounts and bans should probably be central too, every single thread shouldn't have to individually ban the same griefer with a bot. It might be a good idea to have a global "killfile" too, to encourage clients to stop mirroring bad deleted content.

I'm imagining this to be a lot more like a chan, where every post is anonymous.

But chans aren't good and never have been good. That's why we're here.

 

[Vecr]

I had an idea for moderation, where you could subscribe to moderators you like, and each moderator would have an unmoderated chat where people could talk about if they are good or not.

 

[TurdFondler]

[words]

I agree with basically everything here. What would the client be written in and which protocols? Is this still the ethereum or ethereum adjacent idea?

The subscribing to mods is an interesting idea, reminds me of Twitter's API where you can get every post the second it's uploaded but you can filter for just the ones you want.

 

[HumanHive]

Regarding permissions and banning, you could leave it up to individual thread owners (if you make it, you obviously own it)

So 99% of threads are either unmoderated or personal hugboxes?

 

[twerjlh]

So 99% of threads are either unmoderated or personal hugboxes?

heavily implying 99% of content on the internet isn't already. At least if you control your own thread you own the consequences of letting it be a shithole or your own personal echo chamber

 

[HumanHive]

heavily implying 99% of content on the internet isn't already. At least if you control your own thread you own the consequences of letting it be a shithole or your own personal echo chamber

Just because most things are shit doesn’t mean new thing should be shit as well. For all of 4chan’s faults, it is pretty good about preventing hugboxes and “Google pays...” scams.

 

[AmpleApricots]

All not strictly moderated (strictly as in, clear outlines what is allowed and what is not) "free speech" zones lately just degenerate into political extremist/conspiracy posting, child porn and other degenerate crap inside of 5 seconds. 2 seconds if you post anonymously (not attached to a nickname) like on a imageboard. You do NOT want light moderation in this eternal September web.

Also RE: distributed P2P storage. Somebody uploads child porn to these distributed forums and then I end up with child porn on my harddrive? No thanks. Before you start talking about encryption, consider that for many countries that doesn't protect you from legal prosecution. Also yes, the law also can make a difference if you actively download stuff to your harddrive or have it "appear" in your browser, although I am aware that technically it's the same. Courts often do not understand this stuff well or tend to interpret it in a way that doesnt make them feel they wasted their time and I would never risk it.

 

[Kosher Dill]

Also RE: distributed P2P storage. Somebody uploads child porn to these distributed forums and then I end up with child porn on my harddrive? No thanks.

This is what I can't figure out any good solution to.
I mean, someone could link in a CP image right now in this thread, and my browser would load it and I'd end up with the image in my cache on disk and so on, but at least I wouldn't be redistributing it in that case.
I wonder how common it is for people to try "poisoning" this place. How are we stopping people from hiding behind Tor and posting CP right now?

What would the client be written in and which protocols? Is this still the ethereum or ethereum adjacent idea?

I think it depends on exactly what underlying technologies would be used. My example wouldn't literally work as written because it'd apparently cost millions to run it on Ethereum itself. Point being, my idea would require some sort of distributed consensus system (whether that's a blockchain or something else), plus distributed storage.
We could use the BAT blockchain instead of ETH

 

[Vecr]

This is what I can't figure out any good solution to.
I mean, someone could link in a CP image right now in this thread, and my browser would load it and I'd end up with the image in my cache on disk and so on, but at least I wouldn't be redistributing it in that case.
I wonder how common it is for people to try "poisoning" this place. How are we stopping people from hiding behind Tor and posting CP right now?

Could a coordinating server distribute encryption keys so each computer does not know what's stored on their drives unless they try to access it through the server, like everyone else? Someone might be able to figure out a similar, distributed setup.