Programming thread

[Kosher Salt]

Why what? "Why do I not bother too much with redundant ephemeral documentation for my traditionally write-only code for projects that management tends to nix before they ever get traction?" Or "Why do I hope to be out of the software dev game before pointless make-work like proper, disciplined documentation and testing becomes the actual (and not just ostensible) industry standard?"

In either case, those questions kind of answer themselves, no?

Why are you killing trees when there are perfectly good archival media formats that are machine-readable?

 

[Knight of the Rope]

Why are you killing trees when there are perfectly good archival media formats that are machine-readable?

You're right. I suppose it's about time I caught up with the 21st Century and started using some more modern media formats. I can always murder trees later on my own time.

 

[Anstiv]

You're right. I suppose it's about time I caught up with the 21st Century and started using some more modern media formats. I can always murder trees later on my own time.

Maybe get an iPad/tablet if your worflow really incorporates hand written notes that heavily? I use one for sheet music and it's really great actually. Im sure you could find one for pretty cheap if it's 2-3 years old.

 

[stares at error messages]

No. It fails because of its absurdly unreadable syntax (which all of its successors continue to double down on), and fancy yet hardly useful in real world programming idea of S-expressions (i.e. code is data). Yeah, it is very powerful concept, but like for backend implementations, such as compilation & optimization, but not for human operations.

Lisp is as much machine code as lambda calculus is polish notation. If memory serves, at one point Sun Corporation made JVM processors, a processor that has virtual machine instructions in silicon, but stopped when they realised it was a retarded advertising gimmick no one cared about. So, it really doesn't matter if it's in silicon or not to make it a opt-code style computer input; autocode or X86 assembler.

Tabs, or spaces? Go!

I hit the tab key and the indentation is logically corrected from the retard who does not know the joys of using a mac-lisp interpreter to edit software.

Why the fuck is everyone not jumping to replace javascript with typescript. This seems like the most braindead simple choice to me.

You like strongly typed languages, other people learn to love the bomb. Typescript: So it's another thing I have to compile. So it's another thing I have to test. So it's another layer of abstraction. So it's another REPL I have to have installed if I get stuck on something. So it's even more removed from the only place I use these shit languages; the browser. So it's just not that practical.

It might see strange I'm taking this position on Typescript after saying C++ isn't strongly typed enough. I think a language should have meaningful types if it has types, but if it doesn't then getting by is very possible. In Python3 without strong timing, working in your own code base, while very uncollaborative, just don't miss use things if you don't expect an exception. Also exceptions endup as a form of control flow, which they suck at; monad are better.

Why are you killing trees when there are perfectly good archival media formats that are machine-readable?

Those trees had it coming. Some of their friends decided to fall on my house, so I decided to send a message and cut down their whole family. Who's laughing now, you sheet of A4!

 

[Kosher Salt]

Those trees had it coming. Some of their friends decided to fall on my house, so I decided to send a message and cut down their whole family. Who's laughing now, you sheet of A4!

It's not really that I care for the trees, it's more that if you're ever in the unfortunate position of needing to recover the data from your paper records you're basically going to have to re-type it.

 

[Gorilla Tessellator]

So I was a witness to a conversation where single threaded server was being compared with a multithreaded server. The participants discussed pros and cons.

Here are the pros that were listed for a single threaded server:

• its performance is better at the cost of resources as you spawn more processes
• creating threads and context switching is expensive
• it's easier for programmers as they don't need to deal with locks and other multithreading difficulties

I have to admit I was a bit puzzled by the third statement. Handling requests with multithreaded server is a problem that was solved in many commonly used frameworks, and it's not like there is a need to reinvent the wheel. There are already robust implementations of thread pools that do not share resources, so race conditions are avoided. Of course there is no advantage of using many thread if you don't have many cores available, but it doesn't seem to be the scenario that was being discussed.

Any thoughts?

 

[Least Concern]

Any thoughts?

"Server" is an overloaded term in computing. How was it used in terms of this discussion? Web, file, database…?

Also, what level are we at? Actually programming software for it, or just setting one up using pre-existing software?

 

[glow]

• it's easier for programmers as they don't need to deal with locks and other multithreading difficulties

I guess by not having multithreading in the first place you don't give a lower skilled developer the rope they need to hang themselves?

You didn't specify which tech you're talking about but usually it's web servers. Presumably in this environment IO operations still are completed in a background thread during await/async type actions? It's not really an issue unless you're doing CPU bound work. Ideally you'll run multiple instances of the server in a load-balanced configuration to increase your tps.

This is normally fine for most web applications which are effectively just front-ends to databases with minimal logic for validating user inputs, implementing business workflows, etc.

This is not at all fine if you need to do anything CPU bound or anything that can be accelerated by multi-threading, for example determining if a given bounding cube intersects with millions of others.

You are correct in saying that many of these cases are covered by frameworks, but a low-skilled programmer or a team without adequate QA procedures can cause a serious problem for themselves that is not apparent until the server is being gangbanged in production and the company's Twitter feed is blowing up with complaints.

So in some cases it is more trouble than it's worth but I would never not want this option because it can also save considerable rearchitecting in the event you do need multithreading to accelerate your workload.

 

[Bongocat]

So I was a witness to a conversation where single threaded server was being compared with a multithreaded server. The participants discussed pros and cons.

Here are the pros that were listed for a single threaded server:

• its performance is better at the cost of resources as you spawn more processes
• creating threads and context switching is expensive
• it's easier for programmers as they don't need to deal with locks and other multithreading difficulties

I have to admit I was a bit puzzled by the third statement. Handling requests with multithreaded server is a problem that was solved in many commonly used frameworks, and it's not like there is a need to reinvent the wheel. There are already robust implementations of thread pools that do not share resources, so race conditions are avoided. Of course there is no advantage of using many thread if you don't have many cores available, but it doesn't seem to be the scenario that was being discussed.

Any thoughts?

I'm not a web dev so I can only speak by proxy experience, but it strikes me as one of those theory vs practice things. Multithreading is one of those things that never really stops being a problem. It's more expensive than you think and to avoid race conditions you pretty much have to get lucky every time. The number of times that you get "perfectly concurrent" problems which don't need shared resources is very scarce in my experience.

Solutions like NodeJS/Event loops just seem like a "good enough" solution for most circumstances.

EDIT: To the above, it's not just a "low skill programmer" thing. I see certified seasoned geniuses screw up multithreading all the time. Working on a school project is very different than having your boss call at 3 AM with a critical bug fix that requires modifications to threaded code. It needs to be done in 30 minutes, DON'T SCREW UP!

 

[Gorilla Tessellator]

"Server" is an overloaded term in computing. How was it used in terms of this discussion? Web, file, database…?

Also, what level are we at? Actually programming software for it, or just setting one up using pre-existing software?

It was used more as a service that a client connects to with TCP.
The service can connect to database and perform some business logic. It's a stateless service.

They were talking about architectural choices that were made some time ago, and I believe in part they justify the inertia.

Edit: so we are at the level of actually creating library for the service.

I guess by not having multithreading in the first place you don't give a lower skilled developer the rope they need to hang themselves?

You didn't specify which tech you're talking about but usually it's web servers. Presumably in this environment IO operations still are completed in a background thread during await/async type actions? It's not really an issue unless you're doing CPU bound work. Ideally you'll run multiple instances of the server in a load-balanced configuration to increase your tps.

This is normally fine for most web applications which are effectively just front-ends to databases with minimal logic for validating user inputs, implementing business workflows, etc.

This is not at all fine if you need to do anything CPU bound or anything that can be accelerated by multi-threading, for example determining if a given bounding cube intersects with millions of others.

You are correct in saying that many of these cases are covered by frameworks, but a low-skilled programmer or a team without adequate QA procedures can cause a serious problem for themselves that is not apparent until the server is being gangbanged in production and the company's Twitter feed is blowing up with complaints.

So in some cases it is more trouble than it's worth but I would never not want this option because it can also save considerable rearchitecting in the event you do need multithreading to accelerate your workload.

Thanks for your input. I was wondering if this solution is the best, because the fastest server frameworks do use a thread pool for db connection and handling the requests. (drogon-core for example but also asp.net core)

In my experience, I saw a major speed up when using concurrent split download from S3. This was in Java Spring server using netty-reactor. I am not a fan of this framework, but it's just an example.

Don't get me wrong, I don't think server with a thread pool is the only correct way, but I guess there must be some reasons why people made effort to implement these frameworks this way, and the excuse of "it's too hard to code" seems lame.

 

[Least Concern]

It was used more as a service that a client connects to with TCP.
The service can connect to database and perform some business logic. It's a stateless service.

They were talking about architectural choices that were made some time ago, and I believe in part they justify the inertia.

Edit: so we are at the level of actually creating library for the service.

So this is basically how a normal web server works. Yeah, you don't really need to worry about concurrency in this case because the statelessness means that each request is basically dealing with its own little universe and you're not sharing data with other processes (except through the database but so long as you use transactions you shouldn't have any issues there). You might have issues if you're creating files on the filesystem outside of the database, in which case you'll need to do file locking which isn't much fun but not impossible. If you do expect to have connections which will last a while, I think multithreading is the more performant approach. But I'm just a dumb web dev so take that opinion for what it's worth.

 

[glow]

Thanks for your input. I was wondering if this solution is the best, because the fastest server frameworks do use a thread pool for db connection and handling the requests. (drogon-core for example but also asp.net core)

In all cases, you would still have a thread pool for IO completion. You would not have a thread pool for a DB connection pool however. The typical design is that you maintain a list of open connections and when one is returned to the pool, you perform the operations necessary to reset its state before then giving it to the next consumer that wants it. No threads are required, although some systems do have a thread to poll the connections to ensure they're still alive, that would be a single thread and not a pool of threads. It depends on the performance penalty of possibly having to discover the connection has dropped while it was in the pool and establishing a new one or grabbing another from the pool. (source: I have programmed client libraries for a number of systems like this)

Using your S3 example, it sounds like you were downloading the files one after the other. With the IO completion thread pool you would be completing your S3 download in that thread pool. You need to implement the design correctly of course - performing async/await one after another in a loop will not yield a performance improvement but most of these languages have an API for awaiting multiple task completions before continuing execution and in this situation you would get a good benefit. Of course you must control the parallelism of the execution to avoid getting rate-limited so it's not a silver bullet. Most decent languages have this built into the core library.

For example, you could have two request come in. You would process the requests one after the other but because the actual time spent blocking the event loop is very short, they would appear to run somewhat concurrently. Once the IO work is done, the event loop is notified and takes control to complete the action.

Don't get me wrong, I don't think server with a thread pool is the only correct way, but I guess there must be some reasons why people made effort to implement these frameworks this way, and the excuse of "it's too hard to code" seems lame.

I can think of a few more

• It be related to a limitation of the underlying technology. Node is event driven with background IO completion because browsers are event driven with background completion. I have heard that Node is getting/now has multithreading for CPU bound work but this took a very long time to get done.
• You won't have that level of traffic
• You can easily load balance the service because it is stateless
• You can easily run multiple identical processes to mimic multithreading without the potential pitfalls of actually having it
• The application needs to be very simple because it runs on hardware with limited power
• Your workload is mostly IO based and you will spend minimal time in the event loop
• You know for sure that you will never need to perform CPU bound work
• Your organisation already has a lot of competence developing and deploying this technology

 

[Marvin]

Back when I used to post in this thread more often (before career stuff sucked up my spare time, jesus christ I need a new job), I argued against multithreading and instead for separate processes communicating via IPC.

All my arguments from back then still hold.

Basically, multithreading is too complicated for the benefits it provides. I wouldn't characterize this as a matter of skill though. Like, to start with, multithreading isn't a difficult concept to grasp. I think most people can understand it in simple scenarios, but simple scenarios inevitably become more complicated.

And my argument is that the complexity it does introduce is a waste of your mental effort, both writing multithreaded code but especially debugging multithreaded code.

It's a better use of your limited mental resources writing the largest chunks of code that run as deterministically as possible.

My criticism was specifically about directly fiddling with threads and mutexes yourself. They sprinkle non-deterministic behavior throughout your code and it's a pain in the ass to debug. And even if you're the best programmer in the world, your limited mental resources are still better used elsewhere.

But libraries that hide this from you are fine.

Here's a post I wrote a few years ago about it. (I cringe at some stuff I wrote in the past, the specifics are dubious, but the general idea is correct)

 

[Badungus Kabungus]

Since there's no hardware hacking thread, I'll post it here. It's an acoustic camera (highlights audio sources) using a miniDSP UMA-16 microphone array. Anyone tried building anything like that? I've played around with thermopile arrays and superimposing data from them onto a video feed, this seems very much along the same lines.

DIY Acoustic Camera using UMA-16

Make your own acoustic camera with miniDSP UMA-16 and Acoular

navat.substack.com

 

[❄️❄️❄️]

So the MaterialUI project has gone from v4 to v5, renaming their NPM packages and changed some of the fundamental ways the project works (wtf even is EmotionJS). I'm trying to update, and it's all just shit.

I love my Spring Boot backend. Azure AD auth, Spring Security, Spring Data JPA and Spring HATEOAS have allowed me to build exactly what I wanted with minimum greif. Kubernetes let me deploy it easily. It's boring, and stable. I'm not worried that an update will require me to rip out major parts of my application.

Why can't web dev be as elegant? Why is everything built on hipster frameworks that border on incomprehensible (I am sure the developers are very very clever people...)? Why do these people see complexity and long dependency lists as a good thing?

I just want to write business logic and build a nice UI, not become an expert in the latest JS twattery.

 

[Least Concern]

Why can't web dev be as elegant? Why is everything built on hipster frameworks that border on incomprehensible (I am sure the developers are very very clever people...)? Why do these people see complexity and long dependency lists as a good thing?

I just want to write business logic and build a nice UI, not become an expert in the latest JS twattery.

I avoid using front-end frameworks as much as possible. I'll use them if client work on a pre-existing project means I have to; otherwise it's just plain HTML jazzed up with just the JS and CSS necessary. Even jQuery isn't really necessary anymore.

It means I can't apply for jobs that require ten years of experience in Vue and Angular, but… oh well. (Why do so many job listings for back-end devs require that shit anyway?)

 

[Shoggoth]

So I was a witness to a conversation where single threaded server was being compared with a multithreaded server. The participants discussed pros and cons.

Here are the pros that were listed for a single threaded server:

• its performance is better at the cost of resources as you spawn more processes
• creating threads and context switching is expensive
• it's easier for programmers as they don't need to deal with locks and other multithreading difficulties

I have to admit I was a bit puzzled by the third statement. Handling requests with multithreaded server is a problem that was solved in many commonly used frameworks, and it's not like there is a need to reinvent the wheel. There are already robust implementations of thread pools that do not share resources, so race conditions are avoided. Of course there is no advantage of using many thread if you don't have many cores available, but it doesn't seem to be the scenario that was being discussed.

Any thoughts?

Doing multithreading is hard if your language is ass.
If you have CSP, FRP, Functional effects, Immutability, or monads (these are 5 solutions) the problem is mostly non-existent
But you can't fix stupid.

 

[stares at error messages]

throw "Invalid; you will never be a women no matter how hard you try!";

 

[Finder]

Why can't web dev be as elegant? Why is everything built on hipster frameworks that border on incomprehensible (I am sure the developers are very very clever people...)? Why do these people see complexity and long dependency lists as a good thing?

RDD, resume driven development.

Everyone wants their own thing to put on their resume, their own unique widely used projects, something to pad their experience and give them another talking point. This is the same reason for millions of low effort libraries.

It's an arms race for who has the newest, coolest, uniquest technology on their resume, because whoever does "wins".

 

[Lurk McDurk]

So the MaterialUI project has gone from v4 to v5, renaming their NPM packages and changed some of the fundamental ways the project works (wtf even is EmotionJS). I'm trying to update, and it's all just shit.

I love my Spring Boot backend. Azure AD auth, Spring Security, Spring Data JPA and Spring HATEOAS have allowed me to build exactly what I wanted with minimum greif. Kubernetes let me deploy it easily. It's boring, and stable. I'm not worried that an update will require me to rip out major parts of my application.

Why can't web dev be as elegant? Why is everything built on hipster frameworks that border on incomprehensible (I am sure the developers are very very clever people...)? Why do these people see complexity and long dependency lists as a good thing?

I just want to write business logic and build a nice UI, not become an expert in the latest JS twattery.

Is that you John Wayne, is this me?

I've been two months now dicking around with UI frameworks despite the fact that all of the backend / database / business logic of my current project is fucking done. I use Python/Django rather than Java/Spring but the situation is exactly the fucking same. You can build all sorts of business logic in a short amount of time and then spend the next six months trying to appease the UI whims of whoever might use it.

The whole landscape of UI development now is in every way worse now than it was in 1999. The sad fact is Flash was better than anything made since. UI development should be done by an artist, not by some hipster with a bunch of shit cobbled together with webpack and babel.