Programming thread

Is the demand for Java really as dead as it looks on LinkedIn? I’ve been looking around for backend and data engineering jobs in the US and Europe and nearly everything requires Python or Go, even when I specifically search for Java.

I’m perfectly fine with continuing to work in Python, but the idea of having to learn Go frightens and disgusts me.
Been a minute since I've looked for a job, and been even longer since I have been paid to write code in Java, but you might want to plug Spring in your search terms too. If memory serves, that was their big framework. From what I've seen Golang is almost exclusively used by devops and SRE types. I know the pay ceiling for SRE can be ludicrous, but the on call/fixing other people's broken shit nature of it made me shy away from that path.
 
I was stuck the entire day at work needing to debug crashes with an nvidia c++ api. The code apperantly only worked until recently due to sheer fuckin luck preventing it reaching any sensitive unallocated/illegal memory area and causing segfaults. My best guess is that there is something I deallocated and it crashed everything. The specific line that constantly crashes is some bizarre function dictionary into GPU functions, so I can't even go inside and see what's the problem.

Nvidia is my most hated company for how non-existant their samples are and how their documentations is shit. Most of the times I use their functions it's basically trial and error until they work and then I discover it's better to just write my own code that is faster and won't crash randomly if the input is arbitrarily not the exact size they allow.

To give an example:
 
  • Feels
Reactions: PedoSec
Been a minute since I've looked for a job, and been even longer since I have been paid to write code in Java, but you might want to plug Spring in your search terms too. If memory serves, that was their big framework. From what I've seen Golang is almost exclusively used by devops and SRE types. I know the pay ceiling for SRE can be ludicrous, but the on call/fixing other people's broken shit nature of it made me shy away from that path.
Spring, Android, and Quarkus are the big 3 'frameworks' for Java. I fully expect Quarkus to start dominating over the next decade because it's absolutely insane.

Is the demand for Java really as dead as it looks on LinkedIn? I’ve been looking around for backend and data engineering jobs in the US and Europe and nearly everything requires Python or Go, even when I specifically search for Java.

I’m perfectly fine with continuing to work in Python, but the idea of having to learn Go frightens and disgusts me.
There's a ton of companies looking for Java programmers. Mostly because you have a ton of shit that was written in the 2010s in Python/Ruby/Javascript that now is starting to show its age and becoming difficult to scale-up. IIRC Netflix recently decided to rewrite a ton of their backend web services in Java because of this and I expect there's more than just them in the 'big tech' sphere doing that (or opting for C#). If you only know basic Java from studying it in a CS program, I'd recommend learning Spring and building a few projects because it's a little bit different than just writing a Swing app for your CS3 final.

It's a bit of weird time atm though - the biggest users of Java are normie American companies that do things like fast food or trucking or literal bean counting (agribusiness is big). They don't aggressively post jobs and instead tend to go through recruitment agencies so your best bet is just reaching out to a recruitment agency that specializes in getting foreigners jobs. BUT - recently they've wound down a lot of that because the big tech layoffs have flooded the market enough that they're snapping those guys up. If we see a rate cut in the latter half the year, I'd expect all the laid off FAGMAN guys to be scooped up and you'd have better luck getting in.

EDIT: Also if you want to get into the Java ecosystem but hate the Java language, I'd recommend learning kotlin, scala, groovy, and/or clojure. You can write your shit in whatever syntax you like but you still get to seemlessly interact with all the Java shit you're going to need to actually get a job.
 
Last edited:
EDIT: Also if you want to get into the Java ecosystem but hate the Java language, I'd recommend learning kotlin, scala, groovy, and/or clojure. You can write your shit in whatever syntax you like but you still get to seemlessly interact with all the Java shit you're going to need to actually get a job.
Scala also opens up some data science or machine learning opportunities, if you do that or have any interest in that. Haven't touched it myself but I've met people who use Scala for DS and ML.
 
I need home solutions for quickly querying 100GB+ text data. I downloaded a copy of all Reddit comment history up to 2015 from pushshift.io, which used to have a pretty good search. They have since cucked out and you can't search by username and they've also deleted a bunch of people. So I want to recreate what they had for myself with the historic data.

I downloaded MySQL for starters, but after reading further this might not be the best solution? I know Python and some basic SQL queries. Am I going about this right way? Should I use something like MariaDB? What are some good resources for learning how to do what I want to do or do I need to start at "building a database 101" or something? I'm willing to spend a good amount of time on this.
 
  • Feels
Reactions: Somchai
I need home solutions for quickly querying 100GB+ text data
My go-to for this kind of stuff is Datasette & the related SQLite-utils. It will have you up and running with full-text search and a nice web interface fairly quickly, though honestly I don’t know what the performance will be like with that amount of data (find out and let me know!).
 
I need home solutions for quickly querying 100GB+ text data. I downloaded a copy of all Reddit comment history up to 2015 from pushshift.io, which used to have a pretty good search. They have since cucked out and you can't search by username and they've also deleted a bunch of people. So I want to recreate what they had for myself with the historic data.

I downloaded MySQL for starters, but after reading further this might not be the best solution? I know Python and some basic SQL queries. Am I going about this right way? Should I use something like MariaDB? What are some good resources for learning how to do what I want to do or do I need to start at "building a database 101" or something? I'm willing to spend a good amount of time on this.
Seems I use "lunr" for my text search. Which is an offshoot of "Solr" ( https://solr.apache.org ) which is probably closer to your needs.
 
  • Like
Reactions: melty
My go-to for this kind of stuff is Datasette & the related SQLite-utils. It will have you up and running with full-text search and a nice web interface fairly quickly, though honestly I don’t know what the performance will be like with that amount of data (find out and let me know!).
Seems I use "lunr" for my text search. Which is an offshoot of "Solr" ( https://solr.apache.org ) which is probably closer to your needs.
Thanks! I'll check these out.
I did write a simple search in Python and even tried a multi threaded approach and some other optimizations and it takes like 30 minutes
 
Thanks! I'll check these out.
I did write a simple search in Python and even tried a multi threaded approach and some other optimizations and it takes like 30 minutes
Probably a drop in solution would be best, but if you want to take a stab at a novel homemade solution you could try a matrix profile. I only suggest it because it's been on my to-do to fool around with one for too long.

Long story short, a matrix profile can be queried and spit out a list of indexes ordered by similarity (or dissimilarity). Search time is more reliant on the width of the query than the size of the dataset. Technically you can pause calculation anytime and get a "good enough" approximate answer out, but I don't think that's made it to a public library.
Originally they were made for searching massive time series's, one of the papers about optimizing it is simply called "trillions.pdf". If you're clever with the conversion from char to float there's no reason you couldn't use it to search for a byte string.

For academic interest
For something usable
 
From what I've seen Golang is almost exclusively used by devops and SRE types. I know the pay ceiling for SRE can be ludicrous, but the on call/fixing other people's broken shit nature of it made me shy away from that path.
Nawh Go isn't just devops or SRE, it's also used for heavy use API frameworks. Half my job right now is converting our PHP code into Go to leverage Go's concurrency and stop us exceeding the time limit on certain cloud jobs. It's a decent language for such web applications and should be something one is at least aware of if looking for a job in that field.
 
I need home solutions for quickly querying 100GB+ text data. I downloaded a copy of all Reddit comment history up to 2015 from pushshift.io, which used to have a pretty good search. They have since cucked out and you can't search by username and they've also deleted a bunch of people. So I want to recreate what they had for myself with the historic data.

I downloaded MySQL for starters, but after reading further this might not be the best solution? I know Python and some basic SQL queries. Am I going about this right way? Should I use something like MariaDB? What are some good resources for learning how to do what I want to do or do I need to start at "building a database 101" or something? I'm willing to spend a good amount of time on this.
Elasticsearch
 
Is the demand for Java really as dead as it looks on LinkedIn? I’ve been looking around for backend and data engineering jobs in the US and Europe and nearly everything requires Python or Go, even when I specifically search for Java.

I’m perfectly fine with continuing to work in Python, but the idea of having to learn Go frightens and disgusts me.
Java was popular in orange county when I worked there. You might also look to branch off into .net
 
Nawh Go isn't just devops or SRE, it's also used for heavy use API frameworks. Half my job right now is converting our PHP code into Go to leverage Go's concurrency and stop us exceeding the time limit on certain cloud jobs. It's a decent language for such web applications and should be something one is at least aware of if looking for a job in that field.
That's interesting. I haven't heard of Go replacing PHP until you brought it up. I've played around with Golang for toy projects but mostly when I've seen it brought up its in comparison to languages like Rust and C++ as well as Python. I don't think Go is necessarily a good comparison to the first two languages but you can draw some reasonable comparisons to Python. I've had the displeasure of writing some PHP in the past and it was God-awful. Now that you've pointed out the connection between the two I think Go could replace a lot of the crappy backend code on the internet.
 
Nawh Go isn't just devops or SRE, it's also used for heavy use API frameworks. Half my job right now is converting our PHP code into Go to leverage Go's concurrency and stop us exceeding the time limit on certain cloud jobs. It's a decent language for such web applications and should be something one is at least aware of if looking for a job in that field.
How’s your experience with Golang so far? I’ve tried a few things in it but the error handling kind of skeeves me out.
 
  • Like
Reactions: Aidan
How’s your experience with Golang so far? I’ve tried a few things in it but the error handling kind of skeeves me out.
I'm not the person you are asking, but I spent this past December writing a tiny project (~1000 deployed loc), and my unwarranted opinion is that Go is a decent language to teach people the basics of software engineering: writing software, using source control, versioning, testing, coding conventions, simple debugging, deployments, and so forth. It has a rudimentary syntax and a basic type system, and with this it tries to be uniform with its interfaces (e.g. len() for the length of containers, for being the only looping syntax, etc.) There's not much you need to learn about any aspect of it deeply in order to get okay working code, and that is by design. That being said, there were issues I had with the language that I don't see mentioned often elsewhere: forking existing projects and the community.

Beside my tiny project, I also attempted to fork a project to make my own changes and host on another non-github repository. I cloned it locally and attempted to update the go.mod file using the officially recommended way: go mod edit -replace This ended up with an error about module declares path as "x" but was required as "y" (stack overflow thread: https://archive.is/jgaE6), and after trying everything in that thread, I still couldn't resolve the issue. (I of course looked elsewhere, but pretty much every answer I found was the same as that thread). In the end I got it to work by doing a find and replace of the old URL over the entire project. I ended up trying to fork a different project at a later point and that had no issue when using the replace directive, so it might have been an issue with the first project or me, but the experience left a bad taste in my mouth.

As for the community, it seems like its made up of people who don't value their time. By this I mean people pride themselves on either reinventing the wheel for everything; or if you do want to use existing software the developers think that you should debug their code to learn how to integrate said software. I encourage people to RTFM when trying to learn something, but for what I was trying to do, the projects I saw felt that autogenerated docstrings on functions were the best documentation. Overall it felt like I was dealing with the same pain points I have when dealing with proprietary software, only I wasn't being paid to do it in Golang's case.

So overall I'd recommend the language if you are learning to develop software or if it's for paid work; but if it's for a hobby project, then think deeply about what you are expecting to learn from it, and you will probably find the tools and ecosystem is better established elsewhere.
 
Last edited:
Is there actually any use in learning a functional language if you've been coding professionally in stuff like Python, Java, and C# for years? I've wanted to learn a functional type programming language for the longest time, but the troon/excessively woke shit makes me convinced it should be OCaml or Haskell. Probably OCaml because the community seems to be some computer science PhDs and a single hedge fund, and I've read some pretty convincing arguments that Haskell's lazy evaluation is a bad thing. It seems to hit a sweet spot of being difficult and obscure enough that it won't end up as a weird virtue signalling thing.
Ocaml is great stuff. Yeah, I think language-wide lazy evaluation is overkill and fucks with your mental model of what's happening behinds the scenes.

Ocaml lets you do a lot of stuff purely functionally, but you still have access to side effecting operations as an escape hatch in case you paint yourself into a corner.

It's also a very effective compiler for what's still ultimately a high level, garbage collecting language. As a personal project, I'm doing some low level, network packet processing in ocaml. I'm able to saturate the card I'm experimenting with at 10gbps.
Lisp bugs me because everything looks the same, I've messed around with Racket and Clojure a bit but I just don't see the advantages. At least the Lisp communities are mostly old Linux guys instead of socialist-identifying troons, though.
Sexprs and lisps are good because you can rewrite the language at really low levels to implement new ideas.

Like CLOS clones are the most effective oop system I've ever used. I liked it so much that I was able to implement my own using macros in scheme. CLOS makes C++ and Java's oop systems look like shit you got out of a cereal box.

Macros are definitely a powerful tool and people can use it to write ugly, inscrutable code. But that's the perpetual debate about more programmer control vs less programmer control.
 
How’s your experience with Golang so far? I’ve tried a few things in it but the error handling kind of skeeves me out.
Oh yeah the error handling is tedious as hell at times (the lack of try catch threw me when first learning it), but once you get your head around it being glorified variable checks it can yield some nicely streamlined code. I'm an OOP fanboy, but there's a lot of nice things under the hood with Go, especially if you have any familiarity with C and an interest in web design. Really easy to get a custom API framework off the ground with it for example.
 
Oh yeah the error handling is tedious as hell at times (the lack of try catch threw me when first learning it), but once you get your head around it being glorified variable checks it can yield some nicely streamlined code. I'm an OOP fanboy, but there's a lot of nice things under the hood with Go, especially if you have any familiarity with C and an interest in web design. Really easy to get a custom API framework off the ground with it for example.
I really think that try/catch is such an ugly wart on modern programming languages.

I know some languages tried to curb it a bit by having all methods declare what exceptions they throw, but that was really just a bandaid on what was ultimately a broken concept.

Having an exception tear through like several layers of libraries that you didn't write and then through your user code and you have no clue why is one of the most annoying fucking things.

Having error aware return types is something popular in the statically typed functional languages (ocaml, haskell, erlang, I think) and I'm very pleased it's making its way into the mainstream like in Go.

It's a little frustrating at first that you have to account for errors at literally every function call, but ultimately, it saves you a bunch of headache later. It is much nicer to be forced by the language design to consider every function you call in terms of "how can this shit the bed and how do I need to address it at the call site".

Golang does this with two element tuples (value, err).

In Ocaml does this with a special wrapper type called Result that has either an Ok returnval form or an Error err form. If you want to throw all caution to the wind and force a Result to return its Ok value (if you're confident that the code probably won't error), there's a function in the Result module that will do that and have the whole process panic if you're wrong.

It's a very good thing that it forces you to think about those things later. You aren't surprised if you do later get these catastrophic panics. You typed that code in to risk that possibility.
 
I really think that try/catch is such an ugly wart on modern programming languages.

I know some languages tried to curb it a bit by having all methods declare what exceptions they throw, but that was really just a bandaid on what was ultimately a broken concept.

Having an exception tear through like several layers of libraries that you didn't write and then through your user code and you have no clue why is one of the most annoying fucking things.

Having error aware return types is something popular in the statically typed functional languages (ocaml, haskell, erlang, I think) and I'm very pleased it's making its way into the mainstream like in Go.
Funnily enough this is one of the things I don't miss with try catch. For example the PHP generic Exception class works fine and dandy (mostly) for error handling, but if for some reason you go with the Throwable superclass you can pick up false positives, miss logs, and otherwise have a miserable debugging experience. A fellow dev thought he'd be imaginative and exclusively use Throwable without knowing this, leading us down multiple rabbit holes thinking sensitive code was seriously broken until we picked up on his error handling strategy. Gave me a much better appreciation for more rigid error handling lol.
 
  • Feels
Reactions: Marvin
Go is a decent language to teach people the basics of software engineering: writing software, using source control, versioning, testing, coding conventions, simple debugging, deployments, and so forth.
A better "first language" than Python, by a long shot. I'd also argue TypeScript being a good first language because many jobs require TS and you're forced to use types, which should be standard.
Ocaml is great stuff.
Especially if you're trying to land that sweet $300K/year job at Jane Street.
 
Back