Programming thread

[CrunkLord420]

The only time I've needed to learn any real math is dealing with gamedev stuff. Shaders, physics and 3D rendering.

Not that big of a deal, I've just heard that certain things that are a bit of a pain in the dick with C++ (Memory management, GUIs, etc.) aren't as much of a pain in the dick with C# and the like. Would I be correct in that assumption or have I been misinformed here?

Memory management isn't that big of a deal, people make a big deal out of it because they're used to never thinking about it. Practically the desktop GUI libraries are written for C/C++ and anything else is using a binding library on top of the native code. Lots of people like QTCreator just as a generic C++ IDE, and it has built in WYSIWYG GUI editor.

There are other reasons not to like C++, like the insane toolchain and long compile times. Both of those issues affect C# as well, though.

 

[sadrabbit]

I feel a lot of algorithmic mathematics just sort of comes inherently after you spend a long time solving specific programming problems, but I don't think there's anything particularly unhelpful about doing maths. I'm taking multivariable calc for the data science meme, and soon discrete maths for crytography, but I'm not sure if I'll ever end up using them, nevermind for normal programming jobs.

 

[Shoggoth]

I feel a lot of algorithmic mathematics just sort of comes inherently after you spend a long time solving specific programming problems, but I don't think there's anything particularly unhelpful about doing maths. I'm taking multivariable calc for the data science meme, and soon discrete maths for crytography, but I'm not sure if I'll ever end up using them, nevermind for normal programming jobs.

It's rare that you need it, but it helps having it under your belt, so to speak. You never know when you come across a problem which some algorithm or some mathematical solution could simplify greatly. It's usually when solving hard problems or writing libraries to do very specific things and suddenly you care about performance because your code will be called by 10000 machines 1e6 times a minute and it will cost someone real money if it takes twice as long.

 

[sadrabbit]

It's rare that you need it, but it helps having it under your belt, so to speak. You never know when you come across a problem which some algorithm or some mathematical solution could simplify greatly. It's usually when solving hard problems or writing libraries to do very specific things and suddenly you care about performance because your code will be called by 10000 machines 1e6 times a minute and it will cost someone real money if it takes twice as long.

Oh I never thought about it that way, that's a really good point. It's nice to know that they can be so helpful without their usage being absolutely dire, unlike what someone earlier had been saying.

 

[Splendid]

There are other reasons not to like C++, like the insane toolchain and long compile times. Both of those issues affect C# as well, though.

What the fuck are you talking about? C# does not have long compile times, and the toolchain is about as normal as can be. Are you confusing C# with COBOL?

 

[CrunkLord420]

What the fuck are you talking about? C# does not have long compile times, and the toolchain is about as normal as can be. Are you confusing C# with COBOL?

I'm used to working with Golang these days, basically everything has long compile times in comparison. The toolchain is also similarly slim.

 

[Splendid]

So then it's not so much that C# is worse in this as it is that Go, a niche language, is better.

 

[CrunkLord420]

I would not describe OpenRA as being faster or easier to compile than something like gzdoom or similarly sized projects written in C++. I know you can make C++ slower to compile if you use stuff like constant expression or function templates, but not everything is like that.

 

[Shoggoth]

I'm used to working with Golang these days,

I'm sorry for your loss
Jokes aside, thoughts on the language? I could probably enjoy it if I've never seen FP in my life, but I think I joined a different cult.

 

[CrunkLord420]

I'm sorry for your loss
Jokes aside, thoughts on the language? I could probably enjoy it if I've never seen FP in my life, but I think I joined a different cult.

Golang isn't a functional programming language. It's basically OOP despite what some people might say, maybe OOP-lite.

It's great. I love the benefits of a typed language, with the "batteries included" libraries that makes everything very straight forward. Compilation times are practically instant. I like that I don't have to deal with the makefile madness that comes with other languages. For 90%+ the speed of C, it's worth it.

In my experience getting intellisense or autocompletion for programming languages can be tricky. Libraries are a big deal and it can be a giant chore just to get your stuff to pull from your cmake or whatever. The Golang tools are amazing in this regard, it nails auto-completion even in third party libraries. Vim-go specifically is great, YouCompleteMe also has great Golang support.

 

[Shoggoth]

Golang isn't a functional programming language. It's basically OOP despite what some people might say, maybe OOP-lite.

It's great. I love the benefits of a typed language, with the "batteries included" libraries that makes everything very straight forward. Compilation times are practically instant. I like that I don't have to deal with the makefile madness that comes with other languages. For 90%+ the speed of C, it's worth it.

In my experience getting intellisense or autocompletion for programming languages can be tricky. Libraries are a big deal and it can be a giant chore just to get your stuff to pull from your cmake or whatever. The Golang tools are amazing in this regard, it nails auto-completion even in third party libraries. Vim-go specifically is great, YouCompleteMe also has great Golang support.

I don't think I'd call OOP. Not only because OOP failed and Go is making the rounds, but because there aren't many of the characteristics of it. Embedding and defining functions on types doesn't make one OO.
Go also has lexical closure and decent anonymous functions syntax, which is nice, but it doesn't make it functional. It also usually passes arguments by value and not by reference, which is interesting, but there's plenty of gotchas there.
The build time is impressive. I recommend reading the README in the compiler's library and watching Rob's lecture on the golang assembler if you're interested.

And why aren't you programming Go in Acme, like your grandfathers intended?

 

[CrunkLord420]

The main issue I have with go is the amount of boilerplate you have to write to make interfaces sometimes, where you have to duplicate function code over and over. I hope this will be solved with generics/"contracts".

 

[Where Do You Find Them?]

The problem with mathematics in computing isn't so much that 90% of real world computing tasks don't have a solution that relies on some advanced mathematical concept, but shouldn't be implemented with it even if it does exist. You should write the problem not just as simply as possible but in a way that mirrors the actual real world problem you're solving.

The main reason for this is maintainability. Real world problems change unpredictably and when your code mirrors that problem it'll be in the best position to change alongside it. For example, let's say I'm writing something that takes a product design and makes sure that it meets a specification. I'm given a handbook with all the specs the design must meet. I could look through this book and maybe reduce it down somewhat. Maybe there's even something like "There must be 6 parts. Each part must weigh no more than 10g. The whole must weigh no more than 100g." where I can just remove a rule entirely but I probably shouldn't because a year from now that handbook is going to change and I'll have check over everything. I need to notice that a change like "Each part must weigh no more than 20g" means more than just changing that single constraint.

The only real exception to this is optimization, but most of the time optimization just doesn't matter. Even in fairly extreme cases, let's say it takes 10 seconds to check a design and I can get that down to 0.1. Nobody cares. The designers will wait happily wait 10 seconds at the end of a months long design cycle. You should only be optimizing when it becomes apparent that you have to and even then never do it blind. If you haven't profiled you're probably just wasting your time optimizing things that aren't the bottleneck, which is actually the problem with the example I just used, 10 seconds is fine because your program isn't the bottleneck in the design process.

That's not to say there isn't a lot of math that's really fucking useful, but it's usually either domain specific stuff like coordinate transformations for a physics engine or things that you'll learn in a computer science context and use all the time, like group theory or graph theory.

 

[CrunkLord420]

I've been doing a bunch of 6502 ASM programming recently. Ask me about doing 16-bit math on an 8-bit processor, and using bit shifting for division.

 

[Shoggoth]

Minimal edit distance between strings is also a "mathematical" problem, and for every system you build that has versioned instances of text you might want something like that. Sure, if you have 100 users they'll be happy to wait 10 seconds, but sadly, user experience is important, and if you want to grow your market, slowly responding software isn't acceptable. Not to mention any software which has to process a large number of similar inputs (stream processing, web servers, etc) will get clogged if you don't trim everything down to best performance, or you'll get out-competed

 

[Where Do You Find Them?]

Performance is something people get wrong all the time so I'll cover it briefly.

There is no throughput increase in improving anything that is not a bottleneck. It's the equivalent of buying a faster CPU because your files aren't downloading fast enough. For instance, let's say I have a big list of numbers and I take all of these numbers and add 5 then multiply by 2. People imagine that if I took the same list and just added 5 it would run faster. Surely right? I'm only doing one operation this time and I've gotten rid of the multiplication which is the expensive one.

It doesn't. There is no throughput increase at all. This is because that algorithm isn't bottlenecked by the CPU (on any reasonable hardware). I could do many more operations before it became CPU bottlenecked and started to slow down. The CPU will not be going at 100% anyway because it has to wait on the memory bus. A modern CPU can do about 400 cycles in the amount of time it takes to fetch a memory location.

My bottleneck here is the memory bus so the only way I'm going to see performance increases is by making it touch less memory lines (or have better locality, but that would be negligible) and assuming I'm already using a flat array for my numbers, there's no way to do that. Any time spent optimizing the actual operations done on the numbers is completely wasted. Latency is more likely to show some improvement but it's often marginal. The latency difference between those two would be the length of a single multiply by 2 operation no matter the size of an input block.

A lot of the misconceptions around this come from the misconception that our machines are inherently serial. That it goes "load the number then add 5 and maybe times two and then save it back" and it waits for one operation to finish before it starts the next, but modern (90's is when this started to be used) CPUs don't work like that. They are capable of queuing up operations, working out which ones are reliant upon memory loads and doing them out of order (look up Tomasulo’s algorithm for more information on how this is handled). They can even predicting which side of a branch is more likely to be taken and execute past it despite not having loaded the memory it needs to know for sure yet and then discarding the work it did as a result if it turns out it mispredicted.

It gets a hell of a lot more complicated than that, optimization is an arcane art, but the upshot is that randomly "optimizing" things will usually do literally nothing, especially for the throughput, and "eyeballing" bottlenecks is notoriously difficult. Don't prematurely optimize. Use profiling tools.

 

[CrunkLord420]

Some Taiwanese NES bootleggers ported Earthworm Jim 2 to the NES and created some insane copy protection for it: http://blog.kevtris.org/blogfiles/EWJ2PROT.TXT

The layers of protection are many: they are arranged like so:

1) Bogus writes to registers that don't exist.
2) Reading open bus values and using them later.
3) Bankswitching in the middle of a routine, which drops you into the middle
of another.
4) Use of what looks like "junk" code that really executes
5) Using "deterministic" code (see below) to write instructions into RAM
6) Writing seeming "junk" into RAM, then using it as an address to write to
which performs a bankswitch
7) Using 2 types of hash functions on PRG ROM banks to get values which
are then fed back into the hash to generate new values and code(!)
Manufacturing a jumptable and critical mapper write routines in the RAM
using said deterministic code
9) Piping all bankswitch and other critical functions into RAM, and/or
the jumptable in RAM
10) Putting traps in the code so that if a value is not right, it hangs up
the CPU or crashes it.
11) Using JMP() and PHAHA:RTS to generate impossible to follow branch
points all over the code.
12) Using mapper register mirrors to mask writes to the mapper, and to
mask how many registers there really are.

 

[Yotsubaaa]

A lot of the misconceptions around this come from the misconception that our machines are inherently serial. That it goes "load the number then add 5 and maybe times two and then save it back" and it waits for one operation to finish before it starts the next, but modern (90's is when this started to be used) CPUs don't work like that. They are capable of queuing up operations, working out which ones are reliant upon memory loads and doing them out of order (look up Tomasulo’s algorithm for more information on how this is handled). They can even predicting which side of a branch is more likely to be taken and execute past it despite not having loaded the memory it needs to know for sure yet and then discarding the work it did as a result if it turns out it mispredicted.

I agree absolutely with all your points. I'll also add to your points about benchmarking by further emphasizing: actually, physically do the benchmarks! Don't just eyeball it; no matter how 'obvious' an improvement it appears to be.

Here's an anecdote. Something I saw back at the start of the year, coding for physics-related research. One of the newer researchers thought he was so cool writing all of his polynomials in Horner form, e.g. writing

1+2x+3x^2+4x^3 (with 3 additions and 6 multiplications)​

instead in the form

1+x(2+x(3+4x)) (with 3 additions but only 3 multiplications.)
​

Seems like a great idea when you first see it, right? However, there's three not-so-obvious downsides:
#1. By writing everything in an unintuitive form like this, you've drastically improved your chances of fucking up your arithmetic somewhere,
#2. Depending on arithmetic overflow/machine epsilon and similar considerations, these two (mathematically equivalent) expressions might not even yield the same answer in practice,
#3. Modern compilers automatically do this for us behind the scenes most of the time anyway (and some are even smart enough to take #2 into account while doing it!), so it was a complete waste of effort.

Actually physically doing the benchmarking would reveal #3 immediately. (And probably save you the trouble of finding out about #2 years later when it crops up as an obscure bug that's almost impossible to trace back.)
Always benchmark your optimizations!

 

[DNJACK]

So then it's not so much that C# is worse in this as it is that Go, a niche language, is better.

no. having to use precompiled headers for reasonable compilation time makes c like languages some of the worst for compilatoin toolchain.
We will see once module are introduced but so far c++ has failed to solve language problems reasonably.

 

[Shoggoth]

Performance is something people get wrong all the time so I'll cover it briefly.

There is no throughput increase in improving anything that is not a bottleneck. It's the equivalent of buying a faster CPU because your files aren't downloading fast enough. For instance, let's say I have a big list of numbers and I take all of these numbers and add 5 then multiply by 2. People imagine that if I took the same list and just added 5 it would run faster. Surely right? I'm only doing one operation this time and I've gotten rid of the multiplication which is the expensive one.

It doesn't. There is no throughput increase at all. This is because that algorithm isn't bottlenecked by the CPU (on any reasonable hardware). I could do many more operations before it became CPU bottlenecked and started to slow down. The CPU will not be going at 100% anyway because it has to wait on the memory bus. A modern CPU can do about 400 cycles in the amount of time it takes to fetch a memory location.

My bottleneck here is the memory bus so the only way I'm going to see performance increases is by making it touch less memory lines (or have better locality, but that would be negligible) and assuming I'm already using a flat array for my numbers, there's no way to do that. Any time spent optimizing the actual operations done on the numbers is completely wasted. Latency is more likely to show some improvement but it's often marginal. The latency difference between those two would be the length of a single multiply by 2 operation no matter the size of an input block.

A lot of the misconceptions around this come from the misconception that our machines are inherently serial. That it goes "load the number then add 5 and maybe times two and then save it back" and it waits for one operation to finish before it starts the next, but modern (90's is when this started to be used) CPUs don't work like that. They are capable of queuing up operations, working out which ones are reliant upon memory loads and doing them out of order (look up Tomasulo’s algorithm for more information on how this is handled). They can even predicting which side of a branch is more likely to be taken and execute past it despite not having loaded the memory it needs to know for sure yet and then discarding the work it did as a result if it turns out it mispredicted.

It gets a hell of a lot more complicated than that, optimization is an arcane art, but the upshot is that randomly "optimizing" things will usually do literally nothing, especially for the throughput, and "eyeballing" bottlenecks is notoriously difficult. Don't prematurely optimize. Use profiling tools.

Important post.
Regarding profiling and visibility, even benchmarks can be misleading. A case study in that can be clj-tuple and how it got rejected from making it into clojure's core. While all the benchmarks have shown improvements, it turns out that switching out collections' implementation as they grow from under the JVM's hands dynamically screws with hot code path optimizations and JIT. Don't just bench. Test in production environment.
Another example is Uber's golang troubles regarding stack resizing in runtime. You could only see it in runtime because only then the stack was big enough. Benchmarks didn't catch it.
For visibility in production, metrics and flame graphs are great, If you put meters and timers on the right bit and pieces of your software you can find the true bottlenecks.
codahale: https://metrics-clojure.readthedocs.io/en/latest/
clojure metrics: https://github.com/metrics-clojure/metrics-clojure
jvm async profiler: https://github.com/jvm-profiling-tools/async-profiler
clojure async profiler: https://github.com/clojure-goes-fast/clj-async-profiler
You can also attach to a JVM to generate a flame graph. Pretty nifty.