Computer Model That Locked Down The World Turns Out To Be Sh*tcode

[Yotsubaaa]

So let me get this straight: this model is saying that a tenth of Sweden's population should be dead by now?

read the wrong graph for a moment

To be fair I was running it on their sample data, which they have the following disclaimer for:

So it's (probably) fictitious sample data being run with (probably) fictitious real-world scenarios specified in the parameter files. I wouldn't draw any inference from those graphs.

 

[MrBlueSocks]

I'm just just reporting on the links already posted here, as I have no interest in actually trying to get this shit to run like you nutters are.

github discussion . A professional modeller with impeccable credentials criticises it from that angle, ignoring code quality. Conclusion, it's a bit shit. Also points out that there were no modelling experts involved in the government's SAGE panel. archive

Reddit thread where an insurance modeller comments. Conclusion: This shit is amateur hour,. His own models are far better, yet he still wouldn't consider them good enough to make the decisions that were made off the Imperial study. archive

The comments in the second article have attracted a few more organised detractors now. They seem trollish , in particular; Another Anon Talking Head , earthflattener, thelastnameleft. They start by attacking the author for anonymity, rather than address any arguments. Then some whataboutery and distraction, accusations of far-right political bias. Also use a bad faith "technically you've just shown it's not up to your hoity toity standards, you haven't proved it's wrong" approach. earthflattener in particular seems .... invested. Check for butthurt@imperial.ac.uk .

Overall vibe - Professional private sector SMEs are shocked by the utter incompetence in all of this (as are all the Softies in this thread) , but a few academics are outraged that plebs with real jobs are daring to criticise.

"Uninitialised variables are an advanced technique to make my code reflect the chaotic nature of the real world. You wouldn't understand"

 

[Gustav Schuchardt]

So it's (probably) fictitious sample data being run with (probably) fictitious real-world scenarios specified in the parameter files. I wouldn't draw any inference from those graphs.

Which sucks, right? The whole point of releasing it is not so people can nitpick his shitty code, it's so they can nitpick his (probably shitty) model assumptions. Even though trying to tease out the model assumptions would be almost impossible and even if you managed it the policy damage has already been done.

I know a guy who was critiquing a paper that came out with a very politically useful conclusion. The end of that very long argument was that the paper had made an assumption about tax that was clearly bullshit and clearly designed to produce the conclusion the author of the paper wanted. Of course, it didn't mean shit because the political conclusion of the paper had already been acted on.

That's what this 'We're only basing policy on The Science' boils down to - they're basing in on one study which was designed to produce the conclusion the people pushing for a policy wanted. And by the time it gets debunked many more similarly politically convenient studies will come out and outside academia, no one cares about the details anyway. They'll just trot out platitudes about denying the science and how 'facts don't care about your feelings'.

Even though in this case if your feeling was 'I'm being bullshitted' it was pretty accurate and the facts are just the output of some model that was rigged to produce a conclusion that some policy advocate wanted.

 

[evrae]

I'm just just reporting on the links already posted here, as I have no interest in actually trying to get this shit to run like you nutters are.

github discussion . A professional modeller with impeccable credentials criticises it from that angle, ignoring code quality. Conclusion, it's a bit shit. Also points out that there were no modelling experts involved in the government's SAGE panel. archive

Reddit thread where an insurance modeller comments. Conclusion: This shit is amateur hour,. His own models are far better, yet he still wouldn't consider them good enough to make the decisions that were made off the Imperial study. archive

The comments in the second article have attracted a few more organised detractors now. They seem trollish , in particular; Another Anon Talking Head , earthflattener, thelastnameleft. They start by attacking the author for anonymity, rather than address any arguments. Then some whataboutery and distraction, accusations of far-right political bias. Also use a bad faith "technically you've just shown it's not up to your hoity toity standards, you haven't proved it's wrong" approach. earthflattener in particular seems .... invested. Check for butthurt@imperial.ac.uk .

Overall vibe - Professional private sector SMEs are shocked by the utter incompetence in all of this (as are all the Softies in this thread) , but a few academics are outraged that plebs with real jobs are daring to criticise.

"Uninitialised variables are an advanced technique to make my code reflect the chaotic nature of the real world. You wouldn't understand"

Thank you for posting this. It's somewhat discouraging personally to see that academia here has a bad case of having its head up its own ass ("I'm a scientist, I am above you plebians! You just don't get my intellect at all!" etc.). You'd think that people in science would be able to take criticism since they have to regularly go through the peer-review process but I guess not.
Seems like a few incompetents with a serious case of Dunning-Kruger in an attempt to get more funding, what a disappointment.

 

[Chad Nasty]

Am I reading this right? argv is an array of C-style strings which they are steadily walking through character by character within the if conditionals? That's why I'm stuck at home?

But muh degree! This whole thing should have been called bullshit and thrown out the minute these fags refused the first time to open the souce code.

 

[Pampered Degenerate]

Sadly, like many historical indicators of authority, holding an academic position these days is like having a Twitter checkmark but in real life.

 

[Kosher Dill]

Doesn't know about switches or enums, apparently.

Check InfStat.h for more "should've been enums".

They don't seem to know classes and virtual dispatch either.

In fairness, it does look like much of this code is ancient. A lot of it probably predates C++, or at least its widespread adoption.
EDIT: in fact I'm seeing some suggestions that the code was plain old C before they uploaded it to Github, and they just changed the extensions at that time.

 

[Overly Serious]

From the (now closed) issue on their Github page calling for papers based on it to be withdrawn.

EDIT: And what the Hell, you have to love how on point GitHub terminology is today. "Collaborator".

 

[MrBlueSocks]

Seems like a few incompetents with a serious case of Dunning-Kruger in an attempt to get more funding, what a disappointment.

In fairness, it does look like much of this code is ancient. A lot of it probably predates C++, or at least its widespread adoption.

I think it's a mixture of this. These people aren't thick. They may be arrogant, naive, isolated etc. But they're not unintelligent.

The code is ancient , and it's been built up by the big boss professor over many years. It is no longer 'just some source code' . It has become an ancient holy artefact. The constantly changing PhD students are not there to question the artefact, they are there to tend to the sacred flame. Going from PhD student to established academic is very difficult. It doesn't get easier if you commit heresy against your professors life work when you're still on the bottom rung.
In an ideal world there would be internal checks and criticism , but it's not surprising that it's absent. What is a disgrace though is the lack of external scrutiny. Absolutely your model and implementation of it should be part of any published study. Otherwise it is literally just "A program I wrote says this. Sorry, can't show you my working" . Science can't work if people don't attempt to tear down the sacred idols.

The Dunning-Kruger thing is important, (though I hate using that phrase as it makes you sound like an internet neckbeard wanker. It's been rendered nearly unusable , like gas-lighting etc.) They think coding is an irrelevant housekeeping task, and as long as it compiles, that's job done.

 

[Kosher Dill]

Did anyone else notice the "Pascal-isms" they were too lazy to fix when porting Rand.cpp?

long ignpoi(double mu)
{
/* blablabla */
    long ignpoi, j, k, kflag, l, ll, m;
/* do work */
    ignpoi = k;
    return ignpoi;
}

It's not wrong, but... ugh.

 

[Chad Nasty]

I think it's a mixture of this. These people aren't thick. They may be arrogant, naive, isolated etc. But they're not unintelligent.

The code is ancient , and it's been built up by the big boss professor over many years. It is no longer 'just some source code' . It has become an ancient holy artefact. The constantly changing PhD students are not there to question the artefact, they are there to tend to the sacred flame. Going from PhD student to established academic is very difficult. It doesn't get easier if you commit heresy against your professors life work when you're still on the bottom rung.
In an ideal world there would be internal checks and criticism , but it's not surprising that it's absent. What is a disgrace though is the lack of external scrutiny. Absolutely your model and implementation of it should be part of any published study. Otherwise it is literally just "A program I wrote says this. Sorry, can't show you my working" . Science can't work if people don't attempt to tear down the sacred idols.

The Dunning-Kruger thing is important, (though I hate using that phrase as it makes you sound like an internet neckbeard wanker. It's been rendered nearly unusable , like gas-lighting etc.) They think coding is an irrelevant housekeeping task, and as long as it compiles, that's job done.

Shit, they still run fortran at nasa. This whole fiasco should be a great selling point to if not completely overhaul, at least translate.

 

[Gorilla Tessellator]

All this debacle is not surprising, but still depressing. If you understand how much of the software in the world is shoddy, poorly constructed, brittle and riddled with bugs, you can sometimes wonder how this world is still able to function (somewhat). I wonder how these people can sleep at night knowing that they contributed to this clusterfuck by their sheer incompetence.

 

[Overly Serious]

why have you got that "Run" bit in your command? Isn't it just the four numbers that you need?

Thanks. The "run" parameter came from one of the READMES but I did what you did following your advice and just used the Python script. Unfortunately the most I could allocate to the container was 48GB which it complained it was unable to allocate. Whether that was because it genuinely needed more or because there was some Docker specific problem, I don't know. I removed the Run parameter and got as far as it complaining about a missing Update Timestamp parameter in one of the files and decided to look at this properly when I have more time. Thanks for your help though - you pretty much got me there with the Python suggestion.

I didn't mean IP in a literal / strictly legal sense. At least in principle, everything necessary to produce an implementation and reproduce the results should already be available in the literature, although it isn't more often than not. I meant that if academics start being compelled to give away their codes "for free", then the innovators will lose out to competitors who can just use it to publish lots of papers (i.e. academic output) rapidly. It's basically a similar situation to first world companies developing tech which they give to Chinese factories, who then pump it out at a rate and cost with which the developers can't compete.

Well, then Academia has a decision to make where to draw the boundaries to its community. But I stand by my claim that this specific instance doesn't have such concerns. Even aside from this code seemingly actually being their model, we're talking about tax payer funded research that is being kept from us that paid for it. If it were dangerous research there would be an argument but this is supposedly trying to save millions of lives. They should welcome people pointing out flaws in it. Instead they hid it for a month whilst they applied lipstick. That's beyond negligent. It's moved into putting your academic reputation above people's lives.

 

[dopy]

All this debacle is not surprising, but still depressing. If you understand how much of the software in the world is shoddy, poorly constructed, brittle and riddled with bugs, you can sometimes wonder how this world is still able to function (somewhat). I wonder how these people can sleep at night knowing that they contributed to this clusterfuck by their sheer incompetence.

jonathan blow (who's working on a game-centric C++ replacement called "jai," he streams it often on twitch) and casey muratori constantly have rants on their twitch streams about how shitty and slow software is today compared to 20-30 years ago (just like everyone else does of course). i found out on a stream with jon and sean barrett (creator of stb.h) that sean still uses VC6 (and is currently programming a C compiler in C on it) because it's faster than most everything else. here's a good rant about visual studio and corporate programming culture/twitter snobbery culture by casey:

 

[Pampered Degenerate]

Well, then Academia has a decision to make where to draw the boundaries to its community. But I stand by my claim that this specific instance doesn't have such concerns. Even aside from this code seemingly actually being their model, we're talking about tax payer funded research that is being kept from us that paid for it. If it were dangerous research there would be an argument but this is supposedly trying to save millions of lives. They should welcome people pointing out flaws in it. Instead they hid it for a month whilst they applied lipstick. That's beyond negligent. It's moved into putting your academic reputation above people's lives.

I agree, I was just pointing out the possible downsides and why it is not standard procedure. The ideal situation would be one in which it's possible to have academic codes go through some sort of QA process without being made completely public, but I suspect that it would be unfeasible, as it would be to verify people's experiments before publication.

However, as I wrote earlier in the thread, when a model is going to be used to determine government policy to such a major extent, it should be worked on and validated by an independent multidisciplinary team, not a single academic and his minions. Perhaps there should be something akin to the Met Office set up in the wake of this debacle.

 

[Chad Nasty]

All this debacle is not surprising, but still depressing. If you understand how much of the software in the world is shoddy, poorly constructed, brittle and riddled with bugs, you can sometimes wonder how this world is still able to function (somewhat). I wonder how these people can sleep at night knowing that they contributed to this clusterfuck by their sheer incompetence.

When I got into software I did so because it was a meritocracy and if some dude annihilated your code he did so because he took the time to actually teach you. Since then, all these soyboys have come into the market and have cried so much over "terse" language. Every code review I've had that has torn my asshole wide open has been a great learning experience. The greatest teachers I've had tell me straight up I'm being stupid.

I also think this bullshit code not being released outside of getting their fee-fees hurt, was that the model was a good scapegoat to enact more and more government overreach.

 

[drunk "Jedi master"]

at least it is not in cobal

 

[mindlessobserver]

On the site linked on the first page where the ex-google softie did the code review, most commentators understand and agree with the criticism . But a couple are still either deliberately or ignorantly misunderstanding or minimising the problems.
James
richard

Way to miss the fucking point. This is such a weasely politician answer that I suspect malice rather than ignorance. He's clearly not stupid, but he's trying to muddy the waters like a lawyer. "Oh your just complaining about the randomness, ..." .


The original code is so appallingly bad that I don't know how to explain it to a lay person . They think you're just being a gatekeeper, or nitpicky. I end up either relying on argument from authority "trust me", or repeated emphasis of "really really really shit. Really shit". It's not even teenage bedroom code. That has different characteristics, and isn't as deeply flawed.
I want to be able to explain this to people just how staggeringly fraudulently incompetent this is.
The results of this were believed by the government with nobody ever checking , even slightly, as to how they were arrived at. And the cost is measured in hundreds of Billions. Yet on a normal procurement project they argue the toss for months about minutiae. Government in this country ,is run by lawyer types, not engineer types. They're all just blagging it.

archive of lockdown sceptic code review

This just underscores the need for a departmental level agency in the US government just for IT infrastructure. It's been argued about for years and never done. Because the Relublicans dont want more government and the democrats are beholden to existing interests that would sooner give up control over programming contracts and managing the governments computer systems then gollum would give up the ring of power.

The end result being there is nobody in the room when it comes to big boy decisions about internet policy, or whether or not the program invented by some british faggot depicting our collective doom is actually accurate.

 

[Garm]

I'm just just reporting on the links already posted here, as I have no interest in actually trying to get this shit to run like you nutters are.

github discussion . A professional modeller with impeccable credentials criticises it from that angle, ignoring code quality. Conclusion, it's a bit shit. Also points out that there were no modelling experts involved in the government's SAGE panel. archive

Reddit thread where an insurance modeller comments. Conclusion: This shit is amateur hour,. His own models are far better, yet he still wouldn't consider them good enough to make the decisions that were made off the Imperial study. archive

The comments in the second article have attracted a few more organised detractors now. They seem trollish , in particular; Another Anon Talking Head , earthflattener, thelastnameleft. They start by attacking the author for anonymity, rather than address any arguments. Then some whataboutery and distraction, accusations of far-right political bias. Also use a bad faith "technically you've just shown it's not up to your hoity toity standards, you haven't proved it's wrong" approach. earthflattener in particular seems .... invested. Check for butthurt@imperial.ac.uk .

Overall vibe - Professional private sector SMEs are shocked by the utter incompetence in all of this (as are all the Softies in this thread) , but a few academics are outraged that plebs with real jobs are daring to criticise.

"Uninitialised variables are an advanced technique to make my code reflect the chaotic nature of the real world. You wouldn't understand"

Glad someone is looking into the oppositions talking points. Thank you.

I know enough about programming to get into trouble but not enough to take apart talking points.

 

[Overly Serious]

I know enough about programming to get into trouble but not enough to take apart talking points.

Do you know that the entire program being a single 15,000 line C file is a bad idea? 'Cause if so you know enough to criticise this.