[stcredzero]

Would a Go programmer even dream of bootstrapping fancy data structures from simpler ones?

What use is there for a fancy data structure? In practice, these occasions aren't that common. Many "fancy" data structures tend to exhibit bad cache behaviors if implemented naively.

At scale, the law of large numbers says that even improbable events will occur every now and then. Unfortunately, a program with even one bug is still incorrect.

Are you an undergraduate? Depending on how you interpret the spec (which isn't cut and dried when business requirements meet the real world) almost every page of production code has some kind of bug in it. Also, the law of large numbers isn't that relevant for most codebases and developer populations -- the numbers aren't that large. The effect of hubris is much larger in practice.

[catnaroek]

> What use is there for a fancy data structure?

Improving asymptotic bounds. Providing functionality typically not supported by common data structures. (e.g., I want a key-value container that's a priority queue on keys, but concatenates multiple values associated to the same key)

> Are you an undergraduate?

No.

> Also, the law of large numbers isn't that relevant for most codebases and developer populations

The law of large numbers certainly applies to >100 KLOC codebases, unless your bug rates are somehow magically two or three orders of magnitude lower than the average.

[stcredzero]

> Improving asymptotic bounds. Providing functionality typically not supported by common data structures.

It's nice to know that you remember stuff from the textbook. However, how often do you need to do something like this for real production code? Depending on what it is you normally do, it's entirely possible that you need to do this every other project. It's also possible that you never have a real need to do these things. (It's also possible that you never have a real need to do these things, but you do them anyways, which is far, far worse.)

> The law of large numbers certainly applies to >100 KLOC codebases, unless your bug rates are somehow magically two or three orders of magnitude lower than the average.

True, which is why I'm pretty confident that 15 years of Smalltalk development on many large code bases without running into a heterogeneous collection debugging conundrum is possibly a valid data point.

Contrast that with an endless parade of "hubris coding" in the same timeframe. My impression is that the damage caused by "hubris coding," or the gratuitous worship of "cleverness," far outweighs that caused by insufficient type information by 2 or 3 orders of magnitude. If you're going to be clever about applying your clever, you need to apply it in a fashion where it gives your company's business the biggest bang for the buck. Most coders in their 20's are just trying to impress their fellow programmers.

[catnaroek]

> It's also possible that you never have a real need to do these things.

What I don't have a real need for is the ability to destroy the internal invariants of other modules. :-p

> True, which is why I'm pretty confident that 15 years of Smalltalk development on many large code bases without running into a heterogeneous collection debugging conundrum is possibly a valid data point.

Who says homogeneous collections are the only use case for parametricity? Parametricity is useful whenever you need to make sure that unrelated parts of your program don't accidentally rely on (or, even worse, alter) each other's implementation details. “Modularity”, as they call it elsewhere. Of course, Smalltalk has none of this.

> My impression is that the damage caused by "hubris coding," or the gratuitous worship of "cleverness," far outweighs that caused by insufficient type information by 2 or 3 orders of magnitude.

I don't separate concerns to be “clever”. Au contraire! I separate concerns to deal with my own brain's limited ability to simultaneously process multiple pieces of information. (And I'll be perfectly honest: I also separate concerns because it's beautiful.)

“Hubris” is a term I would reserve for those who write large programs whose constituent parts don't have fixed structure, yet claim they understand what's going on in the code. (Or perhaps they claim “the tests do the understanding”?)

[stcredzero]

What I don't have a real need for is the ability to destroy the internal invariants of other modules. :-p

I only ever recall this happening when someone inadvisedly modified or added a method to a library. In my actual industry practice, type information never did anything to "preserve the internal invariants of other modules." The only time we lamented the lack of type information was in large scale refactorings.

Who says homogeneous collections are the only use case for parametricity?

No one. However, that was your example. My argument is that's a really poor example. And now you are abandoning it.

I don't separate concerns to be “clever”.

I also like separating concerns. I'm also not a dynamic typing bigot, though you seem to be imagining you are arguing with one. You seem to have devolved into abandoning your points of arguments and portraying your discussion partner as a series of strawmen. How in the heck did you get here from parametric algorithms? This smacks of intellectual dishonesty.

“Hubris” is a term I would reserve for those who write large programs whose constituent parts don't have fixed structure

And earlier, you were claiming something about refactoring. Do you see a contradiction here?

[catnaroek]

> In my actual industry practice, type information never did anything to "preserve the internal invariants of other modules."

Most languages don't have abstract types (not to be confused with abstract classes!), so there's that. Abstract types protect invariants of modules from external tampering. This is a mathematical fact.

> However, that was your example. My argument is that's a really poor example. And now you are abandoning it.

I'm not abandoning anything. I'm only saying that the use cases of parametricity go far beyond parametric collections.

> I also like separating concerns.

Good! Then what do you gain from the existence of reflection (which is pretty much the opposite of type abstraction), or the possibility of sending wrong messages? This is as anti-separation-of-concerns as it gets.

Even more worrisome is what you have said in another post: “Decompilation in Smalltalk is trivially perfect, excluding local variable names, so closed source was fairly pointless.” (https://news.ycombinator.com/item?id=12340864) How can you pretend this is compatible with separating concerns? You're talking about inspecting the structure of arbitrary parts of a program!

> I'm also not a dynamic typing bigot, though you seem to be imagining you are arguing with one.

I've just made technical claims. I haven't personally attacked you. If you think I did, my apologies.

> How in the heck did you get here from parametric algorithms? This smacks of intellectual dishonesty.

I also request that you refrain from making personal attacks.

Anyway. Parametricity means more than you think. The inability to inspect the representation of an abstract type is an example of parametricity too.

> And earlier, you were claiming something about refactoring. Do you see a contradiction here?

Nope, I don't see it. Refactoring produces a different program with a different fixed structure. And the difference shows up when the old and new programs have different types. The reason why types are helpful is precisely because they guide the evolution from the old to the new program.