[ctrlmeta]

Do you worry about correctness flaws in Julia that make it hard to write robust and correct code when you get deep into the weeds while developing libraries meant to be used by others or meant to interoperate with other libraries? It is not a problem with Julia but seems to be a problem in Julia's library ecosystem. It bothers me. Does it bother you?

[ssivark]

Would you prefer to add correctness tests to the compositionality that Julia gives “for free” due to multiple dispatch, or would you rather rewrite huge libraries from scratch? Eg: if you want a “new” feature like autodiff on numpy code then you need to reimplement numpy on top of every new autodiff implementation (PyTorch, jax, etc.)

I’m also curious to understand the reasoning, for either choice.

[sundarurfriend]

> Would you prefer to add correctness tests to the compositionality that Julia gives “for free” due to multiple dispatch, or would you rather rewrite huge libraries from scratch?

This is a hypothetical future choice, and in that case I would absolutely choose the former (and I hope we get there).

But the current choice is more like "composability that often isn't tested as generically as it should be, with no interfaces at the language level, even the abstract types used for the composition often not having clear specifications, all of it holding together only because of people fixing things organically as problems arise". Then the choice isn't as clear cut - choosing to go with the more limiting option where things are known to work together well with each other is a respectable, sane choice. If we're going to claim composability as a strength of the language, it shouldn't come with hidden traps, caveats like "composability but really only if the hidden assumptions on both sides happen to be satisfied".

[ssivark]

I appreciate that critique. So what would it take to make this pathway smoother?

Do you think interfaces might be a silver bullet? If so, what’s holding back the language from prioritizing this in the medium term? (For v2.0, say)

Or maybe the most useful improvements are anticipated to come from somewhere else?

[sundarurfriend]

Language level interfaces would be great, but they could also easily have the same problem Abstract supertypes currently have, of being poorly defined semantically, and hence being less useful than they could be. The main change needs to happen in the community, in terms of

* more formal specifications for Abstract types your package defines

--- including in the language itself; I understand there's recently efforts to define what an `AbstractString` is, exactly, and there needs to be a lot more of that

* tests that test the limits of this specification by defining new types that are very different from existing ones, but conform to the specification (rather than the current usual practice of just testing with already existing types only)

* More integration of things like OffsetArrays, DimensionalData, AxisArrays, etc. in test suites

* An explicit "Scope and Limitations" page in package docs (like Revise.jl's Limitations page [1]) that mention what has been tested and what hasn't/isn't meant to be supported

I'm sure there's many other low hanging fruits that the community could address by itself - it's pretty late here and these are what my brain could come up with right now. But I see this largely as a community best practices and culture problem, that language features can help with to some extent, but needs to be solved at the social level.

[1] https://timholy.github.io/Revise.jl/stable/limitations/

[sundarurfriend]

Jacob Nyssen downthread [1] has also made similar points, and more.

[1] https://news.ycombinator.com/item?id=32747387