[jb1991]

It's a lot of fun but as projects got larger and larger for me (thousands of lines, or even tens of thousands), I found the dynamic typing taking up more and more of my time. I've since moved on to statically typed systems where the compiler takes a big load off the cognitive requirements of maintaining and debugging software.

[didibus]

How long have you been programming professionally?

Did you have prior experience with dynamic languages? If so, how much?

Did you have prior experience with statically typed languages? If so, how much?

I'm trying to see if Clojure requires more fundamental programming intuition and experience to sustain in large projects. I work with tens of thousands of Clojure LOC, I don't feel these issues and can't relate, and love Clojure, so I'm curious to understand what context it best applies too. I wouldn't want to force it on a team that wouldn't benefit from it, so I'm interested about learning these aspects, so I'm able to recognize in what context it would make sense for me to influence a team to adopt it or not.

Thanks.

[dkersten]

12ish years (more as student & hobbyist), lots (mainly python, javascript and clojure), lots (mainly Java and C++, sadly inly got to tinker with better type systems). I consider myself a clojure developer and have been using it on and off since summer 09. Im using it for a large project now.

And yet... I agree with GP. I love clojure, but over time I’m becoming less and less sold as dynamic typing. Spec helps, a little. Property-based generative tests (especially when used with spec) helps a little too. Neither are a replacement for proper static types, though, especially an ML-esque type system with type inference. Bonus points if spec validation could pass type data to the compiler/type inference (eg the code path after validation can assume that the data is of the the types described in the spec).

I dream of a statically typed Clojure with type inference, spec-inferred types and optional dynamic typing for REPL experimentation and glue code.

[didibus]

Very interesting, and thanks for answering.

Could I ask what kind of project it is? I wonder if I'm just lucky that Clojure fits perfectly my use case, which is mostly a set of distributed systems of all kinds. So while as a whole there's tens of thousands of LOC. The components have very strong boundaries being as it's a set of services assembled together through RPC, PubSubs and DBs. Maybe that alleviate the lack of a static type checker.

We've adopted Clojure about 3 years ago, team of 10. We've had a few people leave and join throughout. Only one person knew Clojure beforehand. Our stack is about 50% Clojure, 40% Java and 10% Scala. Of all three, Clojure has given us the least issues, has been pretty easy to maintain and generally has fewer defects. Java tend to have the most bugs, almost always related to some shared state. Scala I find the hardest to extend and maintain, but the code base we have for it I think does Scala wrong, it's like the worst mix of OOP and FP.

> I dream of a statically typed Clojure with type inference, spec-inferred types and optional dynamic typing for REPL experimentation and glue code

That's pretty much exactly core.typed: https://blog.ambrosebs.com/2018/09/20/towards-typed-clj.html

That said, the project never managed to get more contributors.

If you're looking for a typed Lisp, I've been keeping my eyes out on Carp: https://github.com/carp-lang/Carp

[dkersten]

I’ve written a lot of different Clojure projects over the years. Some large, some small. Back in 2013–2015 I ran a startup entirely on Clojure(script). My current project is an automation service for cryptocurrency trading bots (that is, all the infrastructure, automation, configuration, dashboard etc for running a bot, but the bot strategy and signals are up to the user — hence automation tool, not bot). It’s a large system with multiple services (some running in different datacenters too). Backend is Clojure (based on duct) and frontend is Clojurescript with re-frame.

Don’t get me wrong, if I restarted the project again from scratch, I’d choose the same setup (or a very similar one). I love Clojure and am very productive in it, but that doesn’t mean I don’t think it could be better still.

> That's pretty much exactly core.typed

I haven’t looked at it in a couple of years, maybe its changed, but when I did, it didn’t really do it for me. It’s still a separate tool that lives separate from Clojure itself and it felt very “heavy”. In my personal opinion and experience, having certain things as separate entities (decomplected as Rich would say) isn’t always a good thing and leads to an inferior thing. I’ve played around with many programming languages in my time (I’m a bit of an enthusiast, I guess. I like trying out languages that are very different from what I already know – that’s how I originally got into Clojure) and it seems like a common theme. The closer a feature is to the compiler/runtime, the better it works and seamless it is over all. Another Clojure example is the limitations core.async has, because its an external library: things like <! cannot be placed inside functions or the go macro can’t see it to transform it as macros cannot look inside function calls. I’ve also encountered an exception recently where the stacktrace only showed core.async and clojure.core code, not a single stack frame referenced MY source files. These problems are hard to solve as an external library.

> If you’re looking for a typed Lisp

I’m not. I like Clojure’s particular mix of sensible syntax, immutability, sequence abstraction and general way of doing things. Other Lisps I’ve looked at don’t have the same emphasis on these things as Clojure does, so I don’t want another Lisp, I want a language that makes the exact same decisions and tradeoffs as Clojure, except on dynamic vs static types (and actually useful error messages). Maybe one day I’ll give it a try, I certainly don’t expect Cognitect to change their language because of what my preferences are.

[didibus]

It's not as simple as that. There are no typed Lisps, even though Lisps have been around since the 60s. That's not just coincidental in my opinion. The closest to a typed Lisp are gradual typed Lisp, like Typed Racket, and that is very similar to how Core.typed does it. There is Shen as well. Carp is the first strongly typed Lisp I'm seeing, and it is experimental and might never take off.

The issue is how would type definitions be introduced, and what kind of types would be most appropriate? As you said yourself, the way core.typed did it felt too "heavy". Yet it isn't clear how to make a more lightweight variant for a Lisp language such as Clojure without rendering the types worthless.

The first, and one of the biggest issue in my mind, is that what everyone loves about Clojure is the data-oriented style. In that style, you represents entities and their relationships using heterogeneous collections. That's where in Clojure you model your domain with Maps, Lists, Vectors, Sets, etc. It is awesome, but no one has figured out a non "heavy" way to statically type it. All methods I know of have bad programmer ergonomics. In effect, adding types back to it almost kills the data-oriented style, and it ends up feeling a lot like modeling with Classes instead. See Haskell's wiki section on this problem: https://wiki.haskell.org/Heterogenous_collections they haven't solved it, and have multiple ways to possibly handle the scenario, and non are ideal.

> I want a language that makes the exact same decisions and tradeoffs as Clojure, except on dynamic vs static types

I would too, but with the caveat that the development experience would be the same, and the programming ergonomics and styles would be retained. And this, I'm afraid, is an open problem that no one has solved yet. It's not just a case of personal preference. Having a language which has the pros of Clojure and the pros of static types, without the cons of static types is hard. That's why for now, you need to choose one or the other.

[kickopotomus]

This has been my experience as well. I have a couple of Clojure(Script) applications that are approximately 3-5K lines each and those have been a pleasure too work on. However, my latest project is now pushing past 20K lines and the mental load has jumped exponentially. There is a much greater need for spec, asserts, type hints, and comments just to keep everything straight.

[slifin]

That's interesting do you have any idea why the mental load jumped? would a static analysis tool working with your type hints help?

Or is there many things to consider at once in the system instead of many individual things?

[kickopotomus]

The biggest issue I keep running into is just the concept of data "shape". I love that clojure gives you so much freedom but it can be quite the footgun in a large system because you see that a function expects a map with keys :foo, :bar, and :baz but what are the values for those keys? Spec helps a little bit here for primitive values but for complex nested structures (e.g. {:a [{:b [1 2]} {:c "bar"}]}), it doesn't do much. So, as data moves through the system, and as the system grows it has become increasingly difficult to track the mutations to the underlying structures.

I do think that a static analysis tool would be of some help. Some sort of tooling to better handle tree structures would be very handy. I often find myself being off-by-one level with get-in calls on tree data (E.g. (get-in m [:a :b :d]) where m is {:a {:b {:c {:d 1}}}}), which is annoying because the NPE gets thrown 3 function calls up the stack.

[escherize]

I'm late to the thread, but I've felt the same pain and wrote a library to help deal with the cognitive load of working on "shapes" of data [1].

[1] https://github.com/escherize/tracks

[sooheon]

Totally not saying "you're holding it wrong", but maybe once you're more than a few couple levels deep into a nested map it's time to look at an in memory db like [Datascript](https://github.com/tonsky/datascript)? Actual Datalog queries can replace get-in vectors growing out of hand, and you also get better mutations with transactions.