Hacker News new | ask | show | jobs
by gered 2809 days ago
I can't say I've personally seen any evidence of "mass migrations" of this sort. However, I can personally say that after 5 years of working with Clojure and with several large Clojure projects under my belt now, I'm starting to realize that I'm losing confidence in using Clojure for large projects and the feeling of how _maintainable_ that project will be in 2-3 years or longer. I think Clojure hits a sweet spot (at least for _me_ personally) for small-to-medium sized projects though, and I don't hesitate to use it in those cases.

The thing I've _consistently_ seen with every large Clojure project I've been a part of is that as the project grows, it becomes harder to make sense of the types used across the project. On the last couple of these projects I've been on, Schema was used to annotate many types, but I've never found this to be quite enough and I still remain firmly convinced that gradual typing just doesn't work well enough generally. I think that with a _very_ highly disciplined team of experienced developers it could probably work well. But we don't all get the opportunity to work on such teams. What I've personally seen is developers are all too willing to either add a partial type spec/annotation (with probably some s/Any's lazily thrown in where there should most definitely _not_ be), or just flat out not adding any at all ("Why would I need it here? This is very simple code and it's obvious what is happening here!" ... sure, maybe it is to you now... what about in 6 months? Or what about to one of the other developers on the team?).

When discussions about static vs dynamic typing come up (which is almost always a fruitless argument in my experience), I've noticed that the dynamic typing proponents tend to get overly focused on the idea of "correctness". This has always disappointed me. Correctness is certainly an important benefit (though I think it tends to get exaggerated a bit much), however, to me the idea of documentation in the form of a type definition is _much_ more valuable over the longer term, especially so for large projects. Developers are usually quite lazy, especially when it comes to documentation, and often when looking at a piece of code in a large code base, the only documentation about the values being passed to a function is going to be the type definitions (aside from reading the code itself of course).

4 comments

> using Clojure for large projects and the feeling of how _maintainable_ that project will be in 2-3 years or longer

Maintainable in what sense? And large in what sense?

I typically work on 1-10 million lines of Java code solutions split up into maybe 20-50 seperate components.

These 20-50 seperate components have untyped boundaries between them.

Setup correctly these untyped application boundaries rarely cause problems - of course you have to put a bit more thought into architecture, documentation, and verification but it’s not particularly onerous and we have been doing it successfully for decades now.

I’d say anecdotally errors caused by this lack of typing are well under 1% of total errors.

> every large Clojure project I've been a part of is that as the project grows, it becomes harder to make sense of the types used across the project.

Are there languages you've seen that don't show an ugly side as projects grow to a large size?

You see these migrations as companies grow past certain engineer counts if they start out with dynamic languages. Sometimes they might even modify the language itself to bolt on types on it.

Ex: Facebook and adding types with Hack & Flow. Dropbox's work to add types to python. Uber used to be a python and nodejs shop, and now new stuff is done in golang and java. Many nodejs shops go from pure js to typescript or flow.

Gradual typing is useful for companies that don't want to immediately migrate everything at once. If you have the right policy of 'type everything you touch', enforced with a linter, a codebase gets migrated relatively quickly.

My question is why anybody thinks that writing a project in a monolithic style is a good idea. Any large project can be broken down into small independent modules, and there are tons of benefits to doing that aside from being able to track types in it.

Independent modules are easier to reason about, they're reusable, and they can be maintained on independent schedules. The biggest advantage though is that this approach allows you to split teams up into smaller groups responsible for the individual modules. One of the biggest problems in maintaining large projects is communication overhead, and typically it's very difficult for teams with over 5-6 people to be productive.

So, if your project outgrows limits of dynamic typing, chances are it should be split up instead of using static typing as a crutch.

Every programming environment has down sides and Clojure is no exception but I still think that using Java instead of Clojure is not better because simply you have 3-10x source code to maintain and the number of bugs are correlating with number of source code lines. On the top, I quite often see null pointer exceptions in Java when in theory that should never happen because we are statically typed.
Pick a more modern language than 20 year old Java, such as Kotlin, and you get the best of both worlds: static typing and type inference.
Yeah it is on my list to check for a while. If it has the same interop with Java as CLojure than I will give it a try.
I'm not sure why you're contrasting Clojure and Java, but nowhere was I recommending using Java over Clojure (nor would I ever offer such a recommendation). Certainly there are much, much better choices for a statically typed language out there.
People usually not just chose languages. The VM, libraries and dependency management all together are chose. Btw. this is why Rich started Clojure on the JVM. If you stay on JVM land I quite often hear that the developers do not want to try Clojure because of the lack of types (which refers to static typing in this context). Good luck implementing a Hadoop project in OCaml or Haskell.
It seems like you're jumping ahead to conclusions waay to quickly here.

Instead of that, it would be better to respond directly to the specific points the OP made, rather than move the argument to a bunch of new topics.

OP did not say anything about other languages, be it Java or otherwise. OP is just describing experience with closure.

In order to understand clojure better, it would be good to not shut out feedback.

This is exactly the point, you can't just chose languages because of type systems there is a lot more to these decisions. OP has an experience with Clojure (which has closures) but these experiences are do not exists in the vacuum, there are tradeoffs that you take with every language.
>On the top, I quite often see null pointer exceptions in Java when in theory that should never happen because we are statically typed.

In this sense Java is actually dynamically typed: there's no way of knowing at compile time whether an object is actually an instance of that object or is null. In languages like Kotlin, Haskell and Ocaml, which specify in the type system whether or not something can be none and force the user to check before using something that can be null, null pointer errors don't happen.

> In this sense Java is actually dynamically typed: there's no way of knowing at compile time whether an object is actually an instance of that object or is null.

That is an extremely unusual definition of dynamic typing. If I have an Integer it might be null but it won’t be a BeanFactory (unless someone did a reflective call or wrote some bytecode or otherwise subverted the type checker).

Well in my ideal word nulls are not part of Integers or Floats or BeanFactory.
Static type systems have scopes in how much they type. Some newer ones add nullable annotations, like kotlin or swift, others don't have it yet.

I hope java gets native support for @nullable, but for now we have stuff like error prone and @nullable annotations.