Hacker News new | ask | show | jobs
by ldite 1350 days ago
Ah clojure. It's all fun and games until five years down the line, you've had 100% dev churn, and you have a 100kloc codebase that nobody understands, full of functions that don't give the slightest hint of the shape of the data they're processing (it's all lists!) yet down the bottom of the callstack there's some function that'll explode if the map doesn't have whatever magical key it expects.

If you're really lucky, someone will have thrown in a bunch of 'specs' that make a bunch of assertions about the data, put them on the API entry points, and then scattered some slightly different specs with slightly more restrictive assertions on various 'internal APIs', resulting in random explosions in production!

And the joy of working with an esoteric language is that it attracts esoteric developers, who often get frustrated by the requirements of being a software engineer in a large company (i.e. everything that's not writing code), which leads to the aforementioned 100% dev churn (after a lot of shouting).

14 comments

As the saying goes, a bad workman always blames his tools. You can make a mess in any language, and Clojure is no exception.

There are plenty of ways to mitigate the problems the author describes. A few of these things would be having coding standards to ensure that code is written in a way everyone is comfortable with. This also covers things like adding schemas and documentation for maintainability. Doing pairing and code reviews so that multiple developers are familiar with different parts of the codebase. Having good tests so that you can have confidence that the code does what's intended when you make changes.

I'm also not sure what makes Clojure esoteric. It's a clean and well designed language that embodies good principles. It's a niche language, but far from being esoteric. I've been using Clojure for over a decade now, and I've seen the opposite of developer churn. Most people are pretty happy to stay at a company using Clojure. The fact that there is high churn is suggestive of a toxic environment that drives people away.

Finally, here's [1] a talk from a founder of a Clojure startup that grew into a large successful company. Clojure didn't get in the way of that.

[1] https://www.youtube.com/watch?v=u-4FiFpkPlQ

I don't understand this reputation either. There are very large systems built on other Lisps, and they didn't collapse. What makes Clojure different?

For example, Emacs has a massive amount of Elisp. Elisp is much more primitive than Clojure, and traditionally libraries don't use e.g. data schemas [1] as runtime contracts for data.

Obviously, once a system built on top of a dynamic language grows beyond certain threshold, you need to be very disciplined as there are no static types to ensure some degree of correctness.

With that said, it would be interesting to have a language from the ML family as a viable contender. Perhaps a modernized version of Standard ML. Or perhaps OCaml, if it gains some traction with multicore and algebraic effects.

F# is nice, but it's a bit of a watered down version of SML due to the lack of functors. Scala is too complex and far from ML in some regards. Haskell is good, but lacks critical developer mass and some libraries / laziness overcomplicate certain usecases.

[1] https://github.com/plumatic/schema

I offered a counter example to the comment author too (Clojure has been wonderful for us), but I do wonder if there is one area where OO languages excel above functional: the shape of the code hints at the shape of the data.

In Lisp, function names describe actions. In OO, classes, inheritance, static properties, and getters and setters give at-a-glance hints as to the shape of the application's data model. In Clojure, we compensate, as you pointed out, with forms of documentation. I'll take that trade-off any day—I never want to go back to classes! But it's perhaps one reason for which Clojure has a reputation of being hard to read by other devs.

since around the early 80s it was common to write large Lisp software using class-oriented or even class-based OOP (a few used also prototype-based OOP). For example Flavors for the MIT Lisp Machine was class-based: multiple-inheritance, mixins, message sending with single dispatch, ... CLOS (the Common Lisp Object System) then changed OOP such that it better fits into Lisp: generic functions with multiple dispatch, classes, multiple-inheritance, meta object protocol, ... One of the drawbacks of those systems is that the code gets assembled at runtime (though cached) and thus to read and understand code one might need support from class/function browsers.
You can make a mess in any language, yes. But some languages tend to produce more messes than others. If your tool is mis-used more than other tools, at some point, it's not the fault of the users.
> If your tool is mis-used more than other tools, at some point, it's not the fault of the users

And Clojure isn't misused more than anything else. In fact, I've got a feeling it's less so.

The difference is just that you can scapegoat it, and people will believe you.

You can't blame Java for the failure of your project, cause everybody knows Java has worked fine for so many other teams, but nobody knows Clojure, so you can blame it for all your failures and people gives you the benefit of the doubt.

When you have a bad code base, and it's Java, people blame the people who wrote the code. When you have a bad code base and it's Clojure, people blame Clojure. Go figure!

I’ve personally had to deal with far more and worse messes in Python, Java and Javascript than I ever did in Clojure. (Started using Clojure ~13 years ago, but have been in jobs that use the other languages I’ve mentioned in that time too, not always exclusively using Clojure, although there were periods where I was).

I have mentioned a few times in the past, on HN and Reddit and elsewhere, that my biggest personal dream language wishlist item is “Clojure but with static types”, however, while that’s something I dream of having, it hasn’t held me or the language back in any meaningful way. The thing that makes Clojure less mess-inducing in my opinion is largely the fact that it’s data is immutable by default. An immutable-Python would be something I’d be interested in trying.

> I have mentioned a few times in the past, on HN and Reddit and elsewhere, that my biggest personal dream language wishlist item is “Clojure but with static types”, however, while that’s something I dream of having, it hasn’t held me or the language back in any meaningful way.

Sadly, Rich Hickey has always been pretty opposed to typing in Clojure. Enforcing values to be not null is basically table stakes for typed systems, and yet he doesn’t seem to think it’s valuable or feasible,

https://github.com/matthiasn/talk-transcripts/blob/master/Hi...

It does make me sad every time I have to deal with macro systems in non Lisp languages (Julia, Scala).

Yeah. By dream of I mean “if I had a on of free time, it’s something I’d love to make because I want it” while knowing it will never happen. I know that Clojure itself will never support it, outside of external projects like Typed Clojure
Are you familiar with Carp [1]?

[1] https://github.com/carp-lang/Carp

YMMV, but "messes" are for me things that manifest in projects spanning both multiple contributors and non-trivial periods of time. And in that framing, in my experience, the most effective individual proxy for "is this project a mess" is whether or not it's implemented in a dynamically typed language. Python, Javascript, Clojure, Ruby, my experience is that they all break down at scale, far more than equivalent projects in statically typed languages.
I’ve seen what I would consider messes in C++ and Java too, so it’s not exclusive to dynamically typed languages, but I’ve certainly seen enough in Python and Javascript to understand what you’re saying. It’s also a reason why I dream of a Clojure-like language with first class static typing, but, alas…
I've certainly seen far worse messes produced in languages like Java than in Clojure myself.
> As the saying goes, a bad workman always blames his tools.

And many times purveyors of a bad tool blame the workman.

If Clojure was a bad tool then people wouldn't be using it in the first place. There is a big barrier to getting started with Clojure because it's semantically different from mainstream languages. The mere fact that people use it and there are successful companies built on it demonstrates that the language does provide value.
You can say that about every language that's ever been used in a successful business. By that argument PHP is about the best language out there.
Your argument is based on a fallacy. The whole reason PHP became popular was that it was easy to get started with. The barrier for using PHP or JavaScript is very low. The opposite is true for a language like Clojure.
This is my experience exactly. I love the language and ecosystem in a lot of ways. I also believe that REPL-driven-development is a ridiculously productive way to work.

But I absolutely hate maintaining an old Clojure codebase (unless it's tiny).

The REPL helps a lot with discovering what the proper way is to call any random function you have in your code, but this is still really super annoying. I really hate to get into a dynamic-versus-static-typing debate, but I've long since come to the conclusion that -- all other things being equal (hah!) -- if I have to dig into a large and old project, I'd much rather have types by my side than not. Code will not ever be adequately documented or commented (and even if it _seems_ to be at first glance, you will always have nagging doubts about how up to date that info really is). This is where type definitions help to figure out the shape of the data that any piece of code is working with. People talk about adding spec/schema definitions but that doesn't solve all the problems with not having type definitions unless you add these spec/schema definitions _everywhere_ ... and let's face it, you just aren't going to do that in any Clojure project. So, best case scenario is you still have a large collection of functions in your project that are calling each other, etc that you are left having to deduce yourself what this random map or list actually contains.

As a Clojure admirer who has learned it enough to get things done (and subsequently forgotten it) twice, I recently had the positive REPL experience that people talk about. Or at least I think I did. I was able to write a web scraping tool almost entirely in the REPL.

However, I find myself running Ruby/Rails (console) in debug mode in RubyMine (Jetbrains IDE). The REPL-like experience seems quite close to that of Clojure, with the added benefit of being able to easily enable and disable breakpoints and see my local data (and snapshots of all previous local data up the call stack).

And honestly, the Clojure thing that always stops me from actually getting a full anything built is the lack of frameworks and approaches which have critical mass. It's always the same answer, "We like to choose our own libraries." That's lovely once you know what you're doing, but the ramp-up time is just SO much slower than with other languages.

Considering the time to working proof of concept is critical quite often, few beginners have time to figure out all the libraries and tooling and integrations to build something in Clojure.

I think despite still wishing I had a Clojure backend with Clojurescript/React frontend, I'll step from my Rails PoC to a Phoenix/Elixir product and be successful and happy (and have a lot of people doing something similar, with similar tools and libraries).

Yeah the (what I personally call it) "choose your own adventure" style approach to Clojure projects, where you don't use a framework like Rails, but just string together your own project from separate libraries, is really both a "pro" AND a "con".

It's great when you know what you're doing, and indeed, I have my own personal Leiningen templates to set up a Clojure project the way I like it and to save myself some time. Bigger project templates, like Luminus, I often find personally aggravating because I often feel like it just barfs a whole bunch of unnecessary and semi-complicated (in my opinion anyway) code in a new project even with the most minimal options chosen. But that's the power of the ecosystem ... you can create your own project templates to meet your own needs.

But a new developer getting their feet wet in this ecosystem? Yeah, it is hard. And even if they use an existing project template like Luminus to bootstrap their project ... well, the project template only helps generate the initial project. Ongoing maintenance for updating dependency versions and keeping a working integration of the libraries it initially set up for you (with respect to newer versions and any API/config changes, etc) ... well, those responsibilities are all on you! Kit (another newer successor to Luminus) _may_ provide some better alternatives here, but it'll still be limited with exactly how much it can help here. But I think it's still much too early to say one way or the other with Kit, so who knows.

(Also thanks for sharing your Ruby/Rails perspective on REPLs. A colleague of mine made some similar comments to me when we were discussing REPLs a while back, and I've not spent any time with Ruby so couldn't comment. It's interesting to hear! Most other REPLs I've used outside Clojure were not too useful as anything other than quick toys for trying short snippets outside of the context of a full project.)

Don't know about Ruby but agree that languages(think Java/C#/Javascript/C++) with really good debuggers don't have anything to envy from the Clojure REPL. But to be fair, companies have invested who knows how many millions to get that level of tooling while in Clojure and the nature of lisps you get it for free and low effort.
i'm starting to feel that REPL driven development can be bad for maintenance, when you have a REPL, you can write ridiculously compact and abstract code that is hard to understand just by reading it.
That's an interesting point I'd not thought of. I guess I'm more looking at it through the lens of "interacting with and modifying a running system" which kinda gives you a debugger (ish), compiler and execution environment all rolled into one. It's kind of nice to work in this kind of "scratch pad"-like environment while you figure something out versus the more traditional edit-compile-run cycle. But I have definitely seen what you describe as well, so I think that aspect is worth considering too, for sure.
It's almost by definition in some sense ... if it was obvious how to write the code you wouldn't need the REPL so by definition if it is uesful it must be producing non-obvious code.

That is a slightly shallow take though because its not really symmetrical like that. You can use a REPL as a way to arrive at the most easily understood rendition of something. But that is very prone to subjective bias as lots of things seem obvious in retrospect that are not at the start.

To the extent that the answer you eventually arrived at is a result of learning a mental model of the data and functions surround it, it will leave a significant residual gap for the next person who comes a long to bridge.

With great power comes great responsibility.

When using the REPL you can quickly experiment until it does what you want, but you need the discipline to distill something maintainable from it afterwards.

I agree. More generally, REPL-driven development rewards "guess-and-check" programming, and discourages "think-then-act" programming. This isn't a formula for success over time.
> ridiculously compact and abstract code that is hard to understand just by reading it.

For a maintenance provrammer against learning anything, abstract and compact are always bad.

Typically though, if you understand a few core concepts abstract and compact can be more readable towards the end goal of a high level understanding.

Oh man, this is not my experience. We may be different because we are a small company, and Clojure is our main language. But, we have a Clojure codebase of 90K lines, and is 10 years old. It has its problems, and while your points about an opaque data model resonate, for us, tests, specs, and assertions provide enough hints to help newcomers. And... where I disagree the most: the quality and professionalism of the developers who work on it are unparalleled. New, senior-level developers, quickly catch on to the system. I am grateful to Clojure __because__ it attracts this caliber of developer.
I'd say 100% developer churn is the real problem here, no inherited 100kloc codebase is going to be a walk in the park if the original devs are not around.

However, I do think that it harder to keep a large code base understandable when using a dynamic language v.s. a statically typed language.

Having problems with understanding the shape of expected data is certainly a problem experienced in larger Clojure code bases, I guess this is just a disadvantage of using Clojure.

Documentation, tests, spec/schema and good naming conventions can mitigate this disadvantage to some extent.

I found some the advice in this blog post useful: https://tech.redplanetlabs.com/2021/06/03/tour-of-our-250k-l...

OK but that's a 500k LOC codebase in javascript/python that nobody understands, maybe the project doesn't even get there

everybody hates their language when they have 100k LOC of tech debt from 8 years ago

You're right that Clojure's sequence soup problem is painful at that scale (really any scale) but have you ever debugged Java? It's barely even possible, the project needs to drive $10M+/yr revenue to just not collapse once it reaches 500k+ LOC Java, 500k XML, 300k SQL ...

> have you ever debugged Java? It's barely even possible

hard disagree ... Java is a dream to debug. I've hacked into complex applications countless times by remote attaching the debugger and setting breakpoints to step through what they are doing. You don't even need source code in many cases. I'd pick Java ahead of any other language on that front.

clojure inherits the exact same debug tooling you describe
except the debugging experience in clojure is much more difficult - try step debugging through a macro.
I don't think it's dream. I believe now most major languages support remote debug like that. My complain about Java debugging is massive use of AOP.
I should have added that I've had similar experiences with (old school, untyped) python codebases - absent the esoteric programmers, which is not a small difference - although with python you tend to hit performance issues if your call stacks get too deep. Ironically the JVM performance lets you dig yourself into a gnarlier pit before you have to face up to it.
Disagree about Java, I've worked on several large Spring applications and making small changes was not a problem at all.
Nope. I have taken over large Java applications with zero documentation and managed to get up to speed maintaining it fairly quickly. The static typing is a huge help.
Java is one of the very few languages out there that is manageable and debuggable at the large scales you're talking about. The tooling around it is second to none, and because it is statically typed with a decent type system, you can refactor quite safely in general.

And let's be frank, XML, other than for Maven (where it can make sense), is not really a thing in modern Java.

> decent type system

I'd call it a bare minimum type system, not "decent"

golang and C would take that spot :-)

Records, pattern matching, string templates, and more to come for Java for which the other two have nothing equivalent.

> The tooling around it is second to none

I feel like you've never looked at C's tooling.

Admittedly, some of it is stuff you just don't need in Java, but things like Valgrind are crazy impressive.

The issues that C faces in terms of memory correctness do not occur in Java. You can use tools like `perf` on a Java program, and I don't see why you couldn't use valgrind for natively compiled Java programs.

For debugging and observability, tell me if C (or any other platform) has anything resembling JFR for the JVM.

> and because it is statically typed with a decent type system, you can refactor quite safely in general.

I wish that would be true. Would make my current job a lot easier.

Not a clojure fan, but:

- dev churn is just a fact of life in this industry

- I'd love to know which language stops you from having 100k LOC codebase that nobody understands! Certainly not C#, typescript, Java...

- "esoteric developers" - that's a bit glass half empty! In my experience people who have taken the time to learn functional programming or any non top-10 language are usually pretty smart and motivated.

Agree that dev chrurn is a fact of life. But smart and motivated doesn’t imply not painting onself in a corner, it requires a different kind discipline which could be achieved with any language more or less. However, I find that the most frequent reason for churn is company culture of overwork and chaotic environment and in a place where ‘devs rule’ churn goes down drastically.
What worries me the most about clojure and languages like it, is that seems to be for people who want to be clever with how they write programs, when that is the opposite of what I want from people I work with.
Yeah. Personally, I really love writing Clojure. But I’ve never run it in production because I have the same reservations. I’d be the overly clever dev in this scenario.
As opposed to the paradigms of beautiful design that are Java codebases.
What does a bad java program have to do with a language who's advantage and use case is to be clever on an expression level?
Knowing how to design and structure maintainable code is a must for any successful code base to remain manageable over many years.

Clojure, in the hands of someone that knows this, works wonders. It gives your team 10x productivity, safer more correct code, that's simple to extend and grow over time, keeping your velocity high and your tech dept low.

But if you don't know how to leverage it's strengths, what best practices to follow, you can end up in a mess like that.

And this is true of all languages honestly. I've seen this for C++, Java, JavaScript, C#, Scala, all kind of code bases that suffered this same fate, with 100% team attrition, and all kind of other issues.

I say this from first hand experience, 6 years on a team that transitioned to Clojure, in that time the team rotated fully twice, during the pandemic it had 100% attrition, even I left (working on another team), but all the new developers, even though they have zero Clojure experience, had no issue understand the code base and start making improvements and building new features, velocity saw no impact whatsoever. You get the usual some of them would rather it be something their more familiar with, others are loving it, some don't care either way. But no complaints about the code base itself, no talks of spaghetti code, or challenges to debug anything, or understand what's going on.

Basically don't expect Clojure to save you from yourself, it's an asset when you know what you're doing, not a safeguard when you don't.

Of course everything you're complaining about means that the company has been successful enough to get to that point...

Startups that act too much like big companies just don't ever get product out the door and fail before you can bitch about technical debt...

Wow, I've finally found my tribe. All of this rings true for me. I kept hearing about how Clojure is this life-altering programming language that would make me forget all others and want to write parentheses for the rest of my life. But after working at a startup using Clojure for a couple of years, I've come to a couple conclusions about it: a) it is indeed a really nice, consistent language that's really nice to read and write, and b) the choice of programming language is a rounding error when determining whether a project or company is successful.
The same could be said about Ruby/Python/JS except for them not being esoteric. And they are wildly successful, even for new projects today.
> esoteric developers, who often get frustrated by the requirements of being a software engineer in a large company (i.e. everything that's not writing code)

And to the article's title, "building a startup on Clojure," you're better off building a startup on a customer base.

>>> full of functions that don't give the slightest hint of the shape of the data they're processing

This is a big problem with Dynamic languages. If you are new to the codebase, you have no idea of what the shape of the data is, what to expect etc.

> And the joy of working with an esoteric language is that it attracts esoteric developers, who often get frustrated by the requirements of being a software engineer in a large company (i.e. everything that's not writing code), which leads to the aforementioned 100% dev churn (after a lot of shouting).

Wow. Pretty insightful :)