[GMoromisato]

Programming is in tension between the Light Side and the Dark Side.

The Light Side is about preventing the programmer from making mistakes: Get rid of go-tos! Add static types! Do not allow a bug to be expressible. The Dark Side is about giving power to the programmer: Macros? Obviously. Operator overloading? Self-modifying code? Multi-line reg-exps? Go to town!

The Light Side knows programmers are flawed and imposes constraints. The Dark Side trusts programmers with power. Neither side is correct all of the time, and a good programmer learns both.

Lisp is interesting in that it is clearly Dark Side programming (the programmer can do anything) but it's still admired by Light Side programmers. Maybe there's something about the simplicity of the language that makes it seem platonic--almost incorruptible. Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.

[astrobe_]

I'll paraphrase someone who commented on HN once (about dynamic vs static typing, IIRC): permissive languages are enablers for solo programmers, but you need more restrictive languages for team programming.

This matches my experience because you don't chose who you work with. Skill level is uneven among the team. Less skilled co-workers will make mistakes that will have more consequences when the language is more permissive.

That's one of the reasons you want code reviews, but then you convert your skilled programmers to teachers. This is not a good trade, because the time they spend improving the skills of other programmers is partially wasted because programmers are free to come and go. AI could change this picture.

The name of the game is therefore to keep your teams small and skilled, but this is can be difficult because of typical company politics - e.g. "bus factor" considerations, shortening time-to-market is often achieved by adding more programmers.

[xedrac]

> Less skilled co-workers will make mistakes that will have more consequences when the language is more permissive.

Even the most skilled engineers are going to make these mistakes sometimes. That's the whole point of more restrictive tooling, and why Rust has exploded.

Don't get me wrong. I love Lisp, and use it in some personal projects. But I wouldn't want to use it in must not fail scenarios unless those situations can wait for a human to fix the running image.

[astrobe_]

Yes. I'm moving the goal post a bit here, but I was actually thinking about design mistakes and dangerous constructs - the kind of thing you do when you write in a "hacky" way - either deliberately or because you don't know better. Although these are not directly under the scope of type checking, a more restrictive language can have a positive influence.

[plumbees]

If the more skilled programmers become managers, then it's theoretically their job to become teachers, as their goal is to be force multipliers as opposed to people who can't delegate and accept a lower quality (temporarily) in exchange for higher throughput later.

[GMoromisato]

I agree with you. Though I wonder if the divide is more about the size of the program. I've got a couple of large (>100K LoC) solo projects and I like using C++ because it is much easier to catch bugs at compile time.

Static typing is a huge advantage in refactoring, and if you don't refactor a large program as it grows, you end up with a mess.

[drob518]

Clojure programmer here. Refactoring is the only time I miss static typing.

[oalae5niMiel7qu]

>This matches my experience because you don't chose who you work with.

That's what job interviews are supposed to be for. It should be easy to hire only the most skilled programmers, especially with the massive number of unemployed programmers out there right now.

[AlotOfReading]

Programming skill is similar to driving skill. You can hire the best people in the world and there'll still be incidents. The people who are most skilled may even be more incident-prone because they have the skills to take more risks.

Don't bet the company on no one ever making a mistake. Set up a system where the unintentional risks are minimized and the consequences of inevitable failures are mitigated by seatbelts and insurance policies.

[oalae5niMiel7qu]

Seat belts are one thing, a car with a top speed of 35 mph is something else entirely.

[3form]

Maybe it's trivial what I'm saying, but hiring the most skilled available person does not solve the skill disparity.

[dosisking]

Just look for the people who like Rust and filter them out

[Arubis]

I know this is obvious trolling, but it’s close to a truth. Selecting for a passion for languages that aren’t anyone’s first language gets you polyglots—folks that’ve had to abstract their knowledge and understand what’s an ecosystem feature vs what’s more broadly applicable.

Joking about Rust users rhymes with joking about vegans; it’s true both parties generally can’t keep from talking about their choice. But you do want to hire someone who’s thought about their options and made a decision.

[throwaway81523]

Lisp generally has precise GC, which I'd say makes it light side. It's even relatively type-safe if you count runtime type-checking.

Highly reliable systems are written in Erlang, which if you squint is another Lisp dialect. There's even a sexp-based version called LFE, for Lisp-flavored Erlang. Erlang's key to reliability is error recovery, rather than exceptional levels of error prevention.

I do like your light side/dark side classification.

[sph]

> Erlang, which if you squint is another Lisp dialect

Prolog disagrees. IIRC the first versions of Erlang were written in Prolog, and you can still see its influence in the syntax.

[derdi]

Neither use as an implementation language nor syntactic similarities make one language a dialect of another. Consider that Java is commonly implemented in C++, and its syntax was designed to be easy to learn for C++ users. Yet nobody calls it a dialect of C++, not even when squinting really hard.

Erlang's computation model is functional, like Lisp, unlike Prolog.

[pjmlp]

Correct, although I would assert Prolog shares many concepts with Lisp.

[drob518]

It does. In Prolog, data is code is data as well.

[gf000]

If you squint enough, JS is a lisp..

Define a goddamn language, syntax is not enough to define one! What are the semantics? Without that you are just talking about syntax trees like they would mean anything

[bsaul]

never understood why people say that: the syntax for defining code seems quite different from the syntax defining data structure. There's no homoiconicity in javascript..

[gf000]

And that's just syntax, it doesn't give you a programming language at all.

JS is a dynamically typed language with prototypical inheritance objects that work like universal key-value maps for the most part. It is also mutable.

Clojure is a dynamically typed language with key-value maps. It is also immutable.

You can surely see where I'm going , the underlying semantic model is the meaningful part. Homoiconicity doesn't give you anything special if your language can parse itself and can eval code. It just makes these completely abstract implementations simpler.

[zelphirkalt]

> Homoiconicity doesn't give you anything special if your language can parse itself and can eval code. It just makes these completely abstract implementations simpler.

Well, in a way it does give you something: By making expression of things like macros simpler, it makes them sometimes worthwhile, and makes it a reasonable request to have this kind of meta programming in your language at all. Without homoiconicity such things become even more difficult endeavors and often unjustifiable for the language design and its implementation.

[gf000]

I mentioned in other comment that homoiconicity doesn't necessarily make writing macros simpler. It makes writing trivial toy examples simpler.

But let's compare it to a modern macro system like rust's or scala's, where you get a typed object representation of the AST, and for anything non-trivial you are better off with this latter.

Also, arguably the best is to have certain features in the language itself, that can be used to build proper abstractions - so you don't have to resolve to using macros in its place.

[throwaway81523]

> If you squint enough, JS is a lisp..

Yes, that is true. I'm not big on the idea that Lisp is defined by parentheses. The implementation strategies are another way to look at it. That doesn't capture it either, but it's an angle to try.

[pjmlp]

It isn't, Dylan and Julia are two Lisps with Algol like syntax.

Indeed there was a proposal to add Algol like syntax to Lisp,

https://en.wikipedia.org/wiki/LISP_2

[gf000]

So then what it is? Because otherwise it's a magical nothing-term to which everything lisp people like applies, but no criticism can ever reach it because "that's not really lisp, see it's different in this other implementation"

[Pay08]

In my mind, it's 2 distinct criteria: interactivity and the manipulation of symbols (whether those are implemented as symbols or identifiers doesn't really matter). I don't know about Erlang, but from the admittedly little JS I've written, I believe it achieves both criteria, even if it's worse at them than a "proper" Lisp.

[dosisking]

JS is Self, which is a dialect of Smalltalk. So not really Lisp imo

[adamddev1]

I think the big thing is that Lisp is (aside from mutable variable assignment) basically all declarative, rather than the imperative paradigm.

Even without static types, and even allowing macro craziness, there's just such a stronger baseline of declarative and functional thinking, you're off to such a good start in clearer thinking and reasoning about a program.

[ludston]

This is just not true. The Lisp family contains all conceivable variants of languages, from statically typed Coalton, to dynamically typed Scheme. From functional, immutable by default Clojure to the imperative-style of Common Lisp.

There are prologs, constraint solvers, ffis, JavaScript alternatives, garbage collected and not garbage collected languages that call themselves lisps.

[agumonkey]

I won't disagree but most books and people who made lisp in the early decades were somehow hinting at not manipulating state but trying to represent ideas and operators to raise the level of expressiveness without shooting yourself in the foot (as much as possible)

[enfield-argent]

I think this depends on the Lisp, no? AFAIK Common Lisp supports a lot of imperative style programming--besides all the mutation/assignment functions, there's the `prog` macro that lets you use goto, `do`/`do*` to iterate over groups of statements, or even the `loop` macro. OTOH the Scheme-style Lisps are much more declarative thanks to TCO and a community that prefers the functional/declarative programming style.

But again, I suppose all the Lisp forms return values, and quibbling about the declarative : imperative :: expressions : statements mapping is just petty semantics

[lgas]

> But again, I suppose all the Lisp forms return values, and quibbling about the declarative : imperative :: expressions : statements mapping is just petty semantics

I would argue that it's very much not just semantics, or at the very least it's certainly not petty. The distinction has a noticeable effect on what the experience of writing code and on the reliability and related properties of the written code.

[frwrfwrfeefwf]

Yes and also how you use it. The more you squeeze performance out of CL the closer the code looks to C in s-expression format, the less you squeeze it the more it looks like a DSL. The bridge between the two is macros so that you can have both the performance and expressiveness.

[patrickmay]

I like your metaphor but I'd argue that trusting programmers with power is the Light Side.

Relevant PG: https://paulgraham.com/langdes.html

[GMoromisato]

That's a great essay--thanks.

And you have a good point. Maybe it's a weakness in the metaphor if one can see it either way.

I fundamentally believe that there is no perfect language, instead one merely chooses a set of trade-offs. What's interesting about Lisp is that it is simultaneously well-respected and yet not widely used in productions (compared to C, C++, Python, JavaScript, and even Rust).

It's almost like Lisp chose a particularly extreme set of trade-offs that yield great power but also repel large swathes of programmers. I think PG might say that it repels dumb programmers, but maybe not.

[oalae5niMiel7qu]

Our entire profession has been moving more and more in the "bondage-and-discipline" direction over the past 10 years or so, tracking with broader sociological trends that are also justified with arguments involving "safety". That by itself can explain why Lisp has such low adoption. While Lisp has never been the most-popular language, it was never as unpopular as it is today. If that pendulum swings the other way, Lisp will get more popular again.

[solumunus]

I think it’s just that the syntax is more difficult to grok. When I first saw code (PHP and JS) it was fairly easy to read and figure out what was going on. I’m now experienced but I’ve never tried Lisp, but reading Lisp code takes considerably more effort than when I first saw the PHP/JS. I’m sure with very little experience it becomes just as easy, but to the novel eye I think it’s simply less intuitive to reason about.

[akkartik]

You mixed up Light and Dark. Creator-knows-best bondage-and-discipline is the Dark side. Trusting people with power is the Light side.

[sph]

Light and dark is propaganda depending on which side you're on.

I'm with you, on the dark side I've seen people slowly turn into crabs, screeching about memory safety and static types. Can you imagine that? Now stick those wings on me, I'm going to fly close to the sun.

[bryanrasmussen]

Programming is in tension between the Light Side and the Dark Side.

The Light Side believes in giving people freedom and power to achieve freedom, to allow themselves to express themselves better through programming, to become better programmers through the tools the light side provides; in the Light Side believes in programming as a way of thinking.

The Dark Side believes that people are incapable of handling freedom and must be restricted and controlled under the service of various organizations - generally companies. The companies will control access to programming and control the programmers via this access. The dark side does not believe in making programmers better programmers, it believes in managing programmers. The dark side does not believe in helping programmers to think, it believes in thinking for them.

[rogue7]

Kind of agree, but in my view preventing the programmer to make mistakes is futile. I have seen awful stuff in languages made to prevent errors.

It's much better to give all the power to the programmer, to allow him to fix his mistakes rather than fantasising about preventing them.

[pfdietz]

I'm not big on static type checking in most situations -- adequate testing should find the type errors too -- but one place it looks very useful is avoiding data races in multithreaded programs. See Rust.

[bigpeopleareold]

... or Ada.

[yogsototh]

I wonder what side you would put Malbolge,

What side for python? Haskell? Idris? Clean? C? K?...

white/dark side is too much of a simplification to me.

Programming language equilibrium is a very complex subject and fascinating one.

After years of experimenting with many of them, it is all a matter of context of usage, personal preferences, objectives, etc.. there is no such thing as "the best programming language ever", neither "best for light side programming" nor "best for dark side programming".

Similarly, there is no such things as: Light side/Dark side in artistic painting. There are tons of paints some are clearly greater than others, but there is no such thing as "the best painting most beautiful for everyone seeing this paint comparatively to any pain".

[^Malbolge]: https://en.wikipedia.org/wiki/Malbolge

[^Clean]: https://en.wikipedia.org/wiki/Clean_(programming_language)

[^K]: https://en.wikipedia.org/wiki/K_(programming_language)

[eimrine]

Malbolge is not an instrument for building, come on programmer.

C is clearly the dark side with to chance to protest.

The only way to consider K as dark side is its speed constraints relative to my favorite PL which is J. K is so fast because its interpreter fits into CPU cache.

Haskell is a border case between sides because the idea of PL is so light that using so much light in programming leans to be really dark. Haskell extracts the cognitive friction from the code environment to the types environment, programming in paper is not a joke for Haskell.

[dosisking]

Python is definitely Light Side

[drannex]

If there is any language that has ever been, Python is the very definition of light side, so much so that anything that needs to be done in the cover of night, is purposefully directed elsewhere by rather shady dark siders (C).

[busfahrer]

> Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.

This isn't so far off, considering that some people consider the "Lisp in Lisp" bit from the 1.5 manual to be the "Maxwell's Equations in Software":

https://michaelnielsen.org/ddi/lisp-as-the-maxwells-equation...

[DonaldFisk]

I have always assumed that Alan Kay compared Lisp to Maxwell's Equations because of the similarity between the interplay of eval and apply in Lisp on the one hand, and the interplay of the electric and magnetic fields in Maxwell's equations on the other.

[agumonkey]

Also because lisp core can be reduced to a few "equations" (at least that was my understanding)

[GMoromisato]

Maybe machine language is the formless chaos and Lisp is the purest manifestation of order. All other languages are points in between.

[gf000]

That makes no sense without specifying what Lisp actually is.

S-expressions are a generic tree, that's not a language, that's just syntax.

Is it CL, Clojure, scheme?

Like these don't even share a basic object model, it's like getting the AST of Haskell and C and then talking about how good that AST is.

[throwaway81523]

See Cardelli's breakdown in his famous article "Typeful programming":

http://www.lucacardelli.name/Papers/TypefulProg.pdf

Page 52 perhaps.

[sdevonoes]

But there are two types of “programming”: 1) programming per se (just you writing code for fun and perhaps building your own tools) and 2) a team of people with real-world constraints writing software for others for revenue

It’s clear why it feels so good to use something like Lisp for personal projects. It’s also clear why the vast majority of companies don’t use it

[oalae5niMiel7qu]

You have it inverted. Your "Light Side" is what previous generations of programmers called "Bondage and Discipline", because this philosophy assumes that a good programming language has built-in shackles to impose the will of a so-called "benevolent dictator" on all programmers.

[tikhonj]

It isn't nearly as much of a trade-off as people say. Languages like Haskell are both remarkably expressive and provide a lot of safety. (And, of course, languages like Python, Java and Go are the opposite.)

[GMoromisato]

Haskell is currently at #18 in LangPop: https://langpop.com/rankings

Almost by definition that implies that it makes some trade-offs that turn off lots of programmers. Still more popular than Lisp, though.

[jimbokun]

Language design has almost nothing to do with language popularity.

People learn JavaScript or TypeScript because they want to write web apps. Swift or Objective C to write iOS or Mac apps. SQL because there’s a database they need to get data out of. Python because there’s a machine learning library they need to use. Etc etc.

Language design is far down the list of priorities.

[GMoromisato]

That's fair, but I was responding to the claim that you don't need to make a trade off. If that were true, then Haskell would be much more popular.

Clearly there's a trade off or else everyone would just use Haskell.

[tikhonj]

That doesn't follow.

For one, there might be trade-offs against dimensions other than safety and expressiveness. Performance, corporate support, libraries, popularity, learning curves, similarity to other languages...

For two, popularity is simply not strong correlated to quality. Popularity is a social function driven by, well, social factors. It's heavily path-dependent and noisy. There is absolutely no guarantee, not even close, that the "best" thing in any sense will be the most popular, or popular at all.

[psd1]

I would refine that: Language design has only indirect relevance to language popularity.

People go where the money is, hence python. But python is widespread because there is employment in it. That adoption by employers follows a network effect, but it originally came about through the language design.

Early-adopter => mainstream => long tail

Adoption hinges on that jump from evangelists to the mass market, and that implies:

- attracts early adopters because of language design - early adopters can pitch the language to their bosses by showing cleat benefits, also because of language design

The most obvious exception is javascript, which was inflicted on the world by browsers. Even so, it had to be sufficiently shit for the php crowd to feel at home. (Unsure which came first, but you get my point.)

Php is exactly popular due to design. It was an exact fit for the myspace dotcom era, and there were no json APIs to illuminate its shonkiness. (I am still angry that [ ] serialises to either [ ] or { } depending on a fucking global variable. I cannot think, being maximally charitable, of any value that comes from conflating arrays and hashmaps. No, i do not want an array containing the keys 0 and "0". I can count on the fingers of one foot the times i have wanted that.)

Sorry - my therapist encourages me to rant about php. She said it's part of healing. I haven't even told her about back-end php yet.

[oalae5niMiel7qu]

When Python got its popularity, programming was an in-demand skill, so the thing that determined adoption was being easy for beginners to learn. Python was extremely beginner-friendly, so it became popular.

Now, the thing that determines popularity is popularity. So Python still wins, even though its beginner friendliness isn't so important anymore.

[Skwid]

I've never much luck selling a language on cleat benefits alone, but I guess there's not many climbers in my department.

[psd1]

Those are track cleats, because sports metaphors drive middle management

[ludston]

That argument seems crazy to me. 18th is so good. There are literally thousands of programming languages that it is competing with.

[lgas]

It may very well be the case that it makes trade-offs that turn of lots of programmers but I don't think it's popularity directly implies it by definition, rather just that it implies there's something about it that prevents lots of programmers from taking it up en masse, and I think that's perhaps just the fact that it is so different from what everyone's used to and it takes some time to adapt to it enough to be able to appreciate the tradeoffs.

[Barrin92]

for a while there's been a strong cultural tendency in programming to mistrust people and trust tools (often even blindly because tools are just assumed to make no mistakes), and so expressiveness has been sacrificed for safety. There can be something to this but it's also self-fulfilling, if you strip people of agency of course they'll unlearn to program.

I've always thought it's a misanthropic philosophy and sucked much joy out of programming but also there's something to be said that there's safety in the expressiveness of Lisp. I read an article a while ago that found a robust correlation simply between length of a code base and errors regardless of languages used. And given how succinct and clear Lisp codebases can be that's valuable in itself.

we're kind of at the logical endpoint of this now with gigantic slop codebases that nobody understands just held together by 20 different tools, and if you ask me if I had to pick one of those or something one tenth the size written by a guy or girl who has been writing Lisp for ten years I'd say thank you I'll go with #2

[coliveira]

The issue is also social, the average software engineer will treat Lisp code as a liability because most people really don't know what to do with it. In the mind of a professional SE, code is "maintainable" only if it's writing in Javascript or some other commercial language.

[skydhash]

Yep, you have to use C# and Java because they are safe. If you want anything extra, you need to use $library, because only $library's maintainer is trusted with the dark corners of the language. Look how powerful we are, millions of libraries. Look at those Lisp guys and their sorry examples of package managers. /s

If there's one thing I've been happy with learning Common Lisp and Clojure is that you only want libraries for things like algorithms, data formats, and protocols/interfaces. Mostly because those often have standards. Anything else should be a snippet to copypaste even when presented as a full program.

[tyeaglet]

Only if you knew the power of the Lisp side…

[malkia]

Hah, didn't think of it this way... To me it was if it allows live updates (+support for them) or not :)

[sdfsfdsdfs3d]

> Do not allow a bug to be expressible

That is a very powerful idea, but unfortunately it cannot be realized with a fixed set of language rules. Some invariants are universal, but some are bound to the domain. Some projects require, say, an int (EvenInt) to take on only even numbers and others odds (OddInt). You can imagine the possibilities here are unbounded ("the numbers should be prefixed by numbers that are divisible by my age squared").

Ideally your base language has the expressive power to formulate new abstractions and constraints. This fundamentally requires the Dark Side. Once the proper primitives are in place a different "language" - this can be literally or figuratively - is used to express the interplay between those primitives.

In effect you are starting with the most general (say, all of Lisp), restrict it to become more and more and specific - only "these modules" - until it cannot be reduced any further. If you can bring your domain down into being expressible as, say, a single config file, that'd be quite ideal. If you don't need the powers of abstraction to express your solution then exposing said powers would only invite in trouble. The mathematical equivalent of introducing unnecessary variables.

As a programmer you are free to traverse the journey from say all of Lisp to a JSON config file in whatever way you please.

The "Light Side" people have converged, or try to converge, on some intermediate state between full powers of abstraction and configuration only. I think this is useful because many if not all problems travel through that intermediate landscape on the way "down" (into their specificity). For example what you call "types" is a significant restriction on your freedom but at the same time it's so enormously general that this restriction can be considered a worthwhile default because just about any problem I can think of can potentially benefit from that restriction.

All this is to say that I don't think it's a dichotomy so much as two interacting polarities whose interplay gives each its strength.

Lisp is a particularly minimal base introducing very few restrictions of its own. But it's not the only one. Forth would be one in another direction.

[psd1]

Well put. Minor quibble: I see what you're driving at with "inexpressibility" but i think i could write EvenInt and OddInt in F#. I don't know Haskell, but i believe you would have to work hard to find a domain constraint that you can't declare in the Haskell type system.

[sdfsfdsdfs3d]

You are, of course, right about EvenInt being easy to express in a type system. In fact, I believe type systems are generally so expressive as to allow just about any constraint to be expressed (eventually..).

I notice a lot of arguments on this topic resolve to "but Turing", which is not completely uncalled for but I think misses the point a bit. Not because it is wrong, but because it highlights the wrong property.

I don't doubt, say, Brainfucks ability to express any arbitrary computation, but I do doubt its ability to do so sanely. Now you may say Haskell's type system is very clean, but some constraints will definitely push it out of its comfort zone. I'm not saying it won't be able to express them, but I am claiming there will be dragons. One example of a constraint that's at least awkward is when the absence of a fixed type is part of the design. There are of course myriad solutions to this problem but they generally all require not quite so straightforward constructions that eventually might make sense with enough exposure, but whose complexity can actually be disproportionate to the value of the guarantee.

It's very much a testament to the genius of languages like F# and Haskell that you have to think hard of practical counter-examples.

That said, I actually think the Dark/Light polarity rears its head again even deep inside Haskell as any sufficiently complicated software system encounters barriers it needs to overcome and those require carefully constructed escape hatches: unsafePerformIO, metaprogramming, that is to say, the (relative) Dark Side.

In that metaphor undisciplined use of Lisp is like a vast, dark gravitational field of possibility and Haskell is like a sea of light with some carefully marked dark patches.

[frwrfwrfeefwf]

you understand. It's also endlessly funny to me it requires anonymous accounts to speak the truth about programming languages.

[dreamcompiler]

I think of C as more of a dark side language because it's basically a portable assembler. Lisp at least prevents you from doing the stupidest things like overrunning array bounds or writing to freed pointers*, which C is perfectly happy to let you do.

* Of course there are no user-visible pointers** in Lisp so the concept of free vs. not free doesn't come up because memory allocation and garbage collection are automatic.

**Under the hood most everything in Lisp is a pointer, but the user cannot see them or mess with them.

[James_K]

I dislike when people use the term “power” to describe making code slightly shorter to type. Operator overloading is not in any meaningful sense more powerful than a language which lacks it.

[adrian_b]

You say that because you lack experience in writing programs for scientific-technical computing, where an abundance of complicated mathematical formulae are necessary.

Most programmers have seen only programs that do more data movement and comparison than actual computation, but nonetheless there are programs with abundant computations, the kind of programs that were originally written in Fortran, but nowadays they may be ported to other languages, or alternatives for them may be now written in more modern languages.

In such programs, overloaded operators do not make programs "slightly shorter to type", but they reduce the amount of text at least an order of magnitude or even much more.

The main advantage is not that you type less, but that you can read the source of some function in one page, so you can see it in its entirety, instead of having the source spread on many pages, forcing you to wander through all those pages continuously, when you try to understand what it does and whether it does it correctly.

I have written such programs, for instance which were full of long formulae where most variables were complex vectors or complex matrices (for computing some radiated electromagnetic fields with the method of boundary elements). Without the operator overloading of C++, reading them would have been extremely tedious and maintaining them would have been very error prone.

For myself, any programming language that lacks operator overloading, e.g. Java, is disqualified, because it definitely is not "powerful" enough.

Operator overloading also increases the safety of a programming language, because you can define distinct data types for each kind of physical quantity, allowing the compiler to detect and reject the invalid operations. Without operator overloading that would inflate too much the size of the source text.

[ux266478]

> where an abundance of complicated mathematical formulae are necessary

I think what you mean to say is, a lot of linear algebra is involved. So multi-dimensional structures like matrices and vectors are expected to have linear arithmetic operators defined over them, because they're used quite a lot.

You don't need operator overloading for that. Your language should be providing them as primitive structures and extending operators over them. Think like how C has a polymorphic addition, where '+' doesn't discriminate between integers, pointers and floats.

There are a lot of problems with user-facing operator overloading, but I think Haskell is actually a really great example of how to do it sanely. At the point in which you have type-families, you can properly restrict the semantic domain of a polymorphic function but still keep it an open set. The addition operator being restricted over types belonging to the Num type-family as an example trivially allows you to use '+' for matrix addition.

https://hackage-content.haskell.org/package/matrix-0.3.6.4/d...

Now you've avoided the worst part of user-facing operator overloading (juniors creating incoherent DSLs by abusing the mechanism) while still providing the mechanism where it's useful.

[gf000]

I agree with you on overloading - the real word here should be "expressivity", which is defined by language constructs that could NOT be syntactic sugared into a local context. Those may occasionally be useful, though not by itself. E.g. gotos are more expressive by the above definition, yet it was largely deemed to not be a useful "power".

[adrian_b]

Certain kinds of GOTOs are indeed proven mathematically to be necessary to write certain programs, in the sense that eliminating the GOTOs can be done only by making the program more inefficient, i.e. by increasing the number of executed instructions and the amount of memory that is used.

Because of this, any decent programming language must include GOTOs, but in many modern languages they are not named GOTOs, because of the bad reputation of the word.

For instance the language Scheme does not have GOTOs, but it has mandatory tail call optimization, which means that a "tail call" is just an alternative name for "GOTO". For example, this allows the writing of a state machine in Scheme, exactly like it would be written in a language with GOTO, but using tail calls instead of GOTOs.

Other languages have labelled loops and they allow exit a.k.a. break with a label and next a.k.a. cycle a.k.a. continue with a label. Such instructions with labelled targets are just GOTOs with a bad placement of the label, which makes reading the source more tedious than with a classic GOTO.

For providing the benefits of GOTO, a restricted variant is sufficient, i.e. a GOTO that may jump only forwards and which cannot jump inside nested blocks.

[never_inline]

With operator overloading you can provide same numeric operation interface to library types such as big integers. Sure, you can also define methods and make sure your primitive types also define those methods but now you have some confusions (like equals vs == in Java).

[GMoromisato]

Maybe you've never been in a codebase that overloads the comma operator.

[user3939382]

Check out Clojure spec. The light dark bifurcation doesn’t hold up.

[agumonkey]

it's dark side-ish at the ast level mostly, which is not the same as the dark side of mutable state over byte arrays

[frwrfwrfeefwf]

Typical ignorant take. Lisp macros allow you to create restricted DSLs to prevent bugs from being expressible, including full static type and structured programming systems. Basically no one understands this until they reach the 'lisp enlightenment', so the decades of beating this drum falls on deaf ears. AI is here now so it's time to let it all go.

[GMoromisato]

Of course you feel this way. You just finished reading SICP or maybe you binged PG essays last weekend. But eventually you'll read Simon Peyton Jones and start screaming about functional programming and algebraic type systems. Then, if you're lucky, you'll get a real job and realize that languages are just a tiny part of software engineering.

I'm excited for you to experience that journey.

[throwaway81523]

Wadler's old article comparing Miranda (a Haskell forebear) to Scheme might be a better place to start.

https://www.cs.kent.ac.uk/people/staff/dat/miranda/wadler87....

[jimbokun]

This was a very pleasant read.

[mindslight]

Do you like Apple ]['s?

You've got a good point. Code generally needs to be read more than it is written. At the very least, you realize this as you try to revisit something you had done previously and are left scratching your head. Encapsulating the complexity in a custom DSL can be great for simplifying all the code you write in the DSL. But it also raises the stakes - if you ever have to revisit the DSL implementation the complexity is there lurking.

[frwrfwrfeefwf]

cute but you don't get lisp yet.

[deterministic]

I would be very interest in seeing how "getting lisp" enables you to write software that is more successful than the C and C++ software that runs the world. Perhaps you have written software in Lisp demonstrating this? Something you can show us?

[__patchbit__]

The reference text on time keeping used to be implemented in Lisp.

Alan Kay says a lot of problems with C/C++ go away when using the higher math more easily expressible in Lisp and like minded languages.

Jürgen Schmidhuber says the problem with computers from a mathematician's point of view are the numbers.

[deterministic]

> Alan Kay says a lot of problems with C/C++ go away when using the higher math more easily expressible in Lisp and like minded languages

That doesn't match my experience at all. Math and numbers are in general not a problem except when (say) implementing triangle hit detection in a game (something I did) where the precision of 32 bit floats does matter.

> Jürgen Schmidhuber says the problem with computers from a mathematician's point of view are the numbers.

That is probably true if you are a mathematician. Most people aren't.

Having said that, LEAN (for example) is written in C++ and used heavily by mathematicians.

[anthk]

Go ate a big lunch from Java, C and a bit of C++ as a systems' language, which Go was designed as "C sucessor without the C++ bullshit". Even the most modern C (Plan9/9front, not what the ANSI C comitee vomits in every iteration) it's a very different beast.

Still, Common Lisp it's in places where trying to build such kind of software in C/C++ would be a reciper for disasters.

https://www.lispworks.com/success-stories/index.html

[deterministic]

> Common Lisp it's in places where trying to build such kind of software in C/C++ would be a reciper for disasters.

I personally never blame the tools. I have written fairly complex software in 68000 assembler and it (1) worked fine and (2) was highly maintainable. However it was of course much slower to modify than using a higher level language.

[jake_and_fatman]

I see what you did there.

"Of course that's your contention. You just got finished readin' some Marxian historian -- Pete Garrison probably. You're gonna be convinced of that 'til next month when you get to James Lemon, and then you're gonna be talkin' about how the economies of Virginia and Pennsylvania were entrepreneurial and capitalist way back in 1740. That's gonna last until next year -- you're gonna be in here regurgitating Gordon Wood, talkin' about, you know, the Pre-revolutionary utopia and the capital-forming effects of military mobilization." Good Will Hunting (1997)

[ux266478]

In an awkward, error-prone, non-mathematically rigorous and easily subverted manner. That whole "make invalid states unrepresentable" thing relies upon the fact the semantic structures you're doing it with can't be shadowed silently by other parts of the code, and the safety constraint itself is guaranteed to be completely pure and deterministic. Lisp macros and reader macros fail at that criteria. They aren't any different from any other collection of procedures and their collective interface. Actually, they're a fair bit worse since the natural cognitive overhead from juggling multiple layers of evaluative indirection lends itself to blindspots of edge cases and unsoundness.

I think "macros are a safety mechanism" might qualify for the top 3 "new lisper mania" things I've ever read.

[frwrfwrfeefwf]

It can be made as mathematically rigorous as anything computable, and as non-awkward as is possible to represent. You either reify the invariants and semantics as statically analysed syntax or they are implicit in your collections of procedures and objects which is far more cognitive load. If you need to judge multiple levels of abstraction while using the DSL then the DSL is the wrong fit for the problem. There is no 'evaluative indirection' because a macro is a _compiler_ and the underlying code can be extremely alien to the DSL semantics.

Any argument against macros must be levelled against compilers in general, as _they are the same thing_. Rust is compiler as a safety mechanism, not different from macros as a safety mechanism.

[gf000]

Again, fallacy by Turing completeness.

It's actually harder to make something not Turing complete - in and of itself this property is absolutely useless and tells you nothing about how practical something is.

[frwrfwrfeefwf]

yes it's desirable to use non turing complete languages as less shit can go wrong, which is why you need macros. You seem to over estimate the difficulty of writing compiler macros, remember you only need to go from the DSL -> Lisp, not lower it into machine code. The DSL's compose perfectly as well, lower one to another or combination of them + lisp.

[deterministic]

I use plenty of DSL's in my daily work maintaining 1 million+ lines of C++ code used by very large companies around the world. No macros needed or wanted. More than 90% of the production code is generated from DSL's.

Outside of C++, Haskell has many great examples of powerful DSL's implemented without macros.

The problem with macros is the compile/run time cost. I could have implemented those DSL's using C++ templates (which are Turing complete by the way) but I prefer not to.

[gf000]

Well, compilers have layers - you ain't outputting assembly from the AST layer directly, usually there is some kind of IR. Like LLVM have plenty of layers.

And for a simple DSL you can usually just use a sufficiently expressive language, no need for macros. Like kotlin/Scala has html DSLs that look pretty decent.

For anything more complex though, you can't just "compose stuff", a type system is a global property so for that you literally have to write something way more complex than a local macro should contain.

[ux266478]

The inflection of the copula "can be" is exactly the problem, as is the 'anything computable'. There's a reason Isabelle is used as a proof assistant, and Prolog isn't. The former guarantees sound, mathematical rigor that can't be trivially violated. The latter doesn't. Prolog is much more powerful and flexible than Isabelle. Power and flexibility are disastrous for safety. As someone else said elsewhere in this thread, safety is about restrictions.

In this case, it actually runs deeper than this, there are structural issues which prevent them from being sound right from the beginning. It's the mere fact that the whole behavior of your DSL can be silently changed without touching the module it's defined in, or the file it's being imported into. A well-meaning junior, in a completely different department, can completely destroy your invariants in a completely unrelated part of the codebase. Whether by shadowing runtime functions, or a package-level collision that exists upstream. Even Scheme's hygienic macros don't actually solve this, they just make it less likely. At the point in which you have no behavioral guarantees of semantics, you do not have a safety mechanism at all.

> There is no 'evaluative indirection' because a macro is a _compiler_ and the underlying code can be extremely alien to the DSL semantics.

This is a trivial contradiction. That is precisely evaluative indirection. Yes, stacking multiple compilers on top of each other, especially when their semantics are non-isomorphic, is a massive burden of mental overhead. Given you already have this problem with the mapping between Lisp and binaries, adding even more complicated layers on top of it is a recipe for disaster. Particularly because: you are going to make mistakes in your logic. Macros provide no means to stop you from blowing your own leg off when writing them. That is not what they are designed for.

Finally, you cannot escape the awkwardness of the underlying metaprogramming system, you will always have to wrestle with quotation and quasiquotation. You have to define a DSL before you can use it.

Only at the point in which you treat and use Lisp like an unserious toy, rather than the powerful tool that it is, can the fantasy of "macros are a safety mechanism" begin to make sense.

[gf000]

This is the "fallacy by Turing completeness".

Sure, and you may as well write full systems in Minecraft and create your own type system inside and whatnot.

Macros can make some small specific uses safe, indeed. But they are not comparable to languages with type systems where everything is type safe, unless you literally have written a new type system and compiler to begin with. Which is trivially true in every other language, C can be 100% safe with a much better type system, I just read a text block and compile it as rust code! Wow!

[sdfsfdsdfs3d]

I have no horses in this race and I'm sorry if I misunderstand but I don't think he means to say Lisp is by nature superior to say Haskell. I take it as a statement on the generality inherent in its design. It is by its very nature very low on restrictions. Which is to say your freedom of expression is very nearly as complete as it'll ever be.

Sure you can beat Minecraft into Lisp and then Lisp into Haskell - as Turing made sure of - but you'll be battling a lot of dragons along the way. It's beautiful that you can beat Haskell into letting go of its type system by, say, creating a C compiler in it. It actually still amazes me the universe has this property.

In general I see "building" (I rather call it "conjuring") a system as going from the most general (all of your programming "language") to the specific (your solution). I can see how the most general of languages makes for a comfortable starting point of that downward journey.

I can also see how not starting from The Universe In All Its Infinity Glory (Lisp) but from The Planet Earth (say, Haskell) is helpful if you want to conjure something made for humans, but if you want to go beyond the mortal plane.. (I'm not saying that's a good idea by the way).

[gf000]

My issue is that a language is just as much about what it disallows, as much as about what it allows.

It's absolutely trivial to allow everything, you just have to make a Turing-complete extension either by design or accidentally.

It's much much harder to carefully choose more limiting primitives that only allow you to build stuff that keep certain important properties about them. You can't subtract stuff after the fact.

E.g. something like rust's borrow checker could not be realistically implemented as "a lisp" macro (what does lisp even mean in this case). For that to work you would literally build a rust compiler over s-expressions and you might as well leave off the s-expressions at that point.

And every part that was not compiled by that macro or whatever would be unsafe and could not interact with the "rust-lisp" compiled parts, as you can no longer assert anything about them (there is nothing mandating the usual single writer, xor multiple read-only references laws)

[sdfsfdsdfs3d]

I agree. Restrictions are core to software development. In fact without them I don't think there is development in the first place. As you know I see software development as going from the general (the full bandwidth of the computational substrate available to you) to the specific (the absolute minimum - if any - computational structure you need to get what you're after).

Rust is more restrictive than Lisp. These restrictions are sane and productive for most general software development in 2026. They form an excellent base to start your work in this day and age. Starting with raw Lisp feels like starting with the general notion of mobility instead of just getting in a car. The car is not inferior at all. It's a set of restrictions on the most general notion of mobility that in fact buy you a lot of comfort like knowing it doesn't need to eat hay or take a poo in the street. The freedom they take away, they give back in strong, useful guarantees about stuff you actually care about ("getting places").

I can see how Rust, Haskell or whatever provide a set of comfortable, general-enough restrictions to make life for the common developer easier and more productive.

This being HN I do want to allow myself some contrarianism and point out that finding the most minimal, yet most general computational substrate is not at all trivial. It's in some ways equivalent to finding the most minimal set of invariants ("laws of physics") that can describe the most wide range of phenomena ("nature"). Which is to say I find it pretty damn impressive. So I only "disagree" with this:

> It's absolutely trivial to allow everything

I don't disagree because it is wrong, it's clearly not. Turing shows all languages collapse into one when it comes to computational power and I don't doubt that. But I do know there is a distance between the computational substrate, the primitives of the language, and the structures it expresses. Let me give an example.

A spreadsheet can represent a computer game. A game engine can represent a spreadsheet. But if you build Excel inside Unity you haven't made Unity spreadsheet-like at the substrate level; you have encoded spreadsheet semantics in Unity's primitives. Functionally they may be equivalent, and that equivalence is beautiful and Turing can tell you all about it, but the relationship between the native primitives and the concepts being expressed is different.

Turing makes building endless towers that are functionally equivalent possible, but what I mean is that there is an architectural difference between a system directly built out of the least semantically committed substrate available to it ("native") or "embedded" in simulations of substrates built on top of those primitives.

I think Lisp is close to the "least semantically committed substrate" you'll ever come across in practice. It's certainly not the only one, but it's a particularly clean and relatively practical one. One can say, for example, raw lambda calculus is equally uncommitted but it's kind of .. hairy. Lisp's primitives are basically the machinery of symbolic manipulation itself. It's bloody amazing.

Sorry for the verbiage. I went way overboard and meandered into foggy, mystical meadows full of mysterious entities so feel free to ignore.

Have a nice weekend!

[gf000]

Thank you for the reply, and I definitely agree with you on many points. I absolutely don't want to take away from Lisp that it is somehow uniquely elegant, and does have a mathematical beauty to it, this is certain.

I just feel this (and its practical ramifications) are often overexaggerated.

[wild_egg]

> unless you literally have written a new type system and compiler

This is entirely possible with plain lisp macros. See https://coalton-lang.github.io

Having that natively available as plain Common Lisp code is a very different thing from your "just read a text block and compile it as rust code!" concept. Or at least, the tool chain gymnastics required to make a C program emit its own source in another language... Why?

[gf000]

Well, I was talking about the fundamental part. Of course you can have more practical implementations, like Scala or rust's macro system, that gives you a proper typed AST.

[frwrfwrfeefwf]

Type safety is a domain specific concern, 99% of the time your domain will not have 'types', often it won't have notions of things at all. But any kind of type safety can be added at any point with macros. Something like rust would be a huge project but it doesn't get easier than macros, see Coalton, and they compose with one another. When you run into the same requirements in another language you are simply stuck source pre-processing. There is no stuck in lisp.

[gf000]

There are plenty of languages with macros though, nothing special about lisps.

And no, language semantics absolutely don't compose. Like you can't just do some kind of optimization in one place if you do some mutation on it in another place. Optimizations work on global assumptions that every part of the codebase have to abide by. Any part not doing it will make the whole thing crumble - and "there is no stuck in lisp" is exactly why you can't have your cake and eat it too.