[kazinator]

> Python defines the lower bound on the code readability

Ignoring ;... comments for a moment, if we squash a Lisp program into one line and remove all non-essential whitespace, it's possible to recover it into nicely formatted code by machine, more or less.

> TLDR: Lisps - Schemes, Racket, CL, PicoLisp, Emacs and TXR Lisps to name a few - can be used to write astonishingly readable and to-the-point code, but the languages do absolutely nothing to discourage using them to write the most unreadable mess under the heavens. As for the reasons for this - I've honestly no idea at all.

This view is unbalanced without noting that Javascript, Rust, C, Perl, Java, Scala, Go, Kotlin, ... and a large number of other languages, have the flexible formatting that allows for unreadable code. Ruby, anyone? https://github.com/mame/quine-relay/blob/master/QR.rb

> As for the reasons for this - I've honestly no idea at all.

Bad formatting is a bug that is fixable in the actual code (greatly assisted by automation) and a minor social problem in programming that is treatable with education and experience. Therefore, it is nearly a non-issue.

[klibertp]

You're right, of course, on all points (BTW, WTF is with the downvotes??), but they are technicalities of interest to lispers. I omitted these because I wanted to present a convincing argument that fnord123 simply had bad luck and encountered bad Lisp code. And that there's a lot of good, readable code written in s-exps out there.

The resistance to s-exps in the general population of programmers is bad "news" (if something 50 years old can be called that...) for Lisps. It's hard to fight it in general terms. Pointing out that C, JS, PERL, etc. are often much worse in terms of readability - while obviously true - doesn't really help in convincing someone to look at s-exps differently. This is why I chose Python for comparison and tried to present a positive argument, saying that you can write code "even more readable than Python at its best" in Lisp.

I ignored automatic formatting because it's not part of the language, but of tooling. The problem with tooling is that not everyone uses it. I've had a "pleasure" of working with a 50+ kloc Clojure code base written mostly by C programmers who didn't know or care about formatting tools - honestly, it was a nightmare. Of course, each file could be automatically reformatted into something sensible, but the fact that it was written the way it was and the language did nothing to prevent that still stands. In Python, you at least would get the indentation right.

Readability is a hard problem in general. You're right that it's also a matter of education in the community. You're right that it's almost negligible a problem for lispers themselves, as they know how to reformat the code automatically with a single key press. It is a problem, though, for people who come into contact with Lisp code for the first time. I wanted to convince fnord123 that it's not the syntax itself, but rather how it is used that's a problem - like with every other kind of syntax out there, by the way. I'd be extremely happy if he reconsidered and tried to read some of the better-written s-exps based code.

[kazinator]

In Lisp languages, we don't use pure S-exps for everything; we have notations. We have 'X instead of (quote X), `(,A ,@B) instead of (list* 'A B), and numerous # notations.

In the area of arithmetic, although the basic operators are functions invoked using (f arg ...), we give them short names like +, -, * and /. Why? The obvious reason is that we would find it irksome to be writing (add ...) and (mul ...).

Lisp can have notations, and they can be had without disturbing the Lisp syntax. Notations that are related to major program organization have payoff.

In TXR Lisp there is relatively small set of new notations, which all have correspondence to S-exp forms, the same way that 'X corresponds to (quote x).

  ;; slot access
  obj.x.y.z  -->  (qref x y z)

Of course, people are going to prefer this to something like:

  (slot-value (slot-value x 'y) 'z)

Then:

  ;; unbound slot access
  .x.y.z --> (uref x y z)

  ;; method call
  obj.x.(f a b)  --> (qref x (f a b))

  ;; x.f(blah).g(foo).xyzzy(x, y) pattern:

  x.(f blah).(g foo).(xyzzy x y)  ;; looks like this

  ;; sequence indexing, function calls (the "DWIM" operator)
  [array i]  --> (dwim array i)
  [f x y]    --> (dwim f x y)

  ;; ranges:
  a..b  --> (rcons a b)

  ;; slice
  [str 0..3] --> [dwim str (rcons 0 3)]

  ;; Python-like negative indexing:

  [str -4..:] --> [dwim str (rcons -4 :)] ;; : means "default value: one index past end sequence (its length)".

  ;; quasistrings -- recently appeared JavaScript in strikingly similar form!
  `@a @b ...` --> (sys:quasi @a " " @b) --> (sys:quasi (sys:var a) " " (sys:var b))

  ;; word list literals
  #"a b c"  --> ("a" "b" "c")

  ;; quasi word list literals
  #`a @b c`  --> (sys:quasilist `a` `@b` `c`)

Some Lisp syntax is streamlined:

   (lambda (a b c : x y  . r) ...)  ;; a b c required, x y optional, r rest

Dot notation allowed without preceding atom:

   (. x)  ->  x

Improper lists can be function calls:

   ;; TXR: verbosity-free wrapping of a function.
   (defun wrapper (. args)
     (wrapped . args))

   ;; CL:
   (defun wrapper (&rest args)
     (apply #'wrapped args))

This works even if the thing in the dot position is a symbol macro expanding to a compound form. The reason is that the code walker/expander will recognize and transform (func ... . rest) into (sys:apply (fun func) ... rest) first, and then expand macros. (I.e. we can't work this into existing Lisps like CL implementations without going down to that level.)

   ;; the : symbol -- symbol named "" in keyword package:
   ;; used as a "third boolean" in various places

   (func 1 2 : 4)  ;; use default value for optional arg, pass 4 for the next one
                   ;; diminishes need for keyword args


   ;; built-in regex syntax
   #/a.*b/

   ;; C-like character escapes
   "\t blah \x1F3 ... \e[32a"

   ;; multi-line strings with leading whitespace control:

   "Four sc \
    ore
   \ and seven years ago"  -> "Four score and seven years ago"


   ;; Simple commenting-out of object with #;

   #; (this is 
         commented out)

Also, there is no programmable reader in TXR Lisp; no reader macros. I'm not a big fan of reader macros. They are only useful for winning "I can have any damn syntax in my language" arguments. Problem is, the whole territory of "any damn syntax" is a wasteland of bad syntax, nt to mention mutually incompatible syntax.

[klibertp]

Ah, yeah, I forgot to mention this explicitly earlier, although I had this in mind when I wrote "s-exps based" syntaxes and when I wrote this part:

> and you can add whatever syntax sugar you need in a couple of lines of a macro. In other words, Lisps give programmers tools for making their code as readable as they want (and are able to)

But, I didn't provide any examples, so big thanks for filling in this gap! :)

---

> The obvious reason is that we would find it irksome to be writing (add ...) and (mul ...).

Yet, in Scheme, we have `add1` and `sub1`, `expt` instead of `^` and so on. The former two are written `1+` and `1-`, respectively, in Emacs Lisp and Common Lisp. What I want to say is that evidently there are people who don't mind the longer names ;) (Personally, I don't mind either, I think)

> Lisp can have notations, and they can be had without disturbing the Lisp syntax.

Plus, you can introduce them without modifying the language implementation, ie. you can add notations to Lisp even if you're just a language user and not the language developer. This is impossible in 99% of other languages.

> Notations that are related to major program organization have payoff.

I'm seriously wondering if I could fit this on a T-shirt! :) It's very true, and when I wrote about "incredibly readable and to the point" code I was thinking exactly about crafting and using special notations and internal DSLs. The payoff of a well thought-out notation can be tremendous.

> In TXR Lisp there is relatively small set of new notations

I don't know about that :D At least TXR Lisp has more syntactic conveniences than Scheme, CL, raw Racket and Emacs Lisp. I think only Clojure comes close - at least object slots' access is written in a similar way. But I think there's no syntactic sugar for slices, nor for string interpolation in Clojure.

The syntax for "rest args" is similar in Schemes - `(define (f . args) ...)` - but they don't have a convenient way of calling a function which accepts a variable number of arguments, you need to use `apply` for that.

> (I.e. we can't work this into existing Lisps like CL implementations without going down to that level.)

So, if I understand correctly, it would be possible to add it to CL or Scheme via reader macros or reader extension respectively? Although I can't imagine the specific implementation right now, I think reader macros/extensions are executed during parsing, before the code is expanded, so it should work? With some caveats of course (some more tokens would be needed to identify where the `.` should be treated specially, I think).

Either way, use of the dot/improper lists for applying variadic functions is great, I love the symmetry between definition and application here. Makes me wonder why isn't this handled like that in other Lisps? It seems so natural once you see it...

Ah, regarding the slices, I, of course, have to ask - what is `dwim` and how is it implemented? If my guess is correct, `dwim` stands for "Do What I Mean", which would certainly fit in this case :D But it seems to work on everything collection-like. In Racket there's a sequence protocol you could use here (used to great effect in data/collection package[1]), in CL it would be a simple generic method with implementation for various kinds of collections, but how is it done in TXR Lisp?

---

Anyway - if there's ever anyone other than us reading this thread ;) - the TLDR would be that Lisps can, and do, have notations above and beyond bare s-exps. These notations tend to have a deep impact on both convenience during writing and on readability later. There are differences between Lisps in terms of support for additional notations, but you can introduce them (aka. steal from other Lisps ;)) yourself if you find a particular extension to be useful in your situation. An example of this would be the "threading macros", `->` and `->>`, which are now available in every Lisp out there, even though they were first implemented in Clojure only (I think - was there "prior art" somewhere else, maybe?) Anyway, that's part of what makes Lisps so incredibly powerful!

---

EDIT: reader macros. I understand your position, and most of the notations can be introduced as normal macros anyway, but... without them you're f+++ed if there's some notation you'd like to have, but which cannot be introduced without modifying the reader. Yes, compared to the number of possible syntax extensions the number of useful syntax extensions is frighteningly small. Yes, it's a major way of introducing incompatibilities. Clojure & Rich Hickey also excluded reader macros, adding to the previous argument one of security, ie. Clojure, when used as a data interchange format, should not have behave differently in different environments because of loaded reader extensions, plus it's rather scary that a simple `read` can format your disk, kill your cat, steal your car and so on. I agree that in most cases they should not be used... but. That "but" still remains in my head :)

An example: there's a package for CL which allows you to alias module names. Writing them long-hand over and over again is less than ideal, to be honest. Due to how CL module system works it's impossible to achieve such aliasing with normal macros, you have to drop one level below. It certainly introduces an incompatibility, but it made writing code so much more pleasant!

So anyway: TXR Lisp and more or less Clojure have a lot of notations built-in and maybe this is enough. But some other Lisps, without reader macros, would be stuck in the '80s as far as notational convenience go. So, to me, even with all the downsides (and in the context of said Lisps), reader macros are an important tool which allows for language evolution and improvement over the years.

---

[1] https://docs.racket-lang.org/collections/collections-example...

[kazinator]

Regarding 1+, add1 and so on, I called these succ and pred in TXR Lisp. This is inspired by a an operator keyword in Pascal. There are also ssucc sssucc and sssucc as well as ppred, ppred and pppred. That is inspired by our caddr friends, and also by Douglas Hofstadter's ..SSS0. Now (+ 1 x) is one character shorter than (succ x), and likely clearer to more people. But, succ can be used as a higher order function without requiring partial application of the 1. Where we would have (op + 1) we can just use succ.

Can we have (func arg . rest) using just read macros in CL? It's not so simple because we cannot blindly hijack the parenthesis and turn every dot notation into apply. It has to be the expander recognizing it. The TXR Lisp expander knows that it's dealing with a function call, because the symbol in the CAR position isn't a macro or special operator. Only then does it check for an improper list, and do the "dot apply transform". So this has to be worked into the code expander. (Thus if we work portably over top of CL, we are writing our own code walker and not relying on the CL one.)

> But some other Lisps, without reader macros, would be stuck in the '80s as far as notational convenience go.

Right; that's because all the usual notational conveniences like quote, backquote and whatnot predate the 1980's by quite a bit, and there hasn't been anything new.

> An example: there's a package for CL which allows you to alias module names.

Are you talking about package local nicknames? That's an example of something being rolled into implementations. If everyone was on board in integrating the feature, programs wouldn't have to hack this.

Though it requires hooking into the reader, I don't believe that the feature changes syntax. If foo is a package local nickname, then foo:whatever is the same syntax.

So at least this will not break editing support.

Most uses of read macros break editor support. Syntax coloring and other features break.

Read macros lack print support. For instance, if the 'X syntax weren't built in, it couldn't be produced with read macros completely. The input notation can be produced with read macros; however, Lisps also print the (quote X) object as 'X.

Almost all the junk I put into TXR Lisp prints, e.g.:

  1> '(qref a b (c) d)
  a.b.(c).d
  2> '(rcons a b)
  a..b

Even the singleton consing dot, at least if in a lambda:

  3> '(lambda x y)
  (lambda (. x)
    y)
  4> '(dwim a b c)
  [a b c]

> I love the symmetry between definition and application here

It's just good old "declaration follows use":

   void (*func)(int);

   (*func)(42);

:)

> The syntax for "rest args" is similar in Schemes - `(define (f . args) ...)`

But I don't think they have (lambda (. args) ...). That's lateral thinking. Why not have a consing dot with nothing in front of it such that (. x) is just x?

Here is the intellectual precedent for it, right in ANSI Lisp:

  $ clisp -q
  [1]> (append '(1) 2)
  (1 . 2)
  [2]> (append 2)
  2
  [3]> (list* 1 2)
  (1 . 2)
  [4]> (list* 2)
  2

I.e. the terminating atom of an inproper list is itself an "empty improper list", identical to that atom itself, just like an empty proper list is equivalent to the atom nil.

Hence if () is the empty list, that being the terminator nil, then (. 3) is the empty improper list terminated by 3, which is just the terminator 3 itself.

> what is `dwim` and how is it implemented?

dwim is implemented half trivially, half not-so-trivially. The basic idea is that it is just a placeholder which displaces the operator to the second position of the form, where it is consequently evaluated as an expression, allowing for Lisp-1 style invocation without a funcall symbol. Well, of course dwim is that symbol, but it's hidden behind the [ ] notation.

The not-so-trivial part is that, dwim also applies a special treatment to those of its arguments which happen to be symbols. These are resolved in a single namespace that folds together variable and function bindings. This allows us to do things like [mapcar list '(1 2 3)], even though we are in what is fundamentally a Lisp-2. Why this is not-so-trivial is that it's deeply ingrained into the expander, evaluator and compiler, which understand the conflated namespace and implement it properly (in the face of lexical functions, macros, symbol macros and such).

All the semantics under dwim for objects being callable as functions is actually implemented in the function call mechanism, not in dwim.

By brief example:

  1> ["abcd" 2..3]
  "c"
  2> (call "abcd" 2..3) ;; no "dwim" here at all
  "c"

Simply, the semantics of calling a sequence with a single argument which is a range object is to perform a subsequence extraction.

Dwim differs from call in that it is an assignment place:

  1> (let ((c (copy "abcd")))
        (set [c 1..3] "X")
        c)
  "aXd"

But this is not any sort of magic; it's played out with macros and some run-time-support:

  2> (expand '(let ((c (copy "abcd")))
        (set [c 1..3] "X")
        c))
  (let ((c (copy "abcd")))
    (let ((#:g0073 c)
          (#:g0075 #R(1 3)))
      (sys:setq c (sys:dwim-set t #:g0073 #:g0075
                                "X"))
      "X")
    c)

There is a special sys:dwim-set run-time function which contains the range assignment semantics. It is allowed to mutate the object, but that's not always possible; therefore, the return value must be captured. The place expander framework takes care of it, generating the sys:setq assignment back to the c variable, just in case a new object is required. (The thing that newbie Lisp programmers forget to do when using functions like remove-if or append!)