[kazinator]

There is no "by reference" in Common Lisp; everything is a value. Some values have reference semantics. This makes no difference unless you're mutating, or making unwarranted assumptions about the eq function.

To understand most code, you can just pretend that all values have reference semantics. If mutation is going on and/or the eq function is being used, you have to prick up your ears and pay attention to that detail.

[Blackthorn]

> Some values have reference semantics. This makes no difference unless you're mutating, or making unwarranted assumptions about the eq function.

That's pretty damn far from "no difference"! Once again, rules are non obvious, do not follow the principle of least surprise.

[kazinator]

If you're mutating any object, it is necessarily a value with reference semantics, period. Objects that do not have reference semantics are immutable.

Some objects that cannot be mutated (like numbers) can have reference or value semantics depending on how they are implemented. For instance, a bignum integer always has reference semantics. Small integers usually have value semantics: they fit into a machine word with no heap part. In that case, all instances of the number 0 or 1, and some range beyond that in both directions, will always be the same object according to eq.

If you're mutating an object (and, thus, something that has reference semantics), the difference that the reference semantics makes is that other parts of your program may hold a reference to that object; your code has not received a copy. If you haven't accounted for that, you probably have a bug.

Sure, this stuff isn't obvious; unless you already know another dynamic language like Ruby, Javascript, Python, ...

Complete neophytes have to be taught it from the fundamentals.

[Blackthorn]

No, there really is no consistency to Common Lisp's mutation.

  $ sbcl
  * (defvar a (list))
  A
  * a
  NIL
  * (defun x (v) (push 'b v) v)
  X
  * (x a)
  (B)
  * a
  NIL

Now if you were to do similar with something like an array, the original variable would be mutated. Just another example of how Common Lisp doesn't have any sort of internal consistency. Once again, rules are non obvious, do not follow the principle of least surprise.

[kazinator]

> Now if you were to do similar with something like an array, the original variable would be mutated.

That is false. To do a similar thing with an array, we need a non-mutating operation which returns a new array which is like an existing array, but with an element prepended.

Then we need a macro to mutate a place to replace an existing array in that place with a new such an array.

Then, exactly the same kind of behavior will be reproduced:

  (defun array-cons (obj array)
    (let ((new-array (make-array (list (1+ (length array))))))
      (replace new-array array :start1 1)
      (setf (aref new-array 0) obj)
      new-array))

  (defmacro apush (val array-var)
    (assert (symbolp array-var) (array-var)
            "fixme: simple implementation: ~a must be symbol")
    `(setf ,array-var (array-cons ,val ,array-var)))


  [1]> (defvar a #())
  A
  [2]> (defun x (v) (apush 'b v) v)
  X
  [3]> (x a)
  #(B)
  [4]> a
  #()

What? Of course; we are not mutating any object here, but a variable: the local variable of x.

  [5]> (apush 1 a)
  #(1)
  [6]> (apush 2 a)
  #(2 1)
  [7]> (apush 3 a)
  #(3 2 1)

Lists work this way because they are made of cells, and those cells are immutable (if you want them to be) for very good reasons. This is part of the essence of Lisp since the dawn of the language.

It makes less sense to treat arrays that way. It's possible, but you need an exotic data structure to do it even halfway efficiently; that structure will never be as efficient as an ordinary mutable array for ordinary array work.

Whereas, treating singly linked lists this way is almost free of additional cost.

[Blackthorn]

> That is false.

No, it's not. Notice how, in my example, the resultant list is updated in the function parameter, but not the initial var defined by defvar. Whereas if I made an array via (make-array), passed it into the function, and updated that by the way the language documentation tells you to (setf and one of the aref functions), you'd end up with both the function parameter and initial var both pointing to the updated value. These are two logically different behaviors! And that's exactly what my criticism stated: "When you pass a value to a function, is it by value or by reference? Who knows? Rules are non obvious, do not follow the principle of least surprise."

> Lists work this way because... It makes less sense to treat arrays that way.

Yes, that's exactly the point. The language is inconsistent and does not follow the principle of least surprise.

[mtrower]

> When you pass a value to a function, is it by value or by reference? Who knows?

Value. Value, value, always value. However, it seems that maybe you don't understand exactly what the value is that you're passing.

> Notice how, in my example, the resultant list is updated in the function parameter

No [0]. It makes a new list (really a new cons cell, which contains the new item and then points to the old list [1]), and assigns that to v. And your example doesn't actually show this update happening, it just shows the return value of push (it so happens that it is updated, however). The original list --- passed in or otherwise --- is never changed. You say that you're "updating" a list, but you're not mutating or updating your data structure at all --- you're making something new, and assigning that to v.

> These are two logically different behaviors!

They are two logically different operations. Are you sure that you understand the data structures you're working with, and the operators that you're calling on them? Did you perhaps try to map concepts from another language into Common Lisp, find functionality that looked similar on the surface, then become surprised when the results were not identical?

Linked lists differ fundamentally in structure from arrays, and so the operators which are commonly used with them differ in turn. Perhaps you would like to compare arrays to vectors, as a closer approximation in data structures, with similar typical strategies of manipulation?

> does not follow the principle of least surprise.

You keep saying this, as if that is that. But nothing about your example is surprising to me, so I suppose this is a matter of perspective.

[0] http://clhs.lisp.se/Body/m_push.htm

[1] https://en.wikipedia.org/wiki/Linked_list