[kazinator]

If you're mutating any object, it is necessarily a value with reference semantics, period. Objects that do not have reference semantics are immutable.

Some objects that cannot be mutated (like numbers) can have reference or value semantics depending on how they are implemented. For instance, a bignum integer always has reference semantics. Small integers usually have value semantics: they fit into a machine word with no heap part. In that case, all instances of the number 0 or 1, and some range beyond that in both directions, will always be the same object according to eq.

If you're mutating an object (and, thus, something that has reference semantics), the difference that the reference semantics makes is that other parts of your program may hold a reference to that object; your code has not received a copy. If you haven't accounted for that, you probably have a bug.

Sure, this stuff isn't obvious; unless you already know another dynamic language like Ruby, Javascript, Python, ...

Complete neophytes have to be taught it from the fundamentals.

[Blackthorn]

No, there really is no consistency to Common Lisp's mutation.

  $ sbcl
  * (defvar a (list))
  A
  * a
  NIL
  * (defun x (v) (push 'b v) v)
  X
  * (x a)
  (B)
  * a
  NIL

Now if you were to do similar with something like an array, the original variable would be mutated. Just another example of how Common Lisp doesn't have any sort of internal consistency. Once again, rules are non obvious, do not follow the principle of least surprise.

[kazinator]

> Now if you were to do similar with something like an array, the original variable would be mutated.

That is false. To do a similar thing with an array, we need a non-mutating operation which returns a new array which is like an existing array, but with an element prepended.

Then we need a macro to mutate a place to replace an existing array in that place with a new such an array.

Then, exactly the same kind of behavior will be reproduced:

  (defun array-cons (obj array)
    (let ((new-array (make-array (list (1+ (length array))))))
      (replace new-array array :start1 1)
      (setf (aref new-array 0) obj)
      new-array))

  (defmacro apush (val array-var)
    (assert (symbolp array-var) (array-var)
            "fixme: simple implementation: ~a must be symbol")
    `(setf ,array-var (array-cons ,val ,array-var)))


  [1]> (defvar a #())
  A
  [2]> (defun x (v) (apush 'b v) v)
  X
  [3]> (x a)
  #(B)
  [4]> a
  #()

What? Of course; we are not mutating any object here, but a variable: the local variable of x.

  [5]> (apush 1 a)
  #(1)
  [6]> (apush 2 a)
  #(2 1)
  [7]> (apush 3 a)
  #(3 2 1)

Lists work this way because they are made of cells, and those cells are immutable (if you want them to be) for very good reasons. This is part of the essence of Lisp since the dawn of the language.

It makes less sense to treat arrays that way. It's possible, but you need an exotic data structure to do it even halfway efficiently; that structure will never be as efficient as an ordinary mutable array for ordinary array work.

Whereas, treating singly linked lists this way is almost free of additional cost.

[Blackthorn]

> That is false.

No, it's not. Notice how, in my example, the resultant list is updated in the function parameter, but not the initial var defined by defvar. Whereas if I made an array via (make-array), passed it into the function, and updated that by the way the language documentation tells you to (setf and one of the aref functions), you'd end up with both the function parameter and initial var both pointing to the updated value. These are two logically different behaviors! And that's exactly what my criticism stated: "When you pass a value to a function, is it by value or by reference? Who knows? Rules are non obvious, do not follow the principle of least surprise."

> Lists work this way because... It makes less sense to treat arrays that way.

Yes, that's exactly the point. The language is inconsistent and does not follow the principle of least surprise.

[mtrower]

> When you pass a value to a function, is it by value or by reference? Who knows?

Value. Value, value, always value. However, it seems that maybe you don't understand exactly what the value is that you're passing.

> Notice how, in my example, the resultant list is updated in the function parameter

No [0]. It makes a new list (really a new cons cell, which contains the new item and then points to the old list [1]), and assigns that to v. And your example doesn't actually show this update happening, it just shows the return value of push (it so happens that it is updated, however). The original list --- passed in or otherwise --- is never changed. You say that you're "updating" a list, but you're not mutating or updating your data structure at all --- you're making something new, and assigning that to v.

> These are two logically different behaviors!

They are two logically different operations. Are you sure that you understand the data structures you're working with, and the operators that you're calling on them? Did you perhaps try to map concepts from another language into Common Lisp, find functionality that looked similar on the surface, then become surprised when the results were not identical?

Linked lists differ fundamentally in structure from arrays, and so the operators which are commonly used with them differ in turn. Perhaps you would like to compare arrays to vectors, as a closer approximation in data structures, with similar typical strategies of manipulation?

> does not follow the principle of least surprise.

You keep saying this, as if that is that. But nothing about your example is surprising to me, so I suppose this is a matter of perspective.

[0] http://clhs.lisp.se/Body/m_push.htm

[1] https://en.wikipedia.org/wiki/Linked_list

[Blackthorn]

> Value. Value, value, always value. However, it seems that maybe you don't understand exactly what the value is that you're passing.

This is a distinction without a difference.

> ...you're making something new, and assigning that to v.

Yes, I know that. The point is that the behavior seen by the user for similar operations is entirely different between lists and other pieces of Common Lisp.

> They are two logically different operations.

Yes, obviously. The point is that Common Lisp does an absolutely crap job of making these things actually consistent from the point of view of the user. The language is littered with entirely inconsistent behavior and choices.

[kazinator]

> Common Lisp does an absolutely crap job of making these things actually consistent from the point of view of the user

Common Lisp provides a decently designed sequences abstraction which allows the encapsulated vectors and traditional unencapsulated lists, as well as strings, to be manipulated not just similarly, but by exactly the same code.

This was developed in recognition of the exactly the issue that you are getting at. Forty years ago, the group of people designing Common Lisp were aware of this desire to have a consistent access method for different kinds of sequences and they did something about it.

The charge of "absolutely crap job" can only be fairly leveled at a language that make no effort to provide for uniform treatment of encapsulated vectors and unencapsulated lists.

What looks "surprising" depends on your background. I agree that Lisp contains surprises for someone who has programming experience, but that experience is limited to Python or Javascript which provide only encapsulated arrays as the principal sequence aggregation mechanism.

I had two decades of programming experience coming to Lisp, and had written C programs which used both unencapsulated lists:

   list_node *list = NULL;

and I had written code with encapsulated ones:

   list_block list = LIST_EMPTY_INITIALIZER(list); // eg circular, expanding to { &list, &list }

leveraging the advantages of both.

So, I wasn't surprised in any way. From the description of NIL and the cons cell linkage in the book I was reading, I instantly recognized it instantly as the unencapsulated style of lists, like "list_node *list = NULL".

In C, it would be obvious that this can't work:

   list_node *list = NULL;
   list_append(list, list_node_new(42));

   // wrongly expecting list to have changed from NULL to non-null!

but that, with an encapsulating list block object instead of a raw pointer to a node, this could work:

   list_block *list = list_new();
   list_append(list, list_node_new(42));

(Because C does not provide either of these, you can't blame the language for misunderstanding anything: only yourself, if you wrote that list yourself, or the library author.)

I remember that was intriguing to me how Lisp is getting away with the unencapsulated single linkage for everything: like how that is the list structure for the entire language. In the light of the functional programming (you can always just keep consing up new conses to transform lists) together with the garbage collection, it very soon clicked for me. I remember thinking that if we could just keep mallocing new nodes and not worry about freeing, that would be pretty nice to work with in C.