[catnaroek]

I'm aware of EQUALP. But the problem remains that it's possible to distinguish between supposedly “equal” values. Lisp and Scala are first and foremost object-oriented languages - whatever values you want to manipulate are always subordinate to objects whose physical identity in memory matters in the language's semantics, no matter how irrelevant they might be for your problem domain.

On the other hand, in Haskell and ML, due to their superior value-oriented design, there's no way to distinguish between “this 123456789” and “that 123456789”, or between “this list [1,2,3]” and “that list [1,2,3]”. There is only one number 123456789 and one one list [1,2,3] in the language's semantics, no matter how many copies of their representation might exist in memory.

[junke]

I am also aware of that. I am not really trying to convince you of anything, mostly trying to prevent casual readers from being taught Common Lisp from you. Look at the example you just gave, this is beyond ridiculous. I would prefer if you were focusing on what you want to fight "for", not what you want to fight "against".

You like having a strong separation between the language and its implementation. I get it. Note however that if you are the Haskell compiler, you can know things the programmer cannot, or you can inject code that can perform manipulations the programmer cannot express. There probably is an identity equality operator down there, that people cannot access. If you like that, I can understand; I can understand the appeal for a strict separation of concerns and the desire to express only the high-level intent. But really, I think this is not much different in CL and that you are focusing on implementation details. In CL the separation is not enforced, and that is your main problem with it.

The CL compiler is defined at the language specification level. You have access to internals, by choice, just as if you were writing Haskell ASTs using an Haskell compiler's internal API. As such, you can cross abstraction barriers whenever you need. We already discussed about this once, you can use purely functional data-structures (see FSET) and EQUALP and code to that functional interface and pretend there is no computer down there. Then, if you want, you can play with packages and symbols to make MAP & co. refer to the parallel version (see LPARALLEL), re-compile and have parallel code (this works best with unqualified symbols and different package definitions for the same file).

In all PL discussions, there is eventually mention of an hypothetical sufficiently smart compiler. The CL point of view is (among other things) that such a compiler is one where a programmer can easily add its own extensions. That allows you to express the intent of the program clearly in one place and have the implementation details elsewhere.

[catnaroek]

> Note however that if you are the Haskell compiler, you can know things the programmer cannot, or you can inject code that can perform manipulations the programmer cannot express.

Those would be implementation details, not part of the language itself. Hence...

> There probably is an identity equality operator down there, that people cannot access.

... this doesn't make sense.

> You have access to internals, by choice, just as if you were writing Haskell ASTs using an Haskell compiler's internal API.

However, there's no way to portably manipulate compiler internals in Haskell. Heck, Haskell syntax, unlike Lisp syntax, can actually be represented in multiple ways: named variables, de Bruijn indices and levels, HOAS, etc. And this is a good thing.

> In all PL discussions, there is eventually mention of an hypothetical sufficiently smart compiler.

I wouldn't have brought it up myself.

> The CL point of view is (among other things) that such a compiler is one where a programmer can easily add its own extensions.

There's no dichotomy between extensibility and abstraction enforcement, even if you try to set up one.

[junke]

> ... this doesn't make sense.

In order to have an ML, you take a subset of Common Lisp and specialize/extend it in a very specific direction. The part that are removed from Lisp are the one you don't care about. I care about being able to use different equivalence classes, including the ones the compiler is going to require, like identity equality, because this matters when I implement abstractions myself.

> However, there's no way to portably manipulate compiler internals in Haskell.

Yes, why not, that would be damn useful. The alternative is to hack each implementation in its own way.

> Heck, Haskell syntax, unlike Lisp syntax, can actually be represented in multiple ways: [...]

Do you get the difference between internal and external representations? When you see (lambda (a) (+ a 1)), you don't know how the code is represented internally and how its value, when executed, is represented internally (when my compiler performs data-flow analysis, it uses a specific internal representation I don't need to know). Yet, you have access to a uniform API, defined at the language level, to manipulate data. That's why you can have the same source code working on the JVM, interpreted using a C/C++ runtime or directly expressed as assembly. In that sense, there is a separation between the language and its implementation. But part of the language specification is also there to guarantee a uniform way of building new extensions and integrate them with the compiler. That is a good thing. What is not enforced is how and when each feature is used. If that matters, dump a specialized Lisp image that you call the "compiler" and call it with Make on a file expressed in your custom DSL.

> There's no dichotomy between extensibility and abstraction enforcement, even if you try to set up one.

I am not trying to set up one. Just read more carefully.

[catnaroek]

> Do you get the difference between internal and external representations? When you see (lambda (a) (+ a 1)), you don't know how the code is represented internally and how its value, when executed [emphasis mine], is represented internally

I'm not talking about representations of semantic objects. I'm talking about representations of syntax. In Lisp's case, you don't get to choose - the macro system critically depends on a particular representation (named variables). Have fun writing macros that operate on HOAS.

[junke]

> In Lisp's case, you don't get to choose - the macro system critically depends on a particular representation (named variables)

Please tell me how Haskell represents HOAS and how this is not possible in Lisp.

Also, read Barzilay's paper (http://scheme2006.cs.uchicago.edu/15-barzilay.pdf), or about the Ergo Project (1988! http://repository.cmu.edu/cgi/viewcontent.cgi?article=2763&c...).

[catnaroek]

> Please tell me how Haskell represents HOAS and how this is not possible in Lisp.

I'm not talking about representing some object language's syntax in Haskell or Lisp. I'm talking about representing Haskell or Lisp's syntax in some other metalanguage (possibly Haskell or Lisp itself). How would you implement a macro expander that operates on HOAS?

> Barzilay's paper (http://scheme2006.cs.uchicago.edu/15-barzilay.pdf)

The object language implemented in that paper isn't the whole of Scheme, and in particular, it doesn't have macros.

> Ergo Project (1988! http://repository.cmu.edu/cgi/viewcontent.cgi?article=2763&c...)

Sorry, I can't find an explicit description of any object language's syntax in this paper.

[kazinator]

In what language do we input code as HOAS, and target that representation with macros?

(Can't we expand macros in a conventional representation with symbols and environments and then go to HOAS?)

(It's not news that macros are weak in some sense; I mean you can shoot yourself in the foot with the classic non-hygienic ones; you can disrespect even the first order abstract syntax that you're working in: the macro can re-locate a piece of raw syntax which contains identifier references into the wrong scope where those references resolve otherwise.)

Anyway, have fun fishing for mackerel with your bear trap?

[catnaroek]

The “conventional” representation of syntax using explicitly named variables is utterly inconvenient and error-prone for anything other than parsing. The very first thing I want to do after parsing is convert the syntax tree to de Bruijn indices/levels or HOAS.

[kazinator]

In Lisp, symbols are very important. And symbols are identity. An identity with other dressing, such as having a character string name which is somewhat of a semantic footnote.

"Functional programming" doesn't restrict the kinds of obejcts you can work with. An identity value, whose virtue is that it is different from other identities, is a legitimate concept which can be treated under functional programming.

I guess your problem is that you don't want non-symbolic objects from behaving like identities; only symbolic ones.

That is to say, two symbols are equal iff they are actually the same symbol, otherwise not---but other kinds of entities are treated differently.

There is something to be said for having just one kind of equality, which is appropriately defined for every kind of object.

Pragmatic issues get in the way though.

Let's consider "this list [1,2,3]" versus "that list [1,2,3]". What if we have lazy lists (as those things tend to crop up in functional languages, particularly non-strictly evaluated ones). Is "this infinite list [1,2,3, ...]" equal to "that infinite list [1,2,3,...]" even if they are generated by completely separate lazy procedures?

Your equality then has to basically test that two Turing computations are equivalent: that the underlying lazy generation procedures themselves are computing the same thing, even if in different ways.

In that vein, what do you do about structures with cycles and shared structure? Is the list #1=(1 . #1#) equal to another one that is also #1=(1 . #1#). (Lisp's equal functions don't have to handle this; but we could specify an equal function which does).

Equality is not so simple that we can just cover all of its nuances with a blanket rule based on some handwaving principle.

[catnaroek]

> An identity value, whose virtue is that it is different from other identities, is a legitimate concept which can be treated under functional programming.

Of course. But I don't want object identities to be the only values I can manipulate. I like having values like the list [1,2,3].

> Your equality then has to basically test that two Turing computations are equivalent: that the underlying lazy generation procedures themselves are computing the same thing, even if in different ways.

Indeed, this means that equality of higher-order values is undecidable. Therefore, don't rely on testing whether higher-order values are equal - you just can't! This is also why laziness by default is such a bad idea.

> In that vein, what do you do about structures with cycles and shared structure?

In Standard ML, `val rec xs = 1 :: xs` simply doesn't compile. The right-hand side of a `val rec` definition must be a function literal. This guarantees that inductive data types (such as lists and trees) mean the right thing, and that structural recursion on them always terminates.

> Equality is not so simple that we can just cover all of its nuances with a blanket rule based on some handwaving principle.

I'm not handwaving anything.