[swhipple]

> HTML is a string of characters (syntax). The DOM is a data structure (semantics). [...] S-expressions are a data structure, different from the DOM, but S-expression syntax is a syntax.

I believe this is where the confusion is coming from. When you parse HTML syntax, you get a data structure; this is the same as when you read sexpr syntax, you also get a data structure. Both these data structures are different from the DOM tree.

Try this example:

    <pre>
      <span>one
      </span>
      <br>
      <span>two</span>
      <br />
    </pre>

Can CL-WHO generate HTML that matches that? (i.e. feed both into a tool like BeautifulSoup and produce the same data structure?)

Outside of CL-WHO and Hiccup-type libraries, you can of course use S-exprs to represent the same data structure. Here's a hypothetical S-expr syntax that might produce the same data structure:

    ((pre)
      "\n  " (span) "one\n  " (/span)
      "\n  " (br)
      "\n  " (span) "two" (/span)
      "\n  " (br/) "\n"
     (/pre))

Which is what I believe JimDabell meant by:

> you can't represent all valid HTML documents as S-expressions, at least not in the convenient way people assume

[lisper]

> Both these data structures are different from the DOM tree.

In the case of S-expressions that is true. In the case of HTML it may or may not be true. It depends on how the HTML parser is implemented. There is a "natural" mapping of HTML onto a parse tree that is different from the DOM, but that is not part of the standard (AFAIK).

> Can CL-WHO generate HTML that matches that?

Yes, though native Common Lisp does not provide c-like string escapes so putting in newlines is a little awkward. You could, of course, bring in a string interpolation library, but here's how you can do it without that:

    ? (defun nl () (who (fmt "~%")))     ; NL = NewLine
    NL
    ? (defun nli () (who (fmt "~%  ")))  ; NLI = NewLine + Indent
    NLI
    ? (princ (html (:pre (nli) (:span "one" (nli)) (nli) (:br (nli) (:span "two") (nl)))))
    
     <pre>
       <span>one
       </span>
       <br>
       <span>two</span>
     </br></pre>

Or you could do this:

    (html (:pre "
      <span>one
      </span>
      <br>
      <span>two</span>
      <br />
    "))

which looks like cheating but is actually closer to the spirit of the original.

The PRE tag is really weird because it actually changes the way things inside it are parsed. You can actually implement that in Lisp too via reader macros. CL-WHO doesn't support that out of the box, but it's not hard.

I can't imagine anyone actually wanting to do that, though. The PRE tag is for presenting pre-formatted text without changing its appearance, so embedding other tags inside it is kinda perverse. [EDIT: I was wrong about this. See below.]

[kazinator]

There are uses for pre with tags embedded.

pre provides the simplified line breaking and usually a monospaced font. However, tags are available to do whatever else.

A major example is that the Vim editor uses pre for formatting syntax colored code to HTML (when you do that with :TOhtml).

The output is a pre block containing various span elements which are styled with CSS.

BTW where in the HTML spec does it say that the interior of pre is parsed differently?

If we are parsing HTML (to Lisp objects or whatever), we should preserve the exact whitespace. The reverse generation should regurgitate the original whitespace.

If we take the license to eliminate newlines, then we ruin pre. The fix is simply not to do that.

[lisper]

> where in the HTML spec does it say that the interior of pre is parsed differently?

I was wrong about that. I had a vague memory of putting HTML inside a PRE tag once and having it come out as if it were escaped, but apparently I hallucinated that.

> A major example is that the Vim editor uses pre for formatting syntax colored code to HTML (when you do that with :TOhtml).

OK, I stand corrected on that too.

> If we are parsing HTML (to Lisp objects or whatever), we should preserve the exact whitespace. The reverse generation should regurgitate the original whitespace. > If we take the license to eliminate newlines, then we ruin pre. The fix is simply not to do that.

Right.

Actually, I just realized that I mis-read the example. I saw <br /> and thought it was </br>. (Maybe the OP edited it?) In any case, the example now reads:

    <pre>
      <span>one
      </span>
      <br>
      <span>two</span>
      <br />
    </pre>

And you can render that in sexpr syntax as:

    (:pre "
      " (:span "one
      ") "
      " (:br) "
      " (:span "two") "
      " (:br) "
    ")

This is a particularly bad example to demonstrate here because the whitespace in the code plays badly with the whitespace in the HN markup. But I tried running this code and it does work. Here is the output copied-and-pasted verbatim from my listener:

    <pre>
      <span>one
      </span>
      <br />
      <span>two</span>
      <br />
    </pre>

Note that both BR tags are rendered as <br />.

[swhipple]

It was <br> and <br /> for my example (</br> isn't a valid tag). The point that I was getting at was that <br> and <br /> self-closing tag are represented differently (<tag>, <tag />, and <tag></tag> are all different) in a parsed SGML data structure (though they both are equivalent in the HTML DOM tree in the browser).

This is why you would need separate tags to emit them properly with an S-expr syntax (tag), (tag/), and (tag)(/tag) in my example.

[lisper]

You can do this:

    <tag> ==> (:tag)
    <tag/> ==> (:tag nil)
    <tag></tag> ==> (:tag "")

Using (:tag/) is a bad idea because that would screw up attributes.

CL-WHO doesn't support this, but that would be easy to change if it ever actually mattered to anyone.

[swhipple]

> In HTML, <tag/> and <tag></tag> are equivalent

In HTML, <script></script> is valid. <script /> is invalid. <br /> is valid. <br></br> is invalid. So they are represented differently.

> Using (:tag/) is a bad idea because that would screw up attributes.

For my example?

    ((:tag/ :attr "value"))               => <tag attr="value" />
    ((:tag  :attr "value") "..." (:/tag)) => <tag attr="value">...</tag>

> You actually can distinguish between those if you really want to. It's just a matter of picking a convention.

That sounds like it could work. So a leading `nil' would be treated as a special case (not a child node):

    (:pre "
      " (:span "one
      ") "
      " (:br) "
      " (:span "two") "
      " (:br nil) "
    ")