[dissenter]

OK, I'm going to send you a list using XML.

Now how do you propose we do that?

This is entirely the problem. The XML didn't solve anything. We still have to negotiate the terms of the transfer. We've agreed to use XML, true, but we are still at square one.

[olavk]

Of course you have to negotiate the terms of the transfer. You have to do that using any format (custom binary, XML, JSON, s-expressions, whatever). XML just defines a lot of common syntactic stuff which you would have to define anyway in any format you decide on.

[sah]

I would argue that there's very little value in standardizing that syntactic stuff. Whatever tiny amount of value there might be is probably destroyed by picking a convention as almost universally inappropriate as XML.

[utnick]

<list><listitem>a</listitem><listitem>b</listitem><listitem>c</listitem></list> ?

The xml solved the problem of me having to write a parser from scratch for whatever terms of transfer you come up with.

And I don't really see the downside for most applications.

[BrandonM]

So the fact that 96% of the characters you are sending are markup is not a downside?

S-expressions solved this problem a long time ago. (a b c) is only 57% markup. I don't think you can get much more succinct than that and still express the idea of a list.

[olavk]

XML have redundancy by design. It's a deliberate trade-off to make it easier to read and write by hand, at the cost of size.

If message size is an issue then gzip the data. Or if you have very specific needs for processing speed, look into something like Googles protocol buffers.

You are optimizing at the wrong level if you are concerned about a few extra characters in a human-readable data exchange format.

[BrandonM]

But is it really easier to read and write? Properly-indented S-expressions are just as readable. Generating XML and then gzip-ing it is a lot more work (and requires a lot more libraries) than generating S-expressions.

Case in point:

  <table>
    <tr>
      <td>a</td>
      <td>b</td>
    </tr>
    <tr>
      <td>c</td>
      <td>d</td>
    </tr>
  </table>

vs:

  (table
    (tr
      (td a)
      (td b))
    (tr
      (td c)
      (td d)))

Perhaps the real problem is that too many people use terrible text editors. Paren-matching and auto-indentation makes writing S-expressions orders of magnitude easier, and at least a constant factor easier than writing XML.

[olavk]

You are right about text-editors. XML was designed to be reasonable easy to write and edit by humans without specialized software. The redundant end-tag helps to catch errors and make structure more explicit.

Sure everyone could just use a fancy specialized editor with paren-matching auto-indentation. But one of the goals of XML was precisely that it should not rely on specialized software to be able to read and write.

Your example with the table is a lot clearer with sexpr syntax because you don't actually have any content in the table. Try again with a few sentences of mixed content, some bolded words, a link, and so on, and you will get my point.

Note that you would also need to gzip your s-expressions if you are concerned about size.

[BrandonM]

The HTML of your comment, with some formatting added:

        <span class="comment">
         <font color=#000000>
          You are right about text-editors. XML was designed to be reasonable
          easy to write and edit by humans without specialized software. The
          redundant end-tag helps to catch errors and make structure more
          explicit.
          <p>
          Sure everyone could just use a fancy specialized editor with paren
          matching auto-indentation. But one of the goals of XML was
          precisely that it should not rely on specialized software to be
          able to read and write.
          <p>
          Your example with the table is a lot clearer with sexpr syntax
          <b>because you don't actually have any content in the table.</b>
          Try again with a few sentences of mixed content, some bolded
          words, a link, and so on, and you will get my point.
          <p>
          Note that you would also need to gzip your s-expressions if you are
          concerned about size.
         </font>
        </span>

The same thing in S-expressions (an invented syntax):

        (span (class . comment)
          (font (color . #000000)
            You are right about text-editors. XML was designed to be reasonable
            easy to write and edit by humans without specialized software. The
            redundant end-tag helps to catch errors and make structure more
            explicit.
            (p)
            Sure everyone could just use a fancy specialized editor with paren
            matching auto-indentation. But one of the goals of XML was
            precisely that it should not rely on specialized software to be
            able to read and write.
            (p)
            Your example with the table is a lot clearer with sexpr syntax
            (b because you don't actually have any content in the table). Try
            again with a few sentences of mixed content, some bolded words, a
            link, and so on, and you will get my point.
            (p)
            Note that you would also need to gzip your s-expressions if you are
            concerned about size.))

It's really not a lot different. Of course, parens would need to be escaped, but this is no different from needing to escape < and >.

But one of the goals of XML was precisely that it should not rely on specialized software to be able to read and write.

But this is the problem with XML... it does rely on special libraries to validate and parse into reasonable data structures. It requires special heuristics to describe how to recover nicely in the event that markup isn't valid. It requires a document describing exactly what the XML needs to look like.

S-expressions are easier to parse, less verbose, and can accomplish all the same tasks and more, all while being more flexible in general.

[jamesbritt]

" Paren-matching and auto-indentation makes writing S-expressions orders of magnitude easier, and at least a constant factor easier than writing XML."

Funny; I find that the tag matching, auto-indentation, and auto-completion in vim make editing XML pretty easy.

Maybe S-expressions are slightly easier. Maybe not.

[jamesbritt]

" I don't think you can get much more succinct than that and still express the idea of a list."

What matters is: a list of what. Telling me (5, 7,11) means nothing.

    <sizes dress='5'  pants='7' shoes='11' /> 

is what's of value.

Would an sexp be terser? Sure. But not enough to matter.

[BrandonM]

That does not seem to be to be typical XML, and if it is, it's really being stretched to do something it's not intended to do, IMO. XML's strength is in representing tree-based structures, but that appears to be an attempt to represent an associative structure. With Sexps, this is just as easy:

  (sizes (dress . 5) (pants . 7) (shoes . 11))

But in doing that, the structure really looks off, even though it's almost exactly mirroring the XML. I think this is a clue that the XML is a bit of a stretch. Much better (in Lisp code) is:

  (let ((sizes '((dress . 5) (pants . 7) (shoes . 11))))
     ; do something with sizes
     ...)

But I guess the real question is what this is trying to represent. If it's the sizes of various people, then the Sexps are quite simple:

  (sizes
    (sally (dress . 5) (pants . 7) (shoes . 11))
    (suzy ...)
    (alice ...)
    ...)

Of course, this could be expressed in XML, but how to do it best?

  <sizes name='sally' dress='5' pants='7' shoes='11' />
  <sizes name='suzy' ... />
  <sizes name='alice' ... />

But then perhaps all these sizes should be wrapped in another tag:

  <sizes-list>
    <sizes name='sally' dress='5' pants='7' shoes='11' />
    <sizes name='suzy' ... />
    <sizes name='alice' ... />
    ...
  </sizes-list>

But now we're getting away from the structure we defined using S-expressions, and besides, name='...' seems to be distinctly different information from the sizes themselves, so something else is strange. Perhaps

  <sizes>
    <sally dress='5' pants='7' shoes='11' />
    <suzy ... />
    <alice ... />
    ...
  </sizes>

Well, that looks nice, and closer to what we are trying to represent, but of course it's impossible to validate (at least from what I know of XML), since the person names are not part of our schema. We'll have to do something like:

  <sizes>
    <person name='sally'>
      <dress size='5' />
      <pants size='7' />
      <shoes size='11' />
    </person>
    <person name='suzy'>
      ...
    </person>
    <person name='alice'>
      ...
    </person>
    ...
  </sizes>

But now we have the problem of having to place every type of clothing in our schema. We better change it some more

  <sizes>
    <person name='sally'>
      <clothing type='dress' size='5' />
      <clothing type='pants' size='7' />
      <clothing type='shoes' size='11' />
    </person>
    <person name='suzy'>
      ...
    </person>
    <person name='alice'>
      ...
    </person>
    ...
  </sizes>

Great! Now we have something that matches our desired structure and is easy to validate. Of course, it's much more verbose, but that made it easier to read and write, right?

Part of the problem with XML is that it causes these huge debates about how to structure and name the data. Another problem is that attributes don't nest nicely; that was the main problem in this instance. In other words, XML can be used nicely to represent a tree structure and reasonably well for lists or simple associative structures. But as soon as those associative elements need to map to something more complicated, you start having issues with how best to structure everything.

With S-expressions easily able to express assoc-lists while also being trivially nestable, these issues don't come up.

[jamesbritt]

"That does not seem to be to be typical XML, and if it is, it's really being stretched to do something it's not intended to do, IMO. XML's strength is in representing tree-based structures, but that appears to be an attempt to represent an associative structure. "

It's fairly typical of the XML I've used, and well within what XML was intended to do. That some (many?) people end up with needlessly verbose markup is not the fault of XML. Some people write verbose Scheme. Go figure.

A major point of XML is just simply tree data, but meta-data. You first showed a basic, non-annotated list; I showed a list with meta-data. Seems that you didn't like how the s-exp version of that XML looked, so you changed that use case.

"But in doing that, the structure really looks off, even though it's almost exactly mirroring the XML. I think this is a clue that the XML is a bit of a stretch. "

Or just maybe it's an example where XML differs from s-exps.

"Part of the problem with XML is that it causes these huge debates about how to structure and name the data."

Not really. I mean, some people like that stuff (I see it as a bike shed thing; it's a chance to show off how complex people can make something), but many other folks find a sparse, good-enough structure and move on. Quite honestly, the way you exaggerated the initial example is pure strawman. And you can have the same arguments about representation using s-expressions.

Don't blame a syntax because it allows people to be dopey.

I can see how nicely s-exp can work for markup, but I'm still curious how name-spaces, schema, ID + IDREF, transclusion, and other XML features are handled in s-expressions.

I mostly get the feeling that the only real gripe about XML is the duplication in the closing tags. (The W3C has explained why they dropped the short-form of XML and went with explicit end tags.)

[utnick]

It is a downside, but most of the time I don't care.

If I am sending some big list many times and speed really matters, then I care, but usually I am not. Size and speed are cheap nowadays.

[BrandonM]

It's not just about the amount of markup, though, it's about the unnecessary complexity of the markup. The software that generates and parses S-expressions is much simpler than that which generates and parses XML. Of course, in Lisp, it's just

  (let ((list (read data)))
     ...)

But even in Python, you could easily hack together (not recommended) something like

  list = ", ".join(data.split())
  ...

Of course, that's not robust, but the library which is robust is much simpler than the one that requires the use of a C sax parser just to be usably fast.

If the argument is that XML is more human-readable, that is implying that it's being human-modified, and then XML creates more work since it's so verbose. If the verbosity is not an issue because it's auto-generated, that implies that it's not being read/modified by humans, and the whole point of using XML in the first place is lost. I just can't see any problem that XML solves that S-expressions didn't already solve in a simpler way.

[jamesbritt]

Do sexp, out of the box, have a standard notion of ID and IDREF, transclusion, and schema?

There's more to XML than just annoying angle braces.

[olavk]

S-expressions are nice but not superior to XML for all use cases. S-expression syntax are optimized for lists of names and numbers. XML-syntax is optimized for structured documents. Since XML is used just as much for data as for documents these days, s-expressions would perhaps be just as good as XML for a common data exchange meta format. But that train left the station a decade ago.

I suspect a reason XML catched on and s-expressions didn't (outside of the Lisp-niche) is that XML tackled difficult internationalization issues like different encodings and character sets head on.

[anamax]

And strings, I forgot strings. (For some reason, XML folk often confuse strings with sequences of atoms. They also confuse numbers and numerals.)

And numbers in different bases.

S-expressions predate unicode. However, unicode support was added in. It's not that big a deal.

[anamax]

> S-expression syntax are optimized for lists of names and numbers.

Nope. Structs, tables, vectors, and arrays all work just fine.

And there's also support for dags and circular structures.

[trezor]

He said: With XML, we can easily agree and collaborate on a format and both of our languages have builtin libraries to extract the data we need.

You replied: OK, I'm going to send you a list using XML.

The only thing you have proved here is that your blind hatred for XML makes you unable to read and parse what is posted.

When arguing against the evils of a standardised format which there are proper parsers for everywhere, failing to parse stuff yourself is probably not on the list of things you want to do.

[sah]

You missed the point. After you've agreed on XML, you still have to agree on how to represent a list. You can use existing libraries to parse the XML, but you still have to write a parser to transform the resulting DOM into a list.

Plus, given that the availability of good XML parsers is one of the primary advantages claimed for XML, can anyone name some XML parsers that aren't ridiculously slow? Maybe I haven't looked hard enough, but I always seem to find that my own code to parse ad-hoc formats goes 10x faster than, e.g., expat parsing XML.

[trezor]

You missed the point. After you've agreed on XML, you still have to agree on how to represent a list.

No I didn't. Because that's exactly what the guy in the OP said. Agree on a XML schema for the data. After you've done that, writing some simple Xpath to get your data is done in minutes.

The only times my XML code is 100% DOM is when I need to make things from scratch or do XML data manipulation.