[zmix]

I don't know what XPath 3.1 implementation you use, but the two major implementations, I have at hand, SaxonEE and BaseX, both do not understand your code. You write:

  for-each(normalize-unicode(upper-case(json-doc('x.json'))) => tokenize("\s+"),
    function($a) {
      let $a := $a * 10
      load-xquery-module('abc'):some-func(
        function-lookup($a, 1)(array:map($a, function($b) {
          let $c := unparsed-text-lines($b)
          trace($c)
          if ($c) {
            return xml-to-json($b)
          } else {
            error('This is an error')
          }
        })) 
      }
  )

1. fn:json-doc(), by default and typically, returns either an XPath map or array datatype. Therefore fn:tokenize() can not be used with it. Just as fn:normalize-unicode() and fn:upper-case() can not (both take a string as input) The error is: [FOTY0013] Items of type map(*) cannot be atomized.

2. in your anonymous function 'function($a) {...' you use a 'let' expression without 'return'. This is illegal.

3. As is the use of a colon ':' after fn:load-xquery-module(). The colon seperates prefixes from namespaces. Did you mean the question-mark '?'? That would make sense, since fn:load-xquery-module() returns a map. But then 'some-func(...' would not work out, since the returned map has two keys: "variables" and "functions" and 'some-func()' would be referenced in another map, which is the value of the 'functions' key.

4. Calling fn:function-lookup() (as the parameter to some-func()) with a value, that must be a string ($a), which you then multiply with an integer (10)) will already have errored out (multiply string with integer is not possible), but even if this would be possible, the next error would arise since the first parameter to this function must be an xs:QName type, which a number (or string), clearly, is not.

5. The function, you look up in the external module takes an array:map() function as first parameter. Such a function does not exist in the XPath 3.1 standard (see https://maxtoroq.github.io/xpath-ref). You may have studied a version of the specification, which was written, before the array functions have been finalized, which was in 2017. That could mean, that what you wanted to express, would now be array:fold-left(), which is how a map() function is being called in XPath. However, that function takes three parameters.

6. Again, this is not valid XPath grammar:

    let $c := unparsed-text-lines($b)
    trace($c)
    if ($c) {
      return xml-to-json($b)
    } else {
      error('This is an error')
    }

It would need to be:

    let $c := unparsed-text-lines($b)
    return (
             trace($c)
           , if ($c)
             then xml-to-json($b)
             else error('This is an error')
           )

Also, $c can not evaluate to a boolean ([FORG0006] Effective boolean value not defined for xs:string+), since it is a sequence of strings. Of course, such things could be implementation dependent...

I don't want to go any further, since I can not totally recapitulate, what your code is supposed to do. It may be some 'blackhattish' dark magic, that fucks up some engines, but the engines I use do not even go through with compilation, due to the invalid XPath code. As is, your code does not make much sense. At least, if we talk about XPath 3.1.

Your bashing of XPath 3.1 (and also 2.1) makes no sense either, since you seem to use a completely non-standard processor, with a different syntax and functions, that behave very differently from XPath 3.1, or, even worse, did not understand the language.

Many of the changes to XPath, starting with version 2.1, stem from the fact, that XPath became a subset to XQuery, a fully functional and declarative programming language, that satisfies the need, to query XML documents as they would be databases. This was done in order to satisfy the needs of non-programmers, like in the digital humanities, the publishing industry, etc.

XQuery 3.1, as a superset of XPath 3.1, is a language made in heaven! Nowhere is it that simple, to work with (X)HTML, JSON and XML documents as here! A fully functional, declarative language, that has templating (think handlebars, moustache) built in as an integral part of the language, where you can just intermix you XML code with program code, with easy error tracing (stateless!), quick "to production" development cycles and a painless approach to anything XML!

XML is one of the most misunderstood technologies in our industry and the only solid document technology, I know of. Sadly, also based on this misunderstanding, a whole generation of developers has evolved, which listened to those, who jumped the hypetrain of XML, just to realize, that a document format may not be the best tool for the job (RPC, configuration files, etc.). And instead of admitting to themselves, that they were wrong, they accuse the technology, they abused, teaching the kids to make their lifes more difficult. And these kids now are in charge of browser development, etc. And adding to that, your lightheaded comments do not really help the issue.

[orf]

The code sample is illustrative and shows off as much of the crazy, unneeded dynamism as possible rather than something that would actually work.

> Your bashing of XPath 3.1 (and also 2.1) makes no sense either, since you seem to use a completely non-standard processor, with a different syntax and functions, that behave very differently from XPath 3.1, or, even worse, did not understand the language.

Granted, it’s been a few years since I’ve looked at XPath but I feel that I know it quite well. xcat is a testament to that.

> This was done in order to satisfy the needs of non-programmers, like in the digital humanities, the publishing industry, etc.

This makes no sense. “We made a weird programming language to satisfy the needs of non-programmers”? No, if anything the original xpath was pretty well positioned to be consumed and used by non-programmers. The current, not so much.

The current xpath/xquery language is poorly supported, mostly ignored and incredibly over engineered to the point where it’s almost comically unfit for purpose. Sorry.

> XML is one of the most misunderstood technologies in our industry

It’s pretty well understood and has valid use cases. However it had a history of overly complex, over engineered tooling created by a committee and stuffed full of acronyms.

This quite rightly puts people off, and there are just better formats and technologies to use that isn’t encumbered with XML baggage.

[zmix]

> The code sample is illustrative and shows off as much of the crazy, unneeded dynamism as possible rather than something that would actually work.

You can not make up fantasy code, in a language, that tries to look like the actual thing, but is not, and then complain about the language not working. Your code is not working, since that is not XPath code.

>> This was done in order to satisfy the needs of non-programmers, like in the digital humanities, the publishing industry, etc.

> This makes no sense. “We made a weird programming language to satisfy the needs of non-programmers”?

It's not weird. I am one of these people, and I enjoy it greatly! As an example for some (functioning) XQuery code, everybody is invited to check this: https://gist.github.com/joewiz/6762f1d8826fc291c3884cce3634e... I don't think, that is weird. Or what about this:

  for $contact in $contacts/contact
  where $contact/familyname/data() = "Smith"
  group by $key := $contact/zip
  order by $key
  return <group>{ $contact }</group>

which will return all contacts named 'Smith', placed in the same 'group' element as long as they live in the same location. Not weird at all! But then, this may be a matter of taste.

> The current xpath/xquery language is poorly supported, mostly ignored and incredibly over engineered to the point where it’s almost comically unfit for purpose.

As for ignored and unsupported, that has mostly psychological, social and historical reasons (the needs of programmers vs. the needs of power-users, people who fell for the hype-train and then were turned off, etc.). As for "over-engineered" my experience is, that XQuery is extremely lean. One may argue about some of the datatypes (i.e. dates and times), but they were requested by database/enterprise people. XML is a technology, that had a lot of interest groups, who all wanted their share. The nice thing is: you don't need to use it, if you don't require it. But those, who do, they are happy. Also, these datatypes are not XPath/XQuery, they are XML Schema. What would your example for "over engineering" in XPath be? I am really curios!

> It’s pretty well understood and has valid use cases.

Again, my experience is very different. Most people do not know, whether to "push" or to "pull" when writing XSL-T, which is a strong indicator for them not having understood, at least, XSL-T. They just use it as a programming language and start complaining. Then there are those, who compare it with JSON, which is comparing apples to airplanes. They call it "verbose" while not realizing, that a complex data format, that implies a lot of logic, requires simple tools (XPath one-liner, anyone?), while simple datastructs require much more logic on the side of the programmer. Yes, something as simple as JSON is a low hanging fruit, just like "make money fast". And then you realize the small print. In XML you start lowly, just as in XPath. No need to type anything. Just code on. Do the typing before production release. The rest comes over time, like everyhwere.

Verbosity really happens on the code and the overly complex toolchains (just think ECMAScript and all the difficulties, that stem from combining HTML with JSON, in order to be somewhat semantic)

> However it had a history of overly complex, over engineered tooling created by a committee and stuffed full of acronyms. This quite rightly puts people off, and there are just better formats and technologies to use that isn’t encumbered with XML baggage.

Well, lazy people, who want to speak a foreign language without learning it (best example is your XPath code). I seldomly read the W3C specs, only if I can't help myself any further. There are some nice books on every of these technologies, which are pretty simple to understand. However, one needs to read them, rather than just "coding on".

I do not doubt, that you are a capable programmer. However, judging by your code example, you got no clue about the XPath language. You may know how to abuse functions in a language, that access a server, and I guess, most of these attacks are pretty standard and do not require deeper knowledge of XPath. It's just the functions, that offer access.