[Andrew_nenakhov]

> XML is needlessly complicated

For messaging purposes, it is absolutely identical to JSON in this regard. Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.

Understand this: xml is meant to be read by computers, not humans.

> The criticisms linked are all valid, and there are good reasons it's being abandoned

It's 2022 and there is still nothing better for federated messaging.

[wusspuss]

>For messaging purposes, it is absolutely identical to JSON in this regard.

There's no "JSON mindset" to produce anything comparable to this crime against bandwidth and code brevity:

  <stream:stream from="draugr.de" id="{ID}" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">

Being sent on every connection for no reason, thrice (!)

>Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.

Even XMPP, with XML, could do just fine without namespaces. The only place where namespaces are the differentiating factor in practice is <query/>'s and <x/>'s. Couldn't they just be be called <mam_query/> or whatnot? Oh, and namespaces specify the version. Like for ack's they specify the version 3 every time. But it's something that has to be (and is) clarified when enabling said acks anyway.

[hot_gril]

> xml is meant to be read by computers, not humans.

Online sources say it's meant to be both human and machine readable. It would be less bloated if it were just for machines. Seems like a generalization of HTML, which is meant to be hand-crafted in some cases.

Even if it were for machines only, it'd still need a human-friendly way to manipulate and debug with, and there isn't one. Protocol buffers have the text version for humans and the serialized binary for machines, plus the generated getters/setters for code. XML is a pain to deal with no matter how you abstract it.

JSON with namespaces would just mean another key-value pair in the dict or something. But I'm not sure why that's needed anyway.

> It's 2022 and there is still nothing better for federated messaging.

True, but federated messaging isn't in style anymore, and honestly I hardly even used XMPP. Was working on my own protocol 2 years ago that I truly believed was the best, and had a working reference impl, but had to abandon it. Got 2 jobs already.

[Andrew_nenakhov]

Unless your XMPP client is a human, XML in your XMPP connection is meant to be read by a computer.

I work with XMPP for many years and approximately zero problems we face would be solved by switching to JSON or whatever.

This "JSON is better than XML" is a modern version of old religious debates like Gnome vs KDE, Pascal vs C, Christianity vs Islam.

[hot_gril]

Only if you switch to JSON the way they describe in the joke XEP https://xmpp.org/extensions/xep-0295.html where they force XML into JSON form instead of properly redesigning the messages.

I think most people agree that XMPP messages are hard to read during debugging, and this is somehow not a problem in any of the JSON-based protocols.

[Andrew_nenakhov]

No. Please, stop it.

XML is not a problem. Parsing it is not a problem. Reading it is not a problem, too. None of the problems ever faced by us in the development of XMPP software would have been solved if it was switched from XML to JSON, nor made even a slightly easier to solve.

[wusspuss]

Literally half the OP article along with half the comments here explain how XML is a problem, how it is not equivalent to any other serialization format, etc. IMO the funniest example is stanza ids. That problem wouldn't exist even if they just used said XML in a remotely sane manner. But they really wanted to go all in on it. It's the XML mindset, the need to complicate things.

[easrng]

JSON with namespaces exists, it's called JSON-LD, and it's used by ActivityPub among others.

[andrewshadura]

> It's 2022 and there is still nothing better for federated messaging.

That is untrue. Matrix is undoubtedly much better.

[jhugo]

No matter how many times you say that, it still won't be true. A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear. Matrix is basically a case study in how not to design a federated chat protocol. It's bloated, it ignores all prior art and reinvents the wheel everywhere it can, and its excessive complexity means there is unlikely to ever be a wide ecosystem of good-quality implementations (especially of servers). Just because they made a foundation and threw around the word "standard" a lot doesn't mean it deserves to be the standard for chat protocols.

[eternityforest]

Matrix might have bad core tech, but they are one of the only groups working on the biggest issue with federation.

Tying identity to a server is insane in a decentralized environment. Decentralization is all about keeping Google sized companies from controlling things. But small companies come and go all the time, and nobody trusts them to be around in 10 years.

They aren't solving the SSL problem though. Mumble has a good solution their with their trust on first use connections. Self hosted really should ideally not need a domain name if you want individuals to host it. It should be something you can literally buy a $20 preconfigured hosting node, plug it in, and go.

Tox seems to be most promising of current ones, it just needs to fix it's mobile data use issues.

[Andrew_nenakhov]

> but they are one of the only groups working on the biggest issue with federation.

This issue you are talking about is reliably solved ny owning your own domain. You can self-host it or use the commercial provider for it, either way you can change provider and retain your address.

And regarding these matrix guys, they are so overtaken by a NIH syndrome [1] that they couldn't even follow a common URI syntax. Are you really sure they are best fit to develop standard protocols of any kind?

[1]: https://en.m.wikipedia.org/wiki/Not_invented_here

[eternityforest]

I have no idea who is the best fit to develop standard protocols... the bar is pretty low for P2P at this point.

There's very few projects that look at all interesting since the BitTorrent days, most everything is blockchain focused, and aiming to completely replace centralized at all costs with no ability to use central servers, even if it performs badly.

Owning your own domain has some disadvantages though. Especially with traditional federation. For one thing, your RasPi at home is probably not as reliable as Google cloud, and neither is a $5 month VPS.

Now you've got a single point of failure, that's on consumer hardware, with no professional IT staff managing it. I don't exactly want to pay for something that's less reliable than Gmail and WhatsApp, since an outage at the wrong time could hurt me way more than Google spying is likely to.

The only way I think self hosted makes sense for a typical consumer is if it can seamlessly transition to fallback servers, and you can talk offline on a LAN, and it doesn't rely on domains.

Like, I should be able to put a DHT record that says "I have linked accounts at these servers, try mine if you can, but if that fails here's a backup that we can meet at, that forwards to my main server".

[Arathorn]

https://spec.matrix.org/v1.2/appendices/#matrix-uri-scheme details Matrix’s IANA-registered RFC3986 compliant URI scheme, if you’re interested.

[Andrew_nenakhov]

Yeah, for some reason common user@server wasn't good for you. Likely, because of an envious desire to have handles that look like @twitter

That's what NIH syndrome does to people.

[jhugo]

A $20 preconfigured hosting node could easily include a domain name. Tying identity to a domain name uses the existing, actually decentralised, widely adopted since ~40 years, infrastructure of the DNS. Instead of throwing existing technology out and trying to build everything from scratch (which we should see clearly for what it really is: an attempt to retain more control), Internet standards like XMPP build upon what came before.

[IYasha]

Domain name isn't your property and can be taken away on a whim. It is BAD for ID. It's NOT in your possession, you only rent it. If your domain gets taken away, unpaid, hacked, banned - you're done. Eventually I even stopped using DNS and feeding domain registrators. At least because I hate paying for things that should be mine.

[sam_lowry_]

There are many cloud instances below 5$/ month and domain names in that price range/year.

[wusspuss]

>No matter how many times you say that, it still won't be true. A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear.

Maybe judge not by reading protocol description but by trying and using said protocol, like, chatting with people a lot? With the latter approach you'll notice a very handy feature of Matrix - not losing messages.

[jhugo]

I've experienced lost messages multiple times with Matrix, but I suspect this was due to the shitty clients rather than the protocol.

Not losing messages is also a feature of XMPP. In the distant past, especially before stream management, various kinds of network issue could certainly cause lost messages. These days, almost all implementations support the necessary features to make lost messages virtually impossible (of course, being federated, it's always possible that e.g. the remote server is permanently shut down just after successfully accepting your message, but that risk exists in all protocols including Matrix.)

[Andrew_nenakhov]

Btw in Xabber we've decisively solved the message loss problem by introducing an extension that forces a client-set message id on a message and expects a server-set and a timestamp in a confirmation receipt.

[jhugo]

Is this extension on the XMPP standards track? From my perspective, the message loss problem has been solved for some time using only standard extensions.

[andrewshadura]

> A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear.

You're right, Matrix immediately stands out as what XMPP should have been from the beginning.

[joepie91_]

> Matrix is basically a case study in how not to design a federated chat protocol. It's bloated, it ignores all prior art and reinvents the wheel everywhere it can, and its excessive complexity means there is unlikely to ever be a wide ecosystem of good-quality implementations (especially of servers).

Hi, I've worked on client and server implementations of both XMPP and Matrix. This is a wildly inaccurate - but unfortunately common - view of things.

It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

(Sidenote: if there's one thing I've learned over the years, it's that somebody talking about "bloat" is a huge red flag that they don't have a very deep understanding of something. If you have a genuine complaint about unnecessary complexity, you should be able to point it out specifically, instead of a handwavy term like that.)

Anyway, to get back to the point... much of the complexity in Matrix comes from a single design objective that differs from XMPP: it implements decentralized rooms instead of centralized ones. That is, rooms are not dependent on a single server for their continued existence, they will remain available regardless of server availability.

This might seem irrelevant, but it solves a huge problem that XMPP has been struggling with for years, and that IMO is a major reason why its MUCs never became widespread: the problem of putting trust in a server operator for the continued existence of your community.

Every single time you create a MUC in XMPP, you are essentially betting on the continued existence of whatever server you are creating that MUC on. If it ever goes away, so does your community - and unlike with a list of private contacts, it's hard to rebuild a community. That's a tremendous cognitive overhead for starting a community! And one that most non-techies - understandably - do not want to have to deal with. Trust is hard.

Matrix doesn't have this problem because of its decentralized rooms; it doesn't matter how many servers go away, as long as the homeserver of at least one room participant still exists, the room continues functioning as it always has. That means that you create a room "on Matrix", not "on a specific homeserver". That's much easier for users to deal with.

The tradeoff is that unlike with XMPP, where you have a single authorative server for handling things like access control and moderation and membership, this needs to function entirely decentrally in Matrix - so now, the network is suddenly a distributed system that needs to handle state synchronization and everything that that entails.

And that is where the server-side complexity in Matrix comes from. It is necessary, unavoidable complexity if you want to solve the problem of "where do I put my community" in a way that actually works for regular users. It sucks, but there simply is no simpler way to solve this problem.

The reason XMPP doesn't have this complexity is because it doesn't even try to solve this problem. If the rumoured decentralized MUCs ever become a thing, they will involve similar complexity.

Edit: Also, it would be nice if y'all recognized that the priority should be to get the general public over to open comms systems, instead of constantly trying to start wars against Matrix folks. The constant loud misinformation and attacks are getting very exhausting.

[jhugo]

> It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

I agree with everything you said about decentralised MUCs, and for the "communities" [1] use-case it will be great when XMPP eventually gains this capability (sadly, a lot of effort in IM protocol development has been split to a shinier new thing these days, so I guess progress is slower everywhere than it could be).

I also agree that decentralised MUCs are inherently complex. What I don't agree with is all the other reinventing of the wheel in Matrix.

If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

[1] My use case for XMPP has always been more like an open WhatsApp/Signal replacement than an open Discord replacement, where the largest MUC is a few people who all know the server operator personally, so I tend not to notice this missing feature — but I completely agree that it is a missing feature.

[Arathorn]

> If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

Yes, we have always been completely transparent in why we (the Matrix team) built on new foundations rather than extending XMPP. (In practice, laying out our reasoning hasn't been productive however, as XMPP folk just see it as an attack). We started Matrix in 2014 after a few years of shipping XMPP-based messaging apps for telcos (ejabberd + XMPPFramework + Smack + xmpp.js as a stack). Some of the main reasons we chose to start over were:

* We wanted to provide completely different primitives: replicating room history & state rather than passing messages; HTTP RPC as the primary transport rather than XML streams. We wanted to remove the concept of DMs entirely, and instead model everything around replicated chat rooms. We wanted it to be impossible to communicate without sharing ownership of the conversation between the servers. All these are fundamental architectural changes.

* We wanted a single monolithic spec that defined the features available in the protocol for a given version. There should only ever be one official way to do each operation, rather than competing alternatives. (Matrix is still completely extensible though: anyone can propose new changes to the spec as Matrix Spec Changes, and all the APIs support extensible data. However, which MSCs make it into the spec itself is up to the Spec Core Team - a 50/50 mix of original Matrix core team members and folks who joined subsequently from the community).

The reasons were categorically not about "extracting value", but trying to build a successful protocol for everyone who wants to use Matrix - whether that's the core team who nowadays work at Element, or any of the hundreds of other independent companies building on Matrix. It's true to say that we wanted more control, though, given that in order to produce a single monolithic spec document you necessarily need more coordination over "what the protocol is".

Now, just as you could have built DVCS semantics on top of Subversion (as for instance SVK did) - so you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility. Or alternatively you start a new stack (like Git did) with different governance, and built from the ground up to have totally different semantics and behaviour.

So, to be crystal clear: this was not a case of NIH - but just trying to build something with a completely different design to see if it worked out better. And just as the world can support (say) SVG and Canvas as completely different ways of drawing pretty graphics in web browsers, then so can the world support both XMPP & Matrix as completely different ways of doing open real-time communication on the net.

Edit: and in terms of our transparency and where this was all discussed: Matrix evolves transparently via https://matrix.org/docs/spec/proposals - and discussions of the rationale back in the day can be found in the history of the respective public Matrix rooms such as #matrix-dev:matrix.org. (We don't use mailing lists to develop Matrix; we use Matrix.)

[jhugo]

Thanks for the detailed reply.

> you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility.

All of these except decentralised MUC are implementation details, and relatively minor ones at that. Nobody using Matrix needs to know or care what the underlying transport is [1] or whether it uses XML or JSON. So calling them out as design goals that led to the genesis of Matrix strikes me as slightly strange, since they have basically zero effect on the utility of the system.

Wanting a single monolithic spec is a perfectly reasonable desire, but there's no reason that couldn't be an overlay of the XMPP specs — an agreed set of extensions that all compliant clients implement — giving the best of both worlds. (This already exists for XMPP, in fact, in the form of yearly compliance suites, the most recent being [2].)

Are there other strong reasons why Matrix could not have simply been a decentralised MUC extension to XMPP?

[1] Of course, there has been an HTTP transport for XMPP since 2003, so even if we accept that transport does sometimes matter to the user — e.g. in restricted networks — this wouldn't be a good justification for starting from scratch.

[2] https://xmpp.org/extensions/xep-0459.html

[Andrew_nenakhov]

As a person who works with XMPP, I can say with some degree of authority that MUCs are, indeed, an abomination. Everything about them is broken: the roles and affiliations model, and the concept of connecting to it in the IRC-channel style which is a non-starter on mobile clients which can't be connected all the time.

So we actually dropped support for MUCs and created our own Group chat protocol, which even provides a fallback compatibility with the very basic XMPP. Extensibility for the win!

[Andrew_nenakhov]

Lol no. It is a monolithic monster that will never see any other independent server vendor. Matrix is a product with a public API, not a protocol.

[andrewshadura]

This is, again, untrue.

[Andrew_nenakhov]

As jhugo said in a sister subthread, it doesn't become true if you repeat it again and again.

[joepie91_]

> For messaging purposes, it is absolutely identical to JSON in this regard. Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.

It's definitely not identical. There's one major difference, that rarely seems to get mentioned but seems to be responsible for most of the pain:

JSON maps directly to language-native data structures, but XML does not.

The reason for this is that XML distinguishes between child nodes and attributes, but in the basic data structures that most languages have, there's no way to represent this in a conflict-free manner without doing something like `{ attributes: { ... }, children: [ ... ] }`, which is an awkward structure to query.

The result is that when you're working with XML, you almost always end up with a "data API" that's designed specifically for XML (eg. classes representing nodes with getters for attributes and children), whereas with JSON you're generally working with simple nested lists and structs.

This introduces relatively much complexity in what's a "hot path" in terms of development effort - every single time you want to interact with the deserialized data (which is all the time!), you have to deal with an XML-specific data structure instead of treating it as "just another source-agnostic pile of structs/objects".

The result is tightly coupled code that requires the reader to be familiar with the specific XML API/representation being used, instead of it 1:1 mapping from the serialized data to a language-native data structure. That's not good!

While it is true that you are likely to implement a namespace-like mechanism in JSON as well, for open protocols, the difference is that the format itself does not specially define namespaces - which means that the 1:1 mapping always remains true, and it's the protocol developer's choice to eg. reserve a `children` property to contain child nodes, with the understanding that that property then cannot be used for other things anymore.

XML deserializers cannot do this; it's entirely possible for XML to exist that looks like `<tag children="yes"><subtag/></tag>`, and therefore there is no possible 1:1 mapping that a deserializer can use that's guaranteed not to make certain data inaccessible. And since child nodes are not represented as an attribute in XML's data model, you cannot expect protocol designers to avoid a specific attribute either.

Bottom line: I suspect that this seemingly small design difference has far-reaching second-order effects in how deserializers are implemented, and that that is responsible for most of the pain that people experience in working with XML.

[Andrew_nenakhov]

> JSON maps directly to language-native data structures, but XML does not.

So what? It is irrelevant. In all the problems we have ever faced when creating federated chat applications on various platforms, I can't recall a single time when stream format made any difference.

For example, making your messages appear in consistent order on all of your and your chat partner's devices is a real problem. The development of the efficient strategy to achieve this has nothing to do with format, yet, has a much more visible impact on user experience. In fairness, the debate of what is better suited for the protocol development, Message Sequence Charts [1] or Sequence Diagrams [2] has more sense than debating about XML vs JSON.

[1]: https://en.wikipedia.org/wiki/Message_sequence_chart

[2]: https://en.wikipedia.org/wiki/Sequence_diagram