[joepie91_]

> Matrix is basically a case study in how not to design a federated chat protocol. It's bloated, it ignores all prior art and reinvents the wheel everywhere it can, and its excessive complexity means there is unlikely to ever be a wide ecosystem of good-quality implementations (especially of servers).

Hi, I've worked on client and server implementations of both XMPP and Matrix. This is a wildly inaccurate - but unfortunately common - view of things.

It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

(Sidenote: if there's one thing I've learned over the years, it's that somebody talking about "bloat" is a huge red flag that they don't have a very deep understanding of something. If you have a genuine complaint about unnecessary complexity, you should be able to point it out specifically, instead of a handwavy term like that.)

Anyway, to get back to the point... much of the complexity in Matrix comes from a single design objective that differs from XMPP: it implements decentralized rooms instead of centralized ones. That is, rooms are not dependent on a single server for their continued existence, they will remain available regardless of server availability.

This might seem irrelevant, but it solves a huge problem that XMPP has been struggling with for years, and that IMO is a major reason why its MUCs never became widespread: the problem of putting trust in a server operator for the continued existence of your community.

Every single time you create a MUC in XMPP, you are essentially betting on the continued existence of whatever server you are creating that MUC on. If it ever goes away, so does your community - and unlike with a list of private contacts, it's hard to rebuild a community. That's a tremendous cognitive overhead for starting a community! And one that most non-techies - understandably - do not want to have to deal with. Trust is hard.

Matrix doesn't have this problem because of its decentralized rooms; it doesn't matter how many servers go away, as long as the homeserver of at least one room participant still exists, the room continues functioning as it always has. That means that you create a room "on Matrix", not "on a specific homeserver". That's much easier for users to deal with.

The tradeoff is that unlike with XMPP, where you have a single authorative server for handling things like access control and moderation and membership, this needs to function entirely decentrally in Matrix - so now, the network is suddenly a distributed system that needs to handle state synchronization and everything that that entails.

And that is where the server-side complexity in Matrix comes from. It is necessary, unavoidable complexity if you want to solve the problem of "where do I put my community" in a way that actually works for regular users. It sucks, but there simply is no simpler way to solve this problem.

The reason XMPP doesn't have this complexity is because it doesn't even try to solve this problem. If the rumoured decentralized MUCs ever become a thing, they will involve similar complexity.

Edit: Also, it would be nice if y'all recognized that the priority should be to get the general public over to open comms systems, instead of constantly trying to start wars against Matrix folks. The constant loud misinformation and attacks are getting very exhausting.

[jhugo]

> It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

I agree with everything you said about decentralised MUCs, and for the "communities" [1] use-case it will be great when XMPP eventually gains this capability (sadly, a lot of effort in IM protocol development has been split to a shinier new thing these days, so I guess progress is slower everywhere than it could be).

I also agree that decentralised MUCs are inherently complex. What I don't agree with is all the other reinventing of the wheel in Matrix.

If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

[1] My use case for XMPP has always been more like an open WhatsApp/Signal replacement than an open Discord replacement, where the largest MUC is a few people who all know the server operator personally, so I tend not to notice this missing feature — but I completely agree that it is a missing feature.

[Arathorn]

> If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

Yes, we have always been completely transparent in why we (the Matrix team) built on new foundations rather than extending XMPP. (In practice, laying out our reasoning hasn't been productive however, as XMPP folk just see it as an attack). We started Matrix in 2014 after a few years of shipping XMPP-based messaging apps for telcos (ejabberd + XMPPFramework + Smack + xmpp.js as a stack). Some of the main reasons we chose to start over were:

* We wanted to provide completely different primitives: replicating room history & state rather than passing messages; HTTP RPC as the primary transport rather than XML streams. We wanted to remove the concept of DMs entirely, and instead model everything around replicated chat rooms. We wanted it to be impossible to communicate without sharing ownership of the conversation between the servers. All these are fundamental architectural changes.

* We wanted a single monolithic spec that defined the features available in the protocol for a given version. There should only ever be one official way to do each operation, rather than competing alternatives. (Matrix is still completely extensible though: anyone can propose new changes to the spec as Matrix Spec Changes, and all the APIs support extensible data. However, which MSCs make it into the spec itself is up to the Spec Core Team - a 50/50 mix of original Matrix core team members and folks who joined subsequently from the community).

The reasons were categorically not about "extracting value", but trying to build a successful protocol for everyone who wants to use Matrix - whether that's the core team who nowadays work at Element, or any of the hundreds of other independent companies building on Matrix. It's true to say that we wanted more control, though, given that in order to produce a single monolithic spec document you necessarily need more coordination over "what the protocol is".

Now, just as you could have built DVCS semantics on top of Subversion (as for instance SVK did) - so you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility. Or alternatively you start a new stack (like Git did) with different governance, and built from the ground up to have totally different semantics and behaviour.

So, to be crystal clear: this was not a case of NIH - but just trying to build something with a completely different design to see if it worked out better. And just as the world can support (say) SVG and Canvas as completely different ways of drawing pretty graphics in web browsers, then so can the world support both XMPP & Matrix as completely different ways of doing open real-time communication on the net.

Edit: and in terms of our transparency and where this was all discussed: Matrix evolves transparently via https://matrix.org/docs/spec/proposals - and discussions of the rationale back in the day can be found in the history of the respective public Matrix rooms such as #matrix-dev:matrix.org. (We don't use mailing lists to develop Matrix; we use Matrix.)

[jhugo]

Thanks for the detailed reply.

> you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility.

All of these except decentralised MUC are implementation details, and relatively minor ones at that. Nobody using Matrix needs to know or care what the underlying transport is [1] or whether it uses XML or JSON. So calling them out as design goals that led to the genesis of Matrix strikes me as slightly strange, since they have basically zero effect on the utility of the system.

Wanting a single monolithic spec is a perfectly reasonable desire, but there's no reason that couldn't be an overlay of the XMPP specs — an agreed set of extensions that all compliant clients implement — giving the best of both worlds. (This already exists for XMPP, in fact, in the form of yearly compliance suites, the most recent being [2].)

Are there other strong reasons why Matrix could not have simply been a decentralised MUC extension to XMPP?

[1] Of course, there has been an HTTP transport for XMPP since 2003, so even if we accept that transport does sometimes matter to the user — e.g. in restricted networks — this wouldn't be a good justification for starting from scratch.

[2] https://xmpp.org/extensions/xep-0459.html

[Andrew_nenakhov]

As a person who works with XMPP, I can say with some degree of authority that MUCs are, indeed, an abomination. Everything about them is broken: the roles and affiliations model, and the concept of connecting to it in the IRC-channel style which is a non-starter on mobile clients which can't be connected all the time.

So we actually dropped support for MUCs and created our own Group chat protocol, which even provides a fallback compatibility with the very basic XMPP. Extensibility for the win!