[jhugo]

> It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

I agree with everything you said about decentralised MUCs, and for the "communities" [1] use-case it will be great when XMPP eventually gains this capability (sadly, a lot of effort in IM protocol development has been split to a shinier new thing these days, so I guess progress is slower everywhere than it could be).

I also agree that decentralised MUCs are inherently complex. What I don't agree with is all the other reinventing of the wheel in Matrix.

If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

[1] My use case for XMPP has always been more like an open WhatsApp/Signal replacement than an open Discord replacement, where the largest MUC is a few people who all know the server operator personally, so I tend not to notice this missing feature — but I completely agree that it is a missing feature.

[Arathorn]

> If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

Yes, we have always been completely transparent in why we (the Matrix team) built on new foundations rather than extending XMPP. (In practice, laying out our reasoning hasn't been productive however, as XMPP folk just see it as an attack). We started Matrix in 2014 after a few years of shipping XMPP-based messaging apps for telcos (ejabberd + XMPPFramework + Smack + xmpp.js as a stack). Some of the main reasons we chose to start over were:

* We wanted to provide completely different primitives: replicating room history & state rather than passing messages; HTTP RPC as the primary transport rather than XML streams. We wanted to remove the concept of DMs entirely, and instead model everything around replicated chat rooms. We wanted it to be impossible to communicate without sharing ownership of the conversation between the servers. All these are fundamental architectural changes.

* We wanted a single monolithic spec that defined the features available in the protocol for a given version. There should only ever be one official way to do each operation, rather than competing alternatives. (Matrix is still completely extensible though: anyone can propose new changes to the spec as Matrix Spec Changes, and all the APIs support extensible data. However, which MSCs make it into the spec itself is up to the Spec Core Team - a 50/50 mix of original Matrix core team members and folks who joined subsequently from the community).

The reasons were categorically not about "extracting value", but trying to build a successful protocol for everyone who wants to use Matrix - whether that's the core team who nowadays work at Element, or any of the hundreds of other independent companies building on Matrix. It's true to say that we wanted more control, though, given that in order to produce a single monolithic spec document you necessarily need more coordination over "what the protocol is".

Now, just as you could have built DVCS semantics on top of Subversion (as for instance SVK did) - so you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility. Or alternatively you start a new stack (like Git did) with different governance, and built from the ground up to have totally different semantics and behaviour.

So, to be crystal clear: this was not a case of NIH - but just trying to build something with a completely different design to see if it worked out better. And just as the world can support (say) SVG and Canvas as completely different ways of drawing pretty graphics in web browsers, then so can the world support both XMPP & Matrix as completely different ways of doing open real-time communication on the net.

Edit: and in terms of our transparency and where this was all discussed: Matrix evolves transparently via https://matrix.org/docs/spec/proposals - and discussions of the rationale back in the day can be found in the history of the respective public Matrix rooms such as #matrix-dev:matrix.org. (We don't use mailing lists to develop Matrix; we use Matrix.)

[jhugo]

Thanks for the detailed reply.

> you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility.

All of these except decentralised MUC are implementation details, and relatively minor ones at that. Nobody using Matrix needs to know or care what the underlying transport is [1] or whether it uses XML or JSON. So calling them out as design goals that led to the genesis of Matrix strikes me as slightly strange, since they have basically zero effect on the utility of the system.

Wanting a single monolithic spec is a perfectly reasonable desire, but there's no reason that couldn't be an overlay of the XMPP specs — an agreed set of extensions that all compliant clients implement — giving the best of both worlds. (This already exists for XMPP, in fact, in the form of yearly compliance suites, the most recent being [2].)

Are there other strong reasons why Matrix could not have simply been a decentralised MUC extension to XMPP?

[1] Of course, there has been an HTTP transport for XMPP since 2003, so even if we accept that transport does sometimes matter to the user — e.g. in restricted networks — this wouldn't be a good justification for starting from scratch.

[2] https://xmpp.org/extensions/xep-0459.html