[jhugo]

No matter how many times you say that, it still won't be true. A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear. Matrix is basically a case study in how not to design a federated chat protocol. It's bloated, it ignores all prior art and reinvents the wheel everywhere it can, and its excessive complexity means there is unlikely to ever be a wide ecosystem of good-quality implementations (especially of servers). Just because they made a foundation and threw around the word "standard" a lot doesn't mean it deserves to be the standard for chat protocols.

[eternityforest]

Matrix might have bad core tech, but they are one of the only groups working on the biggest issue with federation.

Tying identity to a server is insane in a decentralized environment. Decentralization is all about keeping Google sized companies from controlling things. But small companies come and go all the time, and nobody trusts them to be around in 10 years.

They aren't solving the SSL problem though. Mumble has a good solution their with their trust on first use connections. Self hosted really should ideally not need a domain name if you want individuals to host it. It should be something you can literally buy a $20 preconfigured hosting node, plug it in, and go.

Tox seems to be most promising of current ones, it just needs to fix it's mobile data use issues.

[Andrew_nenakhov]

> but they are one of the only groups working on the biggest issue with federation.

This issue you are talking about is reliably solved ny owning your own domain. You can self-host it or use the commercial provider for it, either way you can change provider and retain your address.

And regarding these matrix guys, they are so overtaken by a NIH syndrome [1] that they couldn't even follow a common URI syntax. Are you really sure they are best fit to develop standard protocols of any kind?

[1]: https://en.m.wikipedia.org/wiki/Not_invented_here

[eternityforest]

I have no idea who is the best fit to develop standard protocols... the bar is pretty low for P2P at this point.

There's very few projects that look at all interesting since the BitTorrent days, most everything is blockchain focused, and aiming to completely replace centralized at all costs with no ability to use central servers, even if it performs badly.

Owning your own domain has some disadvantages though. Especially with traditional federation. For one thing, your RasPi at home is probably not as reliable as Google cloud, and neither is a $5 month VPS.

Now you've got a single point of failure, that's on consumer hardware, with no professional IT staff managing it. I don't exactly want to pay for something that's less reliable than Gmail and WhatsApp, since an outage at the wrong time could hurt me way more than Google spying is likely to.

The only way I think self hosted makes sense for a typical consumer is if it can seamlessly transition to fallback servers, and you can talk offline on a LAN, and it doesn't rely on domains.

Like, I should be able to put a DHT record that says "I have linked accounts at these servers, try mine if you can, but if that fails here's a backup that we can meet at, that forwards to my main server".

[Arathorn]

https://spec.matrix.org/v1.2/appendices/#matrix-uri-scheme details Matrix’s IANA-registered RFC3986 compliant URI scheme, if you’re interested.

[Andrew_nenakhov]

Yeah, for some reason common user@server wasn't good for you. Likely, because of an envious desire to have handles that look like @twitter

That's what NIH syndrome does to people.

[jhugo]

A $20 preconfigured hosting node could easily include a domain name. Tying identity to a domain name uses the existing, actually decentralised, widely adopted since ~40 years, infrastructure of the DNS. Instead of throwing existing technology out and trying to build everything from scratch (which we should see clearly for what it really is: an attempt to retain more control), Internet standards like XMPP build upon what came before.

[IYasha]

Domain name isn't your property and can be taken away on a whim. It is BAD for ID. It's NOT in your possession, you only rent it. If your domain gets taken away, unpaid, hacked, banned - you're done. Eventually I even stopped using DNS and feeding domain registrators. At least because I hate paying for things that should be mine.

[sam_lowry_]

There are many cloud instances below 5$/ month and domain names in that price range/year.

[wusspuss]

>No matter how many times you say that, it still won't be true. A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear.

Maybe judge not by reading protocol description but by trying and using said protocol, like, chatting with people a lot? With the latter approach you'll notice a very handy feature of Matrix - not losing messages.

[jhugo]

I've experienced lost messages multiple times with Matrix, but I suspect this was due to the shitty clients rather than the protocol.

Not losing messages is also a feature of XMPP. In the distant past, especially before stream management, various kinds of network issue could certainly cause lost messages. These days, almost all implementations support the necessary features to make lost messages virtually impossible (of course, being federated, it's always possible that e.g. the remote server is permanently shut down just after successfully accepting your message, but that risk exists in all protocols including Matrix.)

[Andrew_nenakhov]

Btw in Xabber we've decisively solved the message loss problem by introducing an extension that forces a client-set message id on a message and expects a server-set and a timestamp in a confirmation receipt.

[jhugo]

Is this extension on the XMPP standards track? From my perspective, the message loss problem has been solved for some time using only standard extensions.

[Andrew_nenakhov]

It will be, eventually. Though i think that XSF needs a revolt to stop them doing their thing the way they do. From what I get from their council meetings, they feel everything is fine. It is not.

Regarding the standard extensions, the most common way to prevent message loss is by using stream management, however there are edge cases in SM when the messages can still be lost in milti-device scenario. It is not surprising, SM was not meant to prevent message loss per se, just to resume a stream.

The main problem of XMPP on modern mobile devices is this: XMPP is designed to be always connected, while modern OS shut down apps when they are in background. So to make XMPP fit in the modern world we needed to make a transition from a resumption of stream to a synchronisation of state.

[andrewshadura]

> A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear.

You're right, Matrix immediately stands out as what XMPP should have been from the beginning.

[joepie91_]

> Matrix is basically a case study in how not to design a federated chat protocol. It's bloated, it ignores all prior art and reinvents the wheel everywhere it can, and its excessive complexity means there is unlikely to ever be a wide ecosystem of good-quality implementations (especially of servers).

Hi, I've worked on client and server implementations of both XMPP and Matrix. This is a wildly inaccurate - but unfortunately common - view of things.

It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

(Sidenote: if there's one thing I've learned over the years, it's that somebody talking about "bloat" is a huge red flag that they don't have a very deep understanding of something. If you have a genuine complaint about unnecessary complexity, you should be able to point it out specifically, instead of a handwavy term like that.)

Anyway, to get back to the point... much of the complexity in Matrix comes from a single design objective that differs from XMPP: it implements decentralized rooms instead of centralized ones. That is, rooms are not dependent on a single server for their continued existence, they will remain available regardless of server availability.

This might seem irrelevant, but it solves a huge problem that XMPP has been struggling with for years, and that IMO is a major reason why its MUCs never became widespread: the problem of putting trust in a server operator for the continued existence of your community.

Every single time you create a MUC in XMPP, you are essentially betting on the continued existence of whatever server you are creating that MUC on. If it ever goes away, so does your community - and unlike with a list of private contacts, it's hard to rebuild a community. That's a tremendous cognitive overhead for starting a community! And one that most non-techies - understandably - do not want to have to deal with. Trust is hard.

Matrix doesn't have this problem because of its decentralized rooms; it doesn't matter how many servers go away, as long as the homeserver of at least one room participant still exists, the room continues functioning as it always has. That means that you create a room "on Matrix", not "on a specific homeserver". That's much easier for users to deal with.

The tradeoff is that unlike with XMPP, where you have a single authorative server for handling things like access control and moderation and membership, this needs to function entirely decentrally in Matrix - so now, the network is suddenly a distributed system that needs to handle state synchronization and everything that that entails.

And that is where the server-side complexity in Matrix comes from. It is necessary, unavoidable complexity if you want to solve the problem of "where do I put my community" in a way that actually works for regular users. It sucks, but there simply is no simpler way to solve this problem.

The reason XMPP doesn't have this complexity is because it doesn't even try to solve this problem. If the rumoured decentralized MUCs ever become a thing, they will involve similar complexity.

Edit: Also, it would be nice if y'all recognized that the priority should be to get the general public over to open comms systems, instead of constantly trying to start wars against Matrix folks. The constant loud misinformation and attacks are getting very exhausting.

[jhugo]

> It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

I agree with everything you said about decentralised MUCs, and for the "communities" [1] use-case it will be great when XMPP eventually gains this capability (sadly, a lot of effort in IM protocol development has been split to a shinier new thing these days, so I guess progress is slower everywhere than it could be).

I also agree that decentralised MUCs are inherently complex. What I don't agree with is all the other reinventing of the wheel in Matrix.

If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

[1] My use case for XMPP has always been more like an open WhatsApp/Signal replacement than an open Discord replacement, where the largest MUC is a few people who all know the server operator personally, so I tend not to notice this missing feature — but I completely agree that it is a missing feature.

[Arathorn]

> If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

Yes, we have always been completely transparent in why we (the Matrix team) built on new foundations rather than extending XMPP. (In practice, laying out our reasoning hasn't been productive however, as XMPP folk just see it as an attack). We started Matrix in 2014 after a few years of shipping XMPP-based messaging apps for telcos (ejabberd + XMPPFramework + Smack + xmpp.js as a stack). Some of the main reasons we chose to start over were:

* We wanted to provide completely different primitives: replicating room history & state rather than passing messages; HTTP RPC as the primary transport rather than XML streams. We wanted to remove the concept of DMs entirely, and instead model everything around replicated chat rooms. We wanted it to be impossible to communicate without sharing ownership of the conversation between the servers. All these are fundamental architectural changes.

* We wanted a single monolithic spec that defined the features available in the protocol for a given version. There should only ever be one official way to do each operation, rather than competing alternatives. (Matrix is still completely extensible though: anyone can propose new changes to the spec as Matrix Spec Changes, and all the APIs support extensible data. However, which MSCs make it into the spec itself is up to the Spec Core Team - a 50/50 mix of original Matrix core team members and folks who joined subsequently from the community).

The reasons were categorically not about "extracting value", but trying to build a successful protocol for everyone who wants to use Matrix - whether that's the core team who nowadays work at Element, or any of the hundreds of other independent companies building on Matrix. It's true to say that we wanted more control, though, given that in order to produce a single monolithic spec document you necessarily need more coordination over "what the protocol is".

Now, just as you could have built DVCS semantics on top of Subversion (as for instance SVK did) - so you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility. Or alternatively you start a new stack (like Git did) with different governance, and built from the ground up to have totally different semantics and behaviour.

So, to be crystal clear: this was not a case of NIH - but just trying to build something with a completely different design to see if it worked out better. And just as the world can support (say) SVG and Canvas as completely different ways of drawing pretty graphics in web browsers, then so can the world support both XMPP & Matrix as completely different ways of doing open real-time communication on the net.

Edit: and in terms of our transparency and where this was all discussed: Matrix evolves transparently via https://matrix.org/docs/spec/proposals - and discussions of the rationale back in the day can be found in the history of the respective public Matrix rooms such as #matrix-dev:matrix.org. (We don't use mailing lists to develop Matrix; we use Matrix.)

[jhugo]

Thanks for the detailed reply.

> you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility.

All of these except decentralised MUC are implementation details, and relatively minor ones at that. Nobody using Matrix needs to know or care what the underlying transport is [1] or whether it uses XML or JSON. So calling them out as design goals that led to the genesis of Matrix strikes me as slightly strange, since they have basically zero effect on the utility of the system.

Wanting a single monolithic spec is a perfectly reasonable desire, but there's no reason that couldn't be an overlay of the XMPP specs — an agreed set of extensions that all compliant clients implement — giving the best of both worlds. (This already exists for XMPP, in fact, in the form of yearly compliance suites, the most recent being [2].)

Are there other strong reasons why Matrix could not have simply been a decentralised MUC extension to XMPP?

[1] Of course, there has been an HTTP transport for XMPP since 2003, so even if we accept that transport does sometimes matter to the user — e.g. in restricted networks — this wouldn't be a good justification for starting from scratch.

[2] https://xmpp.org/extensions/xep-0459.html

[Andrew_nenakhov]

As a person who works with XMPP, I can say with some degree of authority that MUCs are, indeed, an abomination. Everything about them is broken: the roles and affiliations model, and the concept of connecting to it in the IRC-channel style which is a non-starter on mobile clients which can't be connected all the time.

So we actually dropped support for MUCs and created our own Group chat protocol, which even provides a fallback compatibility with the very basic XMPP. Extensibility for the win!