[MattJ100]

As someone who has been working with XMPP for a long time, I can safely say that most of the criticisms here simply have no material effects in practice. Yes, an ack packet can be 19 characters long. Is that terrible?

It would be just as easy to write a post like this about JSON, Matrix and so on. Both have many warts.

But I really don't much care. No technology is perfect, and there will always be people who dislike choices that other people have made. What matters is what we do with the technology. XMPP has proven itself able to evolve over more than two decades, during which the internet and the way we use it has shifted dramatically, and yet it continues to be relevant as new people discover and adopt it.

It's unfair to describe things like multi-device sync as "afterthoughts" when XMPP was one of the first real-time messaging protocols to have support for multiple concurrent connections of the same account built in, at a time before the iPhone was released, and most people had a single device with wired access to the internet only.

Support for multi-device sync is part of the suite of protocol features required by modern XMPP implementations, these requirements are updated every year to account for the changing technology landscape: https://xmpp.org/about/compliance-suites/

I personally like the simple delivery model of XMPP compared to the "distributed database" model that Matrix pursues. I think there are use-cases for both approaches. Both XMPP and Matrix have bridges between each other and third-party protocols (proprietary and open ones), which makes any attempt to create false competition between them an unhelpful effort. What's important is that both provide openly-documented protocols with open-source reusable implementations, and (in my opinion) most important of all: decentralized open communication networks.

The problems facing the broader adoption of open networks for modern messaging are not protocol problems.

[hot_gril]

I've worked with XMPP for years, and I say it's bad and outdated. The criticisms linked are all valid, and there are good reasons it's being abandoned. Was a great protocol back when it was the open alternative to AIM.

About the XML bloat you've mentioned: The problem isn't just the wire size, it's working with it as a developer. XML is needlessly complicated to deal with in code and super painful to read when debugging. Tooling for it seems lacking too; I even found that iOS's built in XML parser was very slow at parsing large elements, with time/mem used seemingly growing O(N^2). Found that out when the XMPP-using startup I was working for decided to send all their XMPP IQ messages as just wrapped JSON payloads.

About extensions: As cool as those XEPs are, it makes for a fragmented federated ecosystem when so many basic features are extensions. It's not awful when maybe you can't send large files on one server, but when something like auth is broken...

[Andrew_nenakhov]

> XML is needlessly complicated

For messaging purposes, it is absolutely identical to JSON in this regard. Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.

Understand this: xml is meant to be read by computers, not humans.

> The criticisms linked are all valid, and there are good reasons it's being abandoned

It's 2022 and there is still nothing better for federated messaging.

[wusspuss]

>For messaging purposes, it is absolutely identical to JSON in this regard.

There's no "JSON mindset" to produce anything comparable to this crime against bandwidth and code brevity:

  <stream:stream from="draugr.de" id="{ID}" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">

Being sent on every connection for no reason, thrice (!)

>Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.

Even XMPP, with XML, could do just fine without namespaces. The only place where namespaces are the differentiating factor in practice is <query/>'s and <x/>'s. Couldn't they just be be called <mam_query/> or whatnot? Oh, and namespaces specify the version. Like for ack's they specify the version 3 every time. But it's something that has to be (and is) clarified when enabling said acks anyway.

[hot_gril]

> xml is meant to be read by computers, not humans.

Online sources say it's meant to be both human and machine readable. It would be less bloated if it were just for machines. Seems like a generalization of HTML, which is meant to be hand-crafted in some cases.

Even if it were for machines only, it'd still need a human-friendly way to manipulate and debug with, and there isn't one. Protocol buffers have the text version for humans and the serialized binary for machines, plus the generated getters/setters for code. XML is a pain to deal with no matter how you abstract it.

JSON with namespaces would just mean another key-value pair in the dict or something. But I'm not sure why that's needed anyway.

> It's 2022 and there is still nothing better for federated messaging.

True, but federated messaging isn't in style anymore, and honestly I hardly even used XMPP. Was working on my own protocol 2 years ago that I truly believed was the best, and had a working reference impl, but had to abandon it. Got 2 jobs already.

[Andrew_nenakhov]

Unless your XMPP client is a human, XML in your XMPP connection is meant to be read by a computer.

I work with XMPP for many years and approximately zero problems we face would be solved by switching to JSON or whatever.

This "JSON is better than XML" is a modern version of old religious debates like Gnome vs KDE, Pascal vs C, Christianity vs Islam.

[hot_gril]

Only if you switch to JSON the way they describe in the joke XEP https://xmpp.org/extensions/xep-0295.html where they force XML into JSON form instead of properly redesigning the messages.

I think most people agree that XMPP messages are hard to read during debugging, and this is somehow not a problem in any of the JSON-based protocols.

[Andrew_nenakhov]

No. Please, stop it.

XML is not a problem. Parsing it is not a problem. Reading it is not a problem, too. None of the problems ever faced by us in the development of XMPP software would have been solved if it was switched from XML to JSON, nor made even a slightly easier to solve.

[easrng]

JSON with namespaces exists, it's called JSON-LD, and it's used by ActivityPub among others.

[andrewshadura]

> It's 2022 and there is still nothing better for federated messaging.

That is untrue. Matrix is undoubtedly much better.

[jhugo]

No matter how many times you say that, it still won't be true. A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear. Matrix is basically a case study in how not to design a federated chat protocol. It's bloated, it ignores all prior art and reinvents the wheel everywhere it can, and its excessive complexity means there is unlikely to ever be a wide ecosystem of good-quality implementations (especially of servers). Just because they made a foundation and threw around the word "standard" a lot doesn't mean it deserves to be the standard for chat protocols.

[eternityforest]

Matrix might have bad core tech, but they are one of the only groups working on the biggest issue with federation.

Tying identity to a server is insane in a decentralized environment. Decentralization is all about keeping Google sized companies from controlling things. But small companies come and go all the time, and nobody trusts them to be around in 10 years.

They aren't solving the SSL problem though. Mumble has a good solution their with their trust on first use connections. Self hosted really should ideally not need a domain name if you want individuals to host it. It should be something you can literally buy a $20 preconfigured hosting node, plug it in, and go.

Tox seems to be most promising of current ones, it just needs to fix it's mobile data use issues.

[Andrew_nenakhov]

> but they are one of the only groups working on the biggest issue with federation.

This issue you are talking about is reliably solved ny owning your own domain. You can self-host it or use the commercial provider for it, either way you can change provider and retain your address.

And regarding these matrix guys, they are so overtaken by a NIH syndrome [1] that they couldn't even follow a common URI syntax. Are you really sure they are best fit to develop standard protocols of any kind?

[1]: https://en.m.wikipedia.org/wiki/Not_invented_here

[jhugo]

A $20 preconfigured hosting node could easily include a domain name. Tying identity to a domain name uses the existing, actually decentralised, widely adopted since ~40 years, infrastructure of the DNS. Instead of throwing existing technology out and trying to build everything from scratch (which we should see clearly for what it really is: an attempt to retain more control), Internet standards like XMPP build upon what came before.

[wusspuss]

>No matter how many times you say that, it still won't be true. A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear.

Maybe judge not by reading protocol description but by trying and using said protocol, like, chatting with people a lot? With the latter approach you'll notice a very handy feature of Matrix - not losing messages.

[jhugo]

I've experienced lost messages multiple times with Matrix, but I suspect this was due to the shitty clients rather than the protocol.

Not losing messages is also a feature of XMPP. In the distant past, especially before stream management, various kinds of network issue could certainly cause lost messages. These days, almost all implementations support the necessary features to make lost messages virtually impossible (of course, being federated, it's always possible that e.g. the remote server is permanently shut down just after successfully accepting your message, but that risk exists in all protocols including Matrix.)

[andrewshadura]

> A mere few minutes spent looking at the Matrix protocol and a few minutes spent looking at XMPP (if you can get past the fact that it's XML) will make that clear.

You're right, Matrix immediately stands out as what XMPP should have been from the beginning.

[joepie91_]

> Matrix is basically a case study in how not to design a federated chat protocol. It's bloated, it ignores all prior art and reinvents the wheel everywhere it can, and its excessive complexity means there is unlikely to ever be a wide ecosystem of good-quality implementations (especially of servers).

Hi, I've worked on client and server implementations of both XMPP and Matrix. This is a wildly inaccurate - but unfortunately common - view of things.

It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

(Sidenote: if there's one thing I've learned over the years, it's that somebody talking about "bloat" is a huge red flag that they don't have a very deep understanding of something. If you have a genuine complaint about unnecessary complexity, you should be able to point it out specifically, instead of a handwavy term like that.)

Anyway, to get back to the point... much of the complexity in Matrix comes from a single design objective that differs from XMPP: it implements decentralized rooms instead of centralized ones. That is, rooms are not dependent on a single server for their continued existence, they will remain available regardless of server availability.

This might seem irrelevant, but it solves a huge problem that XMPP has been struggling with for years, and that IMO is a major reason why its MUCs never became widespread: the problem of putting trust in a server operator for the continued existence of your community.

Every single time you create a MUC in XMPP, you are essentially betting on the continued existence of whatever server you are creating that MUC on. If it ever goes away, so does your community - and unlike with a list of private contacts, it's hard to rebuild a community. That's a tremendous cognitive overhead for starting a community! And one that most non-techies - understandably - do not want to have to deal with. Trust is hard.

Matrix doesn't have this problem because of its decentralized rooms; it doesn't matter how many servers go away, as long as the homeserver of at least one room participant still exists, the room continues functioning as it always has. That means that you create a room "on Matrix", not "on a specific homeserver". That's much easier for users to deal with.

The tradeoff is that unlike with XMPP, where you have a single authorative server for handling things like access control and moderation and membership, this needs to function entirely decentrally in Matrix - so now, the network is suddenly a distributed system that needs to handle state synchronization and everything that that entails.

And that is where the server-side complexity in Matrix comes from. It is necessary, unavoidable complexity if you want to solve the problem of "where do I put my community" in a way that actually works for regular users. It sucks, but there simply is no simpler way to solve this problem.

The reason XMPP doesn't have this complexity is because it doesn't even try to solve this problem. If the rumoured decentralized MUCs ever become a thing, they will involve similar complexity.

Edit: Also, it would be nice if y'all recognized that the priority should be to get the general public over to open comms systems, instead of constantly trying to start wars against Matrix folks. The constant loud misinformation and attacks are getting very exhausting.

[jhugo]

> It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.

I agree with everything you said about decentralised MUCs, and for the "communities" [1] use-case it will be great when XMPP eventually gains this capability (sadly, a lot of effort in IM protocol development has been split to a shinier new thing these days, so I guess progress is slower everywhere than it could be).

I also agree that decentralised MUCs are inherently complex. What I don't agree with is all the other reinventing of the wheel in Matrix.

If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?

[1] My use case for XMPP has always been more like an open WhatsApp/Signal replacement than an open Discord replacement, where the largest MUC is a few people who all know the server operator personally, so I tend not to notice this missing feature — but I completely agree that it is a missing feature.

[Andrew_nenakhov]

As a person who works with XMPP, I can say with some degree of authority that MUCs are, indeed, an abomination. Everything about them is broken: the roles and affiliations model, and the concept of connecting to it in the IRC-channel style which is a non-starter on mobile clients which can't be connected all the time.

So we actually dropped support for MUCs and created our own Group chat protocol, which even provides a fallback compatibility with the very basic XMPP. Extensibility for the win!

[Andrew_nenakhov]

Lol no. It is a monolithic monster that will never see any other independent server vendor. Matrix is a product with a public API, not a protocol.

[andrewshadura]

This is, again, untrue.

[Andrew_nenakhov]

As jhugo said in a sister subthread, it doesn't become true if you repeat it again and again.

[joepie91_]

> For messaging purposes, it is absolutely identical to JSON in this regard. Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.

It's definitely not identical. There's one major difference, that rarely seems to get mentioned but seems to be responsible for most of the pain:

JSON maps directly to language-native data structures, but XML does not.

The reason for this is that XML distinguishes between child nodes and attributes, but in the basic data structures that most languages have, there's no way to represent this in a conflict-free manner without doing something like `{ attributes: { ... }, children: [ ... ] }`, which is an awkward structure to query.

The result is that when you're working with XML, you almost always end up with a "data API" that's designed specifically for XML (eg. classes representing nodes with getters for attributes and children), whereas with JSON you're generally working with simple nested lists and structs.

This introduces relatively much complexity in what's a "hot path" in terms of development effort - every single time you want to interact with the deserialized data (which is all the time!), you have to deal with an XML-specific data structure instead of treating it as "just another source-agnostic pile of structs/objects".

The result is tightly coupled code that requires the reader to be familiar with the specific XML API/representation being used, instead of it 1:1 mapping from the serialized data to a language-native data structure. That's not good!

While it is true that you are likely to implement a namespace-like mechanism in JSON as well, for open protocols, the difference is that the format itself does not specially define namespaces - which means that the 1:1 mapping always remains true, and it's the protocol developer's choice to eg. reserve a `children` property to contain child nodes, with the understanding that that property then cannot be used for other things anymore.

XML deserializers cannot do this; it's entirely possible for XML to exist that looks like `<tag children="yes"><subtag/></tag>`, and therefore there is no possible 1:1 mapping that a deserializer can use that's guaranteed not to make certain data inaccessible. And since child nodes are not represented as an attribute in XML's data model, you cannot expect protocol designers to avoid a specific attribute either.

Bottom line: I suspect that this seemingly small design difference has far-reaching second-order effects in how deserializers are implemented, and that that is responsible for most of the pain that people experience in working with XML.

[Andrew_nenakhov]

> JSON maps directly to language-native data structures, but XML does not.

So what? It is irrelevant. In all the problems we have ever faced when creating federated chat applications on various platforms, I can't recall a single time when stream format made any difference.

For example, making your messages appear in consistent order on all of your and your chat partner's devices is a real problem. The development of the efficient strategy to achieve this has nothing to do with format, yet, has a much more visible impact on user experience. In fairness, the debate of what is better suited for the protocol development, Message Sequence Charts [1] or Sequence Diagrams [2] has more sense than debating about XML vs JSON.

[1]: https://en.wikipedia.org/wiki/Message_sequence_chart

[2]: https://en.wikipedia.org/wiki/Sequence_diagram

[eternityforest]

I call it a "pseudostandard". When you need extensions to use it at all, and half the clients are missing just enough of them to be useless for half the users.

Plus it's confusing. You have to look through feature matrices instead of just saying "Oh this is version 4, it has everything I need".

[hot_gril]

In my experience, XMPP has rarely "just worked." Always had to find the right client for the server I was using, or at least mess a lot with the settings. It could've benefitted from being more opinionated.

[zaik]

> "Oh this is version 4, it has everything I need"

You probably have not yet written any JavaScript or CSS if you think that's how things work.

[eternityforest]

JavaScript is very much a pseudostandard these days.

Most of the new stuff is invented by Google and rejected by Mozilla and Apple.

If you are in an intranet environment where you only ever target Chrome and Chrome Mobile and pretend the rest don't exist, it kinda does work like that.

[IYasha]

> Most of the new stuff is invented by Google ...to gain even more control over what's left of the Internet. Yeah, everytime I get "your browser is not supported" after loading a shitton of CSS+JS (literally megabytes), I remember Google and their cronies like framework writers and careless site developers. Just because Google added new CSS keyword doesn't mean everyone should drop old ways of doing things.

[wusspuss]

>I personally like the simple delivery model of XMPP

The moment you ask for reliability it stops being simple. Stream Management + Stanza Ids isn't simple, and it shows - XMPP is the only thing I use today where it occasionally happens that a message isn't delivered and I'm not even notified about it.

On a side note I don't believe that this can't be fixed. Maybe if we had end clients checking directly with each other what's been delivered and what hasn't, like actual humans end up doing anyway, I believe it'd be more reliable. Besides instead of relying on the c2s-s2s-s2c cooperation chain to work, it'd rely on just c2c cooperation, giving much fewer variations to test. Just to clarify I'm not talking about a p2p connection but rather e.g. checking the number of stanzas not with the server, but with the interlocutor's client, through the normal XMPP means, like <iq/>s maybe

[MattJ100]

I'm not really sure how to silently lose messages in XMPP these days, if you use stream management (which everyone does[*]).

The last gap I'm aware of was that if you sent a message to someone on a remote server, and the remote server couldn't be connected to, your server would generate a delivery error (i.e. not silent). But* if you went offline before this delivery error reached you, you would never see it. These days such errors are stored in the user's message archive, and available to all your clients for synchronization purposes (so all of them can see that the message failed to deliver).

So with that said, I'm very interested in hearing any details you can provide about how to silently lose messages in modern XMPP. Clients involved, steps to reproduce, etc. Then we can produce actionable bug reports.

[*]: I believe Pidgin is still a notable exception here. Notable mainly because a lot of people still use it, even though its protocol plugin development has stalled somewhat for many years (the project is still being developed, but their focus has been on core/foundation improvements rather than keeping up with changes to modern internet messaging).

[wusspuss]

>So with that said, I'm very interested in hearing any details you can provide about how to silently lose messages in modern XMPP. Clients involved, steps to reproduce, etc. Then we can produce actionable bug reports.

Conversations, draugr.de, can't really say much on steps to reproduce - it worked reliably and then it didn't just yesterday, ironically just as I tried to send the article to a friend to check out.

If I understand the two generals' problem correctly, there always is a way to silently lose messages, with any protocol, it's just that some approaches to not doing that are more reliable than others

[MaulingMonkey]

> there always is a way to silently lose messages

That's a bit overgeneralized.

A simple protocol for message delivery that will not silently fail: resend it (presumably with a unique identifier of some sort to avoid duplication), pessimisticly and loudly pointing out "failure" (really: ambiguity of delivery), until you get a reciept of delivery.

What the two generals problem points out for this protocol is:

1. the sender of the reciept won't be certain that said reciept was recieved

2. the message may be successfully sent before the sender realizes it

3. if communication is permanently severed, the sender may forever be uncertain if the message ever got through or not (but this can be messaged loudly and pessimistically to the end user)

This creates design problems for coordination of consensus for things like database transactions, but a chat client failing to alert to potentially undelivered messages is just bad, not merely running afoul of some fundamental law of information science.

Of course, it's always possible to successfully send and recieve reciept of a message, only for the recipient's server to suddenly catch on fire and explode before backups are made, thus permanently losing any record of said message, but that's more of a "two generals and an assassin" problem, outside the scope of the original thought experiment. It is also, hopefully, a rarer scenario than packet loss.

[topdancing]

> Conversations, draugr.de, can't really say much on steps to reproduce - it worked reliably and then it didn't just yesterday

So it worked fine for some time, glitched one day, you presumably have done zero debugging as you have no logs or error messages to show us and somehow that means that clients "just lose messages".

[anonymoushn]

> The moment you ask for reliability it stops being simple. Stream Management + Stanza Ids isn't simple, and it shows - XMPP is the only thing I use today where it occasionally happens that a message isn't delivered and I'm not even notified about it.

I guess you're not using Discord.

[wusspuss]

>It would be just as easy to write a post like this about JSON,

JSON has no namespaces, even without taking that into account it's already less verbose, it maps nicely into lists and dicts, I don't know a single JSON parser that can leak the user's IP even as an option, let alone by default (not saying there aren't any at all, just no popular ones), doing a binary version of XMPP based on JSON would take as much as switching over to bencode instead of whatever the unsupported hell EXI is. So no, you can't really write a post like this about JSON. Even if you do, it's not gonna be XML that will be proposed as a better solution.

One thing where XML beats JSON is markup. But with XMPP's unholy approach to markup, it's actually JSON that beats XML.

[progval]

> JSON has no namespaces

Causing all extensible JSON-based protocols to eventually reinvent them:

* https://www.w3.org/TR/json-ld11/#compact-iris

* "If new events are added, the event type key SHOULD follow the Java package naming convention, e.g. com.example.myapp.event." https://spec.matrix.org/v1.2/client-server-api/#types-of-roo...

* https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-22... (no aliases, so it's even more verbose)

* https://github.com/OAI/OpenAPI-Specification/issues/1362#iss...

[InfiniteRand]

Namespaces address a real concern that inevitably comes up in protocols dealing with complex data and combining syntax from different standards or sources. That being said it is very easy to make half-assed solutions to the problem of namespaces, and the full-assed solution for namespaces is always a little complex and brittle, so it isn't surprising to me that most people end up going the half-assed route.

[jancsika]

Wait a sec.

By far the most widely used half-assed solution to namespaces is in the HTML5 parser to special case svgs (and maybe other curiosities? Dunno). And if I understand correctly, that happens to be a full-assed specified algorithm that all current browsers use.

That is all in order to avoid the problems that would have come with requiring the full-assed XML namespaces in that case.

If I were a political operative, I would attempt to bog down the XML apologists on this point whenever they attempt to argue that JSON gets as complicated as XML when you try to include namespaces.

[wusspuss]

It's not the lack of namespaces that causes people to reinvent them. Using or not using namespaces is a choice you take. With XMPP it's 100% dictated by the XML mindset. There is NO reason for

  <starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"><required/></starttls>

to specify urn:ietf:params:xml:ns:xmpp-tls other than formalism/a yearn for bloat. There would never be a name collision without this ns and it convenes no useful info.

There's no reason for acks to specify urn:xmpp:sm:3 each time. Yes, here it at least specifies the version, but that's already negotiated when enabling acks anyway.

There's no reason why <query xmlns='urn:xmpp:mam:2' .../> can't be <mam_query .../>.

Namespaces are a choice, sometimes (e.g. in programming languages) reasonable, sometimes (in a serialization format for passing small texts around) not so much.

It is dictated by a mindset, the XML mindset in case of XMPP, and in the case you bring up apparently by the Java mindset, not to say there's much of a difference between the two

[EuAndreh]

> There's no reason why <query xmlns='urn:xmpp:mam:2' .../> can't be <mam_query .../>.

In this example, "mam_" is effectivelly a NIH namespace.

[inputmice]

> Causing all extensible JSON-based protocols to eventually reinvent them:

That's funny because it's true. To add to your list: JMAP (https://jmap.io/) has namespaces too.

[Andrew_nenakhov]

Switching to JSON over XML would solve approximately zero problems that decentralised communication protocol faces. In fact, it would only make things more complicated and less flexible.

[ajconway]

Yet Matrix is gaining traction while XMPP is not. Maybe the key was to have funding, build and run a reference server and quickly iterate with a reference client for all major platforms.

[MattJ100]

> Yet Matrix is gaining traction while XMPP is not.

I'm curious what you base this on. Without a doubt Matrix is gaining traction. It's a relatively newer project, has some new ideas, and there is a lot of buzz around it. But it has far to go before reaching XMPP's levels of achievement - millions of XMPP-connected users via Google Talk, and popular products such as WhatsApp and Zoom built on top of it. That many of these aren't, or are no longer, interested in interoperability or federation is not a protocol problem, and I don't see Matrix solving that.

XMPP continues to be widely used by individuals, organizations and governments who want more control over their communication. Across the XMPP projects and services I'm involved in, I continue to see plenty of new users and developers joining the community.

The reference client/server is definitely a pattern I agree with you on, however. It has served Matrix well, and others too. I've been a proponent of this approach for some time, and detailed my thoughts in the "Products vs Protocols" talk/post at https://snikket.org/blog/products-vs-protocols/

[wusspuss]

>But it has far to go before reaching XMPP's levels of achievement - millions of XMPP-connected users via Google Talk, and popular products such as WhatsApp and Zoom built on top of it. That many of these aren't, or are no longer, interested in interoperability or federation is not a protocol problem, and I don't see Matrix solving that.

But these aren't the protocol's achievements either. These people have so much money they could make IRC popular.

[zaik]

So when does a communication protocol achieve something?

[hot_gril]

One big goal of XMPP was federated messaging (compare to email), and I'm not really seeing it. At best I'm seeing companies building on top of it, resulting in something hardly resembling the open version. Companies are benefitting from XMPP's freedom to build their own platforms, but that freedom isn't being passed onto end users in the vast majority of cases. Because XMPP, with its amorphous nature, doesn't stand up well enough as a product on its own.

XMPP has succeeded in its other stated goals of being highly extensible and useful for derivative work, but I think that's in spite of the protocol. There are just a lot of solid implementations out there, like ejabberd (which afaik WhatsApp originally started on, and I worked with it in a startup).

[jhugo]

> One big goal of XMPP was federated messaging (compare to email), and I'm not really seeing it. At best I'm seeing companies building on top of it, resulting in something hardly resembling the open version. Companies are benefitting from XMPP's freedom to build their own platforms, but that freedom isn't being passed onto end users in the vast majority of cases. Because XMPP, with its amorphous nature, doesn't stand up well enough as a product on its own.

XMPP is a protocol, not a product, so it's not surprising that it doesn't stand up as a product on its own. If someone can come up with a compelling federated messaging product, it's likely that they would base the technical part on XMPP, because virtually all of the protocol work is done, and it's much more elegant and easier to build upon than something like Matrix.

But that's not a protocol problem. The reasons why we still don't have good open federated messaging have absolutely nothing to do with the protocol used for that messaging. They're probably quite similar to the reasons why federation in email has regressed over the last 40 years, from personal/company mail servers back to massive centralisation in the likes of Gmail, despite the protocol remaining exactly the same.

[mnd999]

WhatsApp has traction, discord is gaining it. Matrix isn’t really anywhere.

[IgorPartola]

After trying Matrix as a Discord replacement with a couple of different clients it was pretty terrible. My Discord usage is fairly simple but I run a community for non-technical people. Matrix UX was pretty shit between confusing clients and just the overall clumsiness. I would have loved to introduce a whole community to it but instead I went back to Discord instead.

[mypalmike]

I hardly spend any time using it, but when I do, I find discord's UI to be complex and confusing.

[IgorPartola]

I run a server for a non-tech community with roughly 300 members ranging in age from 20 to 65. Some of them are extremely non-technical, to the point that they say “hey Siri turn on flashlight” instead of finding the button on their phone. Every single one was able to figure out Discord in a matter of minutes and use it daily with zero complaints. I would say that’s a customer success story right there.

[hot_gril]

> Every single one was able to figure out Discord in a matter of minutes and use it daily with zero complaints.

My parents and their friends couldn't figure out Discord after 2 weeks of honestly trying to use it. Biggest sources of confusion seemed to be DMs vs servers, username vs username+number vs nickname, and the sheer amount of stuff onscreen. They reverted to iMessage.

[mypalmike]

The main server I'm on is for after hours gaming with coworkers at a company I contract with. I can certainly find my way to a room and chat. But I can't use my actual name, though requested to do so, because it's too common (names don't have to be unique in discord, for some reason there can only be so many of each. Silly limitation.). And I'm not categorized in the same way as my coworkers in the user list. And then some rooms seem to be for text and others for watching one person broadcast video. The experience is just kind of all around disjointed feeling. Perhaps curation and community style has a lot to do with that though.

[olah_1]

> Yet Matrix is gaining traction while XMPP is not

Considering the way in which Matrix spreads unencrypted metadata (even rooms and participants in the rooms, let alone chat events) as far as possible across the network, this traction is quite insidious.

[wusspuss]

How much worse than XMPP is Matrix in this regard? On XMPP your and your interlocutor's server can see who you talk to

[zaik]

Not a lot better, but in public MUCs your JID is hidden to everyone except the server the MUC is hosted on. Also MUCs can be configured to only send a limited amount of history to new participants.

For this reason I'm somewhat uncomfortable joining Matrix rooms over bridges, since it leaks my XMPP address and participation to everyone forever.

[Zash]

Maybe the key is network effect and enough marketing to bootstrap it?

[ajconway]

It doesn't look like Matrix has reached the point where there is any significant network effect. What I'm saying is that they are building a product, and the protocol is just one component of that product.

[zaik]

The key to interoperability is agreeing on standards. XMPP being about creating a proper Internet Standard and not a product is what made me choose XMPP over Matrix (also simplicity).

[wusspuss]

Not losing messages is pretty key too. But yeah, given enough marketing you can pull anything off. WhatsApp basically doesn't support multi device to this day and is more popular than either (free) XMPP or Matrix could ever hope to be. There's a web version but it only works if your phone is connected. So I guess the phone acts as a server then? Does that mean they found it easier to set it up like that than to make xmpp (which WhatsApp is said to be based on) reliable with multiple devices?

[memen]

That is not true anymore. The latest version of WhatsApp Web can be used without your phone connected. Not sure if it is rolled out globally for everyone yet, as it was fairly recent for me (couple days now).

[hot_gril]

WhatsApp first penetrated countries where people's primary computing devices are phones, and they maybe don't even own PCs. It's not surprising that it became super popular despite its lack of multi-device support (at least until recently). Wasn't just marketing.

[Andrew_nenakhov]

XMPP working on multiple devices is a problem long solved. Not losing messaging is a solved problem too. I can schedule you a demo if you wish.

[wusspuss]

It is not a long solved problem if it takes a demo to prove. If it were long solved it'd be self evident. This argument wouldn't exist to begin with.

A demo is worthless if experience doesn't prove it in the long run. And I have the latter, and it doesn't prove. Yes, I know it's probably me, or more likely my friends doing something wrong. One of them restricted Conversations from running in the background for example, non-techies do all sorts of silly stuff. But then again, it shouldn't break with non-techies either. Nothing breaks with them aside from XMPP. Not even WhatsApp which is based on XMPP afaik

[IYasha]

While being "techy" I too restrict Conversations from running in background. Moreover, it's annoying and dangerous to keep XMPP running (reconnecting, giving out your metadata) while roaming over different networks, so I'm killing it myself. But, aside from tremendously, ridiculously bad UI concept, I find Conversations to be one of the most reliable OMEMO-enables jabber clients for Android.

[dmitriid]

> Support for multi-device sync is part of the suite of protocol features required by modern XMPP implementations, these requirements are updated every year

The problems is that many of these modern features were unbelievably late.

I still remember "The State of Mobile XMPP in 2016", https://gultsch.de/xmpp_2016.html

At the time, 9 years after iPhone had come on the scene, most modern features were still experimental draft proposals implemented by no one.

5 more years later there's maybe one client that implements the modern features (Conversations for Android IIRC), and that's about it.

And it's an unknown how many servers implement those features.

[seba_dos1]

> 5 more years later there's maybe one client that implements the modern features (Conversations for Android IIRC), and that's about it.

...and yet I'm using three different modern clients that work well with each other.

> And it's an unknown how many servers implement those features.

Enough to make me not have to care about it.

[MattJ100]

> And it's an unknown how many servers implement those features

It's not: https://compliance.conversations.im/tests/

Obviously a new user is not expected to fathom what all this means before getting started, which is why there are curated lists of public XMPP services, e.g. https://providers.xmpp.net/ and https://joinjabber.org/ and for private servers there are projects like Snikket where the server software is developed in parallel with apps to ensure a consistent feature set.

[InfiniteRand]

I think the decline of xmpp can be traced to the rise of text messaging and phone messaging in non-open protocols. One factor that made non-open protocols more palatable on the phone than on the desktop is the easier and relatively more secure nature of phone apps (I don't think phone apps are that secure, but I think they are more secure than installing random exe's on your computer). If you're willing to install all of the apps, yeah having everything in one place would be convenient, but there's only two or three apps you care about so it's not a huge thing. Moreover most of the phone apps liked to keep things closed. That's just my thinking.

[hot_gril]

It's also branding. Regular people don't understand clients being separate from protocols being separate from servers. Multiple users sorta need to be on the same XMPP server to talk to each other (yeah there's federated mode but it's a mess), and in modern times there's no way that server isn't run by a huge company that also distributes its own client. And there's no way, controlling both the client and the server, such company will stick to pure XMPP instead of adding tons of proprietary stuff on top to improve the experience and establish control. Hence WhatsApp, FB Messenger, etc were born?

[TedDoesntTalk]

The protocol can be perfect. But unless there are great clients for it on multiple different platforms, and those clients implement all the awesome optional protocol features, it just doesn’t matter.

* Screen sharing * File sharing * Sync across devices * Group chat without fuckery * Ability to delete and edit messages * Notifications of new messages

I know XMPP supports all of this and more. But find a cross platform client/server that implements all of it?

Then look at the clients for matrix, telegram, signal, teams, whatsapp, and others.

[olah_1]

> I know XMPP supports all of this and more. But find a cross platform client/server that implements all of it?

That is what Snikket is trying to be. https://github.com/snikket-im

[hot_gril]

I forget also how this works when federation is involved. Do both servers need to support those extensions too? I do recall even federation of servers with basic features being painful due to differences in signing and encryption modes.

Btw, iChat used to support all those features, but only when the other side was iChat too.

[xeeeeeeeeeeenu]

While the technical problems were certainly one of the main factors contributing to the decline of XMPP, I think that branding was also a big issue. They had a perfectly good name that was gaining traction ("Jabber") and then they suddenly changed it to XMPP, confusing everyone in the process.

[MattJ100]

Yeah. This wasn't an intentional thing by the community. The "sudden change" was due to the acquisition of Jabber Inc. by Cisco, which included the "Jabber" trademark. There is a trademark agreement with XMPP Standards Foundation to allow open-source projects to apply to use "Jabber" as part of their name. However, unsurprisingly, the community around the protocol rallied around "XMPP" as the safer term, while Cisco moved forward with their "Jabber" product.

I think it has taken too long for the community to realise what an asset a good brand can be, and that it's hard to rally normal users around a protocol and an ecosystem rather than a product. I think that's growing to change though, with a number of user-focused projects around the ecosystem (including Snikket, the one I'm working on).

[tcfhgj]

I can´t find anything regarding Support for multi-device sync there. I don´t know any client which even claims to support something like this.

I only know about MAM which doesn't work with E2EE... and depends on the individual chatroom's server.

> Both XMPP and Matrix have bridges between each other and third-party protocols (proprietary and open ones), which makes any attempt to create false competition between them an unhelpful effort.

Bridges don't really work with E2EE anymore...

[Andrew_nenakhov]

> MAM which doesn't work with E2EE...

Soo first you want E2EE so nobody can tap into your conversations, and then you want to receive text of you messages from the server archive? :hmm emoji:

[Semaphor]

Shouldn’t that be no issue as long as either both clients use the same key, or the messages have been encrypted for both keys?

[Andrew_nenakhov]

It is not an issue at all. I was just highlighting a nonsense criticism about message archive and e2ee.

[tcfhgj]

no, I just want multi device sync like what WhatsApp and Matrix offers

[Andrew_nenakhov]

That is easily possible with XMPP and does not really have anything to do with 'MAM supporting E2EE'. For Message Archive all messages are equal, encrypted or not.

[tcfhgj]

then tell me with what constellation of apps this actually works

[Andrew_nenakhov]

Web/iOS versions of Xabber do it quite well, complete with videocalls and screensharing, etc.

Android version is in rewrite (decided to dump existing app because of tech debt issues), but I have zero reasons to doubt that it will work (hopefully by august).

iOS version is not yet released on Appstore because we need to work out some more issues (not too many), but I can give you access to it, if you're interested.

[zaik]

What did you try? I'm using Conversations on mobile and Gajim on various workstations and they sync nicely (I think?).