[danpalmer]

This is something that many chat apps get wrong and I'm not sure this article is moving in the right direction. The UX is fairly clear in my mind:

1. All up-to-date clients should be displaying the same message order. 2. A single client should not send messages in the wrong order.

Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state even if that means amending history. Why? Because the humans reading it will be confused otherwise! An app getting more data is something we intuitively understand, but if my client shows something and yours shows something else, we will conclude different meanings from it.

Additionally there are some clients that treat each message input by the user as a retriable thing in isolation, which is also clearly incorrect. If I send two messages and the first fails to go through, I almost certainly don't want to retry the second until the first has gone through, otherwise my client has literally sent out of order messages! I use Beeper for chat and this is one of the most frustrating things it does.

[kelnos]

Agreed, absolutely. I was surprised by the author's aside that mentioned a colleague that disagreed, claiming that it's ok or even useful (what?) that two clients (or even the same client in different situations) could show different message orderings... and acknowledging that this point is debatable.

It's not debatable! There is, actually, in reality, one true message ordering (at this time I think we can safely ignore relativistic effects), and that message ordering should be the one that is always displayed. Our technology and the nature of distributed systems may make it impossible to always faithfully determine what the true message ordering is, but at the very least, the implementation should decide on a single message ordering, and always present that same message ordering. (This does mean that clients connected to different homeservers might see different orderings; this is unfortunate but probably unavoidable. But clients connected to the same homeserver should all see the same ordering.)

Anything else is just terrible, awful, horrible UX, that will ultimately confuse users. And it will also reduce user faith in the system as a whole: if they notice the inconsistent ordering, they will assume the system is buggy and unreliable.

At the risk of sounding way too absolutist: this is not debatable, and anyone who thinks it is, is wrong.

[OskarS]

> (This does mean that clients connected to different homeservers might see different orderings; this is unfortunate but probably unavoidable. But clients connected to the same homeserver should all see the same ordering.)

I don't think it's unavoidable at all, this is exactly the kind of problems CRDTs solve beautifully. If you model your chatting as a CRDT (basically, each message has a Lamport timestamp on it, and you have some sensible way to resolve ties deterministically), then assuming all peers have gotten all messages, the ordering should be perfectly consistent. What may happen is that you type a message, press "enter", and then some message might pop up before that when everyone finally sync, but I think that is acceptible UX, I see version of this in things like Slack and Discord frequently.

All peers having the same message order assuming they all have received the same messages is absolutely required for a chat application in 2024, distributed or not.

[bostik]

> clients connected to the same homeserver should all see the same ordering

Ideally, yes. In the real world you get to deal with such inconveniences as unreliable transport, slow networks, server-to-server communications, eventual consistency, routing glitches, reconnections, clock skew, queues, concurrency, retries, and always, ALWAYS the infuriatingly slow speed of light. Oh yes, also multiple client implementations. For the record, many years ago I was involved in writing both a chat server and a client. (Albeit those were two different projects.)

From pure UX standpoint, you want the client to always show any messages it has sent but has not received back. Even for a single server and two clients, all synchronised to the same clock, you can get ordering conflicts. Let's say that you are sending a message at fixed two-second interval, and the other client is sending messages at non-fixed, power law distribution intervals. That's your happy path.

Now consider the same with dozens, or hundreds of clients across hundreds of different networks, each with their own debatable quality.

You want to see the messages you've sent, so they need to be visible on your screen. Having your own messages disappear into the void and only appear once they have been sent back is terrible UX. So you keep a local order and interleave received messages as they come in. But once you receive the message back from the server, you obviously want to reorder the known quantities. With two clients you will the occasional "jump" where one of your messages is moved to its canonical position. With hundreds of clients each user will see those jumps constantly - and with sufficient volume a decent fraction of their own sent messages can "disappear" at any time from their screen as they are reordered and don't fit on the screen.

Now add lots of bad networks and latency floor in hundreds of milliseconds. Network connections/route fluctuate constantly, so even messages sent by the same client less than two seconds apart can arrive in different order at the server. (The client reconnected between the two, and the message sent over the first connection arrives several seconds later than the other one.) The user is confused, because the server is very clearly showing them their own messages in the wrong order.

For one inconvenienced user the server being wrong occasionally is mildly annoying. But when that can happen to any number of users, concurrently, at any time, the overall effect is outright infuriating.

> this is not debatable, and anyone who thinks it is, is wrong

Your server has a known order it sent the messages out. Any disagreement means the client must be wrong.

Each of the clients connected to your server has a known order in which they sent their messages out. Any disagreement means the server must be wrong.

[deredede]

> Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state even if that means amending history. Why? Because the humans reading it will be confused otherwise! An app getting more data is something we intuitively understand, but if my client shows something and yours shows something else, we will conclude different meanings from it.

That's interesting because I have the complete opposite take and would hard disagree with this. I intuitively understand that if we both write messages at the same time, we will see them in different order. Snail mail has worked this way for centuries, and I very much prefer this to an app silently altering the content as time goes. It is confusing when it happens under my eyes (something moved at the top of the screen while I was reading the bottom, what was it?) and easily leads to missed messages especially in group conversations (my buddy sent a message with a poor connection at 11am, it is retried and sent at 2pm and appears before the lengthy discussion others had at noon).

[moring]

Snail mail has never claimed that a history of all messages, with that history having a current state, exists. If you send a paper letter, you don't have it yourself anymore. You might keep a copy, but that's a _copy_, not the letter you sent.

Messenger apps claim that such a history exists by showing you, well, that history. In the same way, messengers claim that a message order exists, by showing you the messages in that order. If something exists, then it is independent of the viewer. So the assumption that the message order is the same for all viewers is founded in how two people look at physical objects.

[deredede]

Messenger apps don't claim that this history should be global and consistent. The order in which messages were sent and received by my device is a perfectly fine (and I'd say intuitive) history. It is the order people (and their records, if they have some) would have had in mind in the old time.

I take a different conclusion from the way people look at physical objects: since your device (or even my other device) is a different physical object than my device, I'd be wholly unsurprised to find a different order there.

[lmm]

> Messenger apps don't claim that this history should be global and consistent.

The fact that we're talking about multiple people looking at the same chat - the fact that we do conceptualise it as "the same chat" and "the history" - implies that we think of it as a single thing. And I think messenger apps generally nudge us that way - e.g. setting the name of the chat usually sets it for everyone.

> It is the order people (and their records, if they have some) would have had in mind in the old time.

I don't think it is. If I pull my correspondence with person X out of my drawer or file, the only dates I have to order them by are the dates written on the letters - which are the letters they and I (if I keep carbons of the ones I send) wrote them on, not the dates I received them. If they sent me a postcard while on holiday and then a letter after returning that arrived sooner, I'll read them in one order on receipt and in a different order when looking back. Likewise if I have a memo of a phone call with them, that may be from before I received a letter that is nevertheless dated earlier.

[deredede]

> I think messenger apps generally nudge us that way - e.g. setting the name of the chat usually sets it for everyone.

That's a good point - maybe it's actually email that warped my mind.

> I'll read them in one order on receipt and in a different order when looking back

Also a good point, I was thinking more about business communication where the date the letter is received matters. Thinking back on it, I think the main difference is that the messenger apps might happily reorder message before (or while) I read them. And if only one order is to be available, the one of the most use to me for an instant messaging app is the one I received the messages in, but I get how for other use cases it would be different.

[kelnos]

Well, this really depends on the protocol and architecture of the system.

If it's a system where the server is merely store-and-forward, where it forgets its knowledge of messages after the recipients have received them, then sure, your stance is reasonable. The client will decide on message ordering; it can either just display in the order received from the server, or use any timestamps stored in the messages to order them (including possibly reordering if messages arrive out of order). The client has no other source of truth it can draw from, and so different clients may order things differently. (Even in this case, though, for many systems like this I would expect the server to timestamp the messages, and for all clients to honor those timestamps, so in practice everyone should see the same ordering.)

But Matrix is not such a system: each homeserver is the system of record for what messages have been received, and what order they go in. In this case, I would expect all users to see the same ordering, assuming all clients are able to query and receive the full history from the server.

[kevincox]

> I intuitively understand that if we both write messages at the same time, we will see them in different order.

I think you are thinking like a distributed systems designer. I would assume that if you asked 10 "random Americans" 9 of them would assume that someone managed to send their message first and would be surprised if their phone and their friends phone showed them messages in different orders.

[dwattttt]

Many people have had the experience of "I tried to call you, but you were calling me!"; I don't imagine they'd be surprised if "something weird" happened when you both tried to send a message at the "same" time.

[Kwpolska]

I would assume 9 of them would not care either way.

[kelnos]

I don't think you're arguing the same point. I agree with you that when two people write a message at nearly the same time, they may (initially) see those messages in a different order. But the server should decide what the ordering is, and inform the clients, which should update their view of the world.

The ordering the server decides may not be "correct" (for whatever definition of correct matters to you), but what is most important in this situation is consistency.

[zarzavat]

You need to know what ordering the other user is seeing, otherwise dangerous misunderstandings could result, for example you reply "yes" but it's in reply to a different message than the other side is seeing.

If I see that another message has arrived and my message could be misunderstood, I can correct that by sending another message. But if I don't know what ordering the other side is seeing then I don't know if my message is ambiguous or not.

The only way achieve that is to show a consistent ordering to both participants (or to force every message to be in reply to another message but that's too nerdy)

[RaftPeople]

I don't use these apps so maybe my solution wouldn't work, but after reading the article, it seems that having a visual indicator of messages that are new but in the past would be a reasonable solution.

Especially if there were simple controls to flip into a mode that minimizes the ones already seen (collapsed and grey for example) while highlighting all of the ones inserted in the past.

Or, if there are many messages already seen and few inserted into hist, show the inserted ones with a small sampling of the already seen (so the user can anchor to already familiar data in the timeline) along with "72 messages hidden that were previously seen" type of thing in between the inserteds to condense the view.

[lmm]

> Additionally there are some clients that treat each message input by the user as a retriable thing in isolation, which is also clearly incorrect. If I send two messages and the first fails to go through, I almost certainly don't want to retry the second until the first has gone through, otherwise my client has literally sent out of order messages!

I don't think that's clearly incorrect. If you sent two messages you presumably want them to be two messages and they should be retried as such. If what you wanted to send was a single, multi-line message, surely you would have just done that?

[hks0]

Human communications are more naunced than DB transactions. If I forget to mention something important I send a new message rather than editing the already sent one, to make sure I catch their attention. Edits can go unnoticed. Imagine this scenario:

[12:00 / sent] Sell the house.

[12:05 / failed] Please feed the baby.

[12:06 / sent] Oh and the cat too.

Now the receiver's going to sell my cat [example inspired by The Art of Multiprocessor Programming].

[johnny22]

They sure are, but I hate when slack combines my messages when I wanted two separate messages on purpose. If i send two messages, it's because I did it on purpose.

[danpalmer]

Not at all. The separation of messages is part of communication, not me trying to game a network protocol. Maybe it's to emphasise a point, maybe it's to time a joke, maybe it's to send a photo and a text message separately.

[kelnos]

I think you are perhaps not aware of how people use messaging apps in the real world. Many people (myself included, sometimes) will break sentences or sentence fragments into different messages. If the messages are displayed out of order on the recipient's side, it would be pretty hard to understand.

And even in the case where people do tend to send one complete thought per message, it still matters: like maybe I send a message, and then have an extra thought, and send a follow-up message that clarifies my first message. If they are displayed out of order, that will be confusing.

Even if two messages are completely unrelated and completely separate thoughts (honestly this feels like a much less common case than the alternative), messages just should be displayed in the order they were sent, because that's what reflects reality best.

[thaumasiotes]

> If what you wanted to send was a single, multi-line message, surely you would have just done that?

No. danpalmer is correct; the break between messages is an integral part of the communication.

[winwang]

I break up my messages, as do many people.

[adastra22]

Do you do so with the expectation that they might arrive out of order, or one fails?

[Spivak]

Out of order no, failing and having to manually re-send which makes it out of order is acceptable.

[kelnos]

I find that unacceptable and frustrating, personally. If a message fails to send, I want the client to hold back any later messages until the failed message is resolved somehow. It should auto-retry and (hopefully) eventually succeed, or I can manually delete it and "release" the following messages.

[agos]

I do so with the expectation that they should arrive in order if no one fails, but apparently it is "debatable" if this is a reasonable expectation

[shawnz]

How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

[dataflow]

> How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

I believe that's impossible? At least if you design it correctly.

For ordering/interleaving purposes, what matters isn't the time you claim to send the message, it's the time the message is received by the server. If you want, you can display the claimed send timestamp beside the message (and prominently highlight it if it is e.g. out of order, or with a long delay, etc.), but that is irrelevant to the ordering.

The point here is that there should be a single consistent order on the server, and that's what all clients ought be displaying. Any messages not yet acknowledged by the server should be displayed differently so that users are aware they haven't been seen yet, and any messages that arrive before those are sent would obviously get inserted above those.

[lmm]

> what matters isn't the time you claim to send the message, it's the time the message is received by the server

There's no "the" server here. If you use the time the message is received by the server, you'll get different views on different servers, and you may see messages from months ago appearing as new, if connectivity breaks down and is later restored.

[dataflow]

> There's no "the" server here.

Can't you assign every conversation to a single authoritative server for handling?

Also, how large of a time skew are you imagining would exist between different servers? That stuff ought to be accurate to at least milliseconds if not micro...

[lmm]

> Can't you assign every conversation to a single authoritative server for handling?

The whole point of Matrix is to be decentralised. In particular people should be able to keep talking when on different sides of a netsplit, by design.

> Also, how large of a time skew are you imagining would exist between different servers? That stuff ought to be accurate to at least milliseconds if not micro...

The question isn't how much time skew there can be between server A and server B, it's how long they can be cut off from each other over the network, which could be hours at least. (And even when things are working well, a normal ping is a few hundred ms, which is enough to change the order of messages)

[dataflow]

> The whole point of Matrix is to be decentralised. In particular people should be able to keep talking when on different sides of a netsplit, by design.

OK but I still don't see the problem. Even with a fully decentralized system where the servers are just pure relays with no authority, you have two options:

1. Display messages in the order in which they claim to have been sent, or

2. Display messages in the order of arrival

Case #2 is the obvious/uninteresting one, there's nothing to say about it.

Case #1 is what people are saying is so impossible to achieve a global order for, but really, what's the big deal? If a client claims to have sent a message at an unusual time (say, > 10 seconds in the past, or after the app was already quit, or whatever criteria you want to set), then just insert it at that point in the conversation, and visually indicate to the user the discrepancy. And clock skews won't really be much of a problem because messages can easily indicate prior messages in the conversation, so that a mere clock skew doesn't insert them before preceding messages.

What's so hard to make user-friendly/intuitive here?

[shawnz]

Aside from the concerns with decentralized servers that the other poster mentioned, this has the disadvantage that your messages are going to get constantly reordered to not match the intended flow of the conversation when you have poor connectivity, which is a bad user experience

[dataflow]

Wasn't the whole point here that the messages wouldn't get reordered? There would be one definite order that everyone would see. Again, if the message isn't timestamped by the sever, it would need to appear visually differently, so that everyone knows about this. And nobody says the server has to accept messages with arbitrarily delays either.

[shawnz]

My point is that some limited reordering maybe should be allowed, but not too much. That is to say, the problem isn't as simple as just doing it one way or another way. Every approach has some disadvantages.

[dataflow]

I know you were trying to reach that conclusion, but my point was that the design I suggested neither seemed to have the problem you suggested, nor is reordering a necessary outcome, from what I can tell.

[fastball]

You can amend displayed order for humans (what matters for 99% of usage), while still allowing anyone interested to see when the message actually arrived at the homeserver (making the suggested gambit impractical).

[kelnos]

I don't think this is really a problem, at least in the case of client->homeserver connections. The homeserver should not be trusting the client's sent timestamp. The homeserver should consider the message sent at the time it receives it, and the client should know this and update the sent timestamp displayed to the user when it is finally able to connect to the homeserver, and the homeserver acknowledges receipt of the message.

The bigger problem is how to handle homeserver<->homeserver comms. My initial feeling is that the homeserver where a destination room is hosted (let's call this one "A") should have the final say, and if there are people on another homeserver ("B") that have joined the room, and are chatting while there's a break in connectivity between the two homeservers, then A should just append all the messages from B to the end of the record (with correspondingly "later" timestamps) to the "official" record, when B is able to communicate with A again.

But this feels messy too; presumably all of those new messages (a conversation that may have been going on for tens of minutes or hours) would be smooshed in to have their timestamps all appear nearly at the same time? No, that's not great either.

Or perhaps B just shouldn't accept messages for that room while it can't communicate with A? That doesn't seem great either.

[shawnz]

Then instead imagine this: the user really is innocent and just happened to coincidentally send the message right after the start of a long period of poor connectivity (like a flight, or a road trip, etc). If you just allow it to go through after an arbitrary delay, with only a log of the received time for liability purposes, then the user wouldn't have any indication of this scenario occurring.

Wouldn't it be better in that case to show an error so that they can make sure the situation is addressed appropriately?

[Spivak]

You have when the client claims it was sent (so where it goes in the displayed history) and can see when it was received. What else could you possibly do?

[shawnz]

For example, you could reject the message and show the user an error but only if there's a discrepancy of >X minutes. But how much discrepancy should be allowed? I don't know, I only mean to show why I think the solution isn't as simple as it appears

[lmm]

> you could reject the message and show the user an error but only if there's a discrepancy of >X minutes

No you can't, not in a federated and decentralised system like this.

The sender can wait for a read receipt from a given receiver user, if the receiver is willing to make those public. But if the message left client A and didn't arrive at client B, there's no objective fact of the matter about whether the message "was sent" or not.

[lmm]

> Wouldn't it be better in that case to show an error so that they can make sure the situation is addressed appropriately?

If you want a single centralised server then you can set things up that way. Presumably if you're using a setup with multiple servers, and took one of the servers on the flight/road trip, you wanted the people on the flight/road trip to be able to keep talking to each other over that server, even though that server is disconnected from the one in the office.

[kiitos]

> How far back should you be able to amend history?

If user U1 sends a message M1 at time T1, then U1 must be able to modify/delete that message M1, in some reasonable sense, at any conceivable time from T1 forwards.

Any protocol that doesn't support some reasonable form of message modification/deletion in this sense, is a toy protocol, and will never be widely adopted.

[shawnz]

I'm not talking about message editing but rather posting new messages with a backdated time.

[kiitos]

I suppose every message has a few timestamps, including

- The timestamp that the user specified as part of the message

- The timestamp of the server that the user submitted the message to, directly

- The timestamp of the server that first received that message

- Any additional timestamp(s) of additional server(s) that received that message

The user can I guess backdate a time, but that would apply only to the first thing, and none of the others?

[shawnz]

The first one is the only one that can accurately describe where the user intended for the message to land in the conversation. All the others are subject to network delays, so if you order messages by anything other than the first one, you are going to get a bad experience if you try to participate in a busy conversation with a poor connection.

[kiitos]

The first one is entirely un-trustable, because the user-submitted timestamp can be at any arbitrary point in time, from t=0 to t=infinity. So the receiving system can use that timestamp as an important bit of signal, but it can't really treat it as strictly authoritative, at least not if it expects to maintain a coherent set of events overall.

[kelnos]

This shouldn't be an issue for systems where a server mediates communication: the server should be timestamping messages, not the clients.

This could indeed be a potential problem for a decentralized system, or one where the server for some reason cannot (or cannot be trusted to) timestamp messages. In that case, I think the best behavior for a client would be to always display messages in the order they've arrived, regardless of any timestamp provided by the sender.

But this problem shouldn't exist for a system like Matrix. Matrix is (somewhat) decentralized, but each homeserver can still decide on the message ordering it will present to its own clients.

[shawnz]

If you don't allow the client to specify the time they sent the messages, then anyone who has a poor connection is going to be subject to an annoying behaviour where their messages are constantly going out of the intended order during busy conversations.

[thomastay]

Obv it depends, but one way to "solve" this it is to show an edit history, or at least the latest edit timestamp along with some visual indicator that the message was edited recently

[shawnz]

I'm not talking about edits, I'm talking about sending new messages which are backdated to appear as part of an older conversation

[thomastay]

icic, yeah that definitely shouldn't be allowed

[glandium]

Matrix allow to edit weeks old messages, already.

[shawnz]

But there's a flag which indicates they've been edited and you can see the edit history, right? So that's not useful for this scenario.