[shawnz]

How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

[dataflow]

> How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

I believe that's impossible? At least if you design it correctly.

For ordering/interleaving purposes, what matters isn't the time you claim to send the message, it's the time the message is received by the server. If you want, you can display the claimed send timestamp beside the message (and prominently highlight it if it is e.g. out of order, or with a long delay, etc.), but that is irrelevant to the ordering.

The point here is that there should be a single consistent order on the server, and that's what all clients ought be displaying. Any messages not yet acknowledged by the server should be displayed differently so that users are aware they haven't been seen yet, and any messages that arrive before those are sent would obviously get inserted above those.

[lmm]

> what matters isn't the time you claim to send the message, it's the time the message is received by the server

There's no "the" server here. If you use the time the message is received by the server, you'll get different views on different servers, and you may see messages from months ago appearing as new, if connectivity breaks down and is later restored.

[dataflow]

> There's no "the" server here.

Can't you assign every conversation to a single authoritative server for handling?

Also, how large of a time skew are you imagining would exist between different servers? That stuff ought to be accurate to at least milliseconds if not micro...

[lmm]

> Can't you assign every conversation to a single authoritative server for handling?

The whole point of Matrix is to be decentralised. In particular people should be able to keep talking when on different sides of a netsplit, by design.

> Also, how large of a time skew are you imagining would exist between different servers? That stuff ought to be accurate to at least milliseconds if not micro...

The question isn't how much time skew there can be between server A and server B, it's how long they can be cut off from each other over the network, which could be hours at least. (And even when things are working well, a normal ping is a few hundred ms, which is enough to change the order of messages)

[dataflow]

> The whole point of Matrix is to be decentralised. In particular people should be able to keep talking when on different sides of a netsplit, by design.

OK but I still don't see the problem. Even with a fully decentralized system where the servers are just pure relays with no authority, you have two options:

1. Display messages in the order in which they claim to have been sent, or

2. Display messages in the order of arrival

Case #2 is the obvious/uninteresting one, there's nothing to say about it.

Case #1 is what people are saying is so impossible to achieve a global order for, but really, what's the big deal? If a client claims to have sent a message at an unusual time (say, > 10 seconds in the past, or after the app was already quit, or whatever criteria you want to set), then just insert it at that point in the conversation, and visually indicate to the user the discrepancy. And clock skews won't really be much of a problem because messages can easily indicate prior messages in the conversation, so that a mere clock skew doesn't insert them before preceding messages.

What's so hard to make user-friendly/intuitive here?

[lmm]

> If a client claims to have sent a message at an unusual time (say, > 10 seconds in the past, or after the app was already quit, or whatever criteria you want to set), then just insert it at that point in the conversation, and visually indicate to the user the discrepancy.

I don't know that OP would be happy with that, and certainly someone would need to a) actually design the UI for it b) figure out what information the client needs from the server to implement that, and whether it's possible for the server to provide that information.

I think you're probably right, FWIW, but someone needs to actually do the legwork of designing and implementing what you're suggesting rather than just handwaving it.

[2Gkashmiri]

Why cant we have a sent time and the receipt time ?

The time the sender claims the client sent the message be appended to the message itself.

Let it reach 500ms or 2 seconds later.

If there is an acceptable skew between the sending time and receiving time, we just accept the sending time.

Edit: what this could do is, the sender when they sent the message, they were aware about x messages before and the clocks being in sync for existing messages, their message even if received 2 seconds later would be put in the origiNAL order of sender intention

[shawnz]

Aside from the concerns with decentralized servers that the other poster mentioned, this has the disadvantage that your messages are going to get constantly reordered to not match the intended flow of the conversation when you have poor connectivity, which is a bad user experience

[dataflow]

Wasn't the whole point here that the messages wouldn't get reordered? There would be one definite order that everyone would see. Again, if the message isn't timestamped by the sever, it would need to appear visually differently, so that everyone knows about this. And nobody says the server has to accept messages with arbitrarily delays either.

[shawnz]

My point is that some limited reordering maybe should be allowed, but not too much. That is to say, the problem isn't as simple as just doing it one way or another way. Every approach has some disadvantages.

[dataflow]

I know you were trying to reach that conclusion, but my point was that the design I suggested neither seemed to have the problem you suggested, nor is reordering a necessary outcome, from what I can tell.

[fastball]

You can amend displayed order for humans (what matters for 99% of usage), while still allowing anyone interested to see when the message actually arrived at the homeserver (making the suggested gambit impractical).

[kelnos]

I don't think this is really a problem, at least in the case of client->homeserver connections. The homeserver should not be trusting the client's sent timestamp. The homeserver should consider the message sent at the time it receives it, and the client should know this and update the sent timestamp displayed to the user when it is finally able to connect to the homeserver, and the homeserver acknowledges receipt of the message.

The bigger problem is how to handle homeserver<->homeserver comms. My initial feeling is that the homeserver where a destination room is hosted (let's call this one "A") should have the final say, and if there are people on another homeserver ("B") that have joined the room, and are chatting while there's a break in connectivity between the two homeservers, then A should just append all the messages from B to the end of the record (with correspondingly "later" timestamps) to the "official" record, when B is able to communicate with A again.

But this feels messy too; presumably all of those new messages (a conversation that may have been going on for tens of minutes or hours) would be smooshed in to have their timestamps all appear nearly at the same time? No, that's not great either.

Or perhaps B just shouldn't accept messages for that room while it can't communicate with A? That doesn't seem great either.

[shawnz]

Then instead imagine this: the user really is innocent and just happened to coincidentally send the message right after the start of a long period of poor connectivity (like a flight, or a road trip, etc). If you just allow it to go through after an arbitrary delay, with only a log of the received time for liability purposes, then the user wouldn't have any indication of this scenario occurring.

Wouldn't it be better in that case to show an error so that they can make sure the situation is addressed appropriately?

[Spivak]

You have when the client claims it was sent (so where it goes in the displayed history) and can see when it was received. What else could you possibly do?

[shawnz]

For example, you could reject the message and show the user an error but only if there's a discrepancy of >X minutes. But how much discrepancy should be allowed? I don't know, I only mean to show why I think the solution isn't as simple as it appears

[lmm]

> you could reject the message and show the user an error but only if there's a discrepancy of >X minutes

No you can't, not in a federated and decentralised system like this.

The sender can wait for a read receipt from a given receiver user, if the receiver is willing to make those public. But if the message left client A and didn't arrive at client B, there's no objective fact of the matter about whether the message "was sent" or not.

[immibis]

Seems like a design deficiency of Matrix then. When IRC federation breaks, everyone can see it, except for the rare people who aren't in a shared channel with anyone on the other side of the break.

[lmm]

> Wouldn't it be better in that case to show an error so that they can make sure the situation is addressed appropriately?

If you want a single centralised server then you can set things up that way. Presumably if you're using a setup with multiple servers, and took one of the servers on the flight/road trip, you wanted the people on the flight/road trip to be able to keep talking to each other over that server, even though that server is disconnected from the one in the office.

[kiitos]

> How far back should you be able to amend history?

If user U1 sends a message M1 at time T1, then U1 must be able to modify/delete that message M1, in some reasonable sense, at any conceivable time from T1 forwards.

Any protocol that doesn't support some reasonable form of message modification/deletion in this sense, is a toy protocol, and will never be widely adopted.

[shawnz]

I'm not talking about message editing but rather posting new messages with a backdated time.

[kiitos]

I suppose every message has a few timestamps, including

- The timestamp that the user specified as part of the message

- The timestamp of the server that the user submitted the message to, directly

- The timestamp of the server that first received that message

- Any additional timestamp(s) of additional server(s) that received that message

The user can I guess backdate a time, but that would apply only to the first thing, and none of the others?

[shawnz]

The first one is the only one that can accurately describe where the user intended for the message to land in the conversation. All the others are subject to network delays, so if you order messages by anything other than the first one, you are going to get a bad experience if you try to participate in a busy conversation with a poor connection.

[kiitos]

The first one is entirely un-trustable, because the user-submitted timestamp can be at any arbitrary point in time, from t=0 to t=infinity. So the receiving system can use that timestamp as an important bit of signal, but it can't really treat it as strictly authoritative, at least not if it expects to maintain a coherent set of events overall.

[shawnz]

Exactly, that's why I'm saying it's a more challenging problem than it appears and there's no one solution that always gives the best experience in every case. I personally think a hybrid approach of allowing some limited discrepancy between user and server timestamps is probably the best you can do.

[kelnos]

This shouldn't be an issue for systems where a server mediates communication: the server should be timestamping messages, not the clients.

This could indeed be a potential problem for a decentralized system, or one where the server for some reason cannot (or cannot be trusted to) timestamp messages. In that case, I think the best behavior for a client would be to always display messages in the order they've arrived, regardless of any timestamp provided by the sender.

But this problem shouldn't exist for a system like Matrix. Matrix is (somewhat) decentralized, but each homeserver can still decide on the message ordering it will present to its own clients.

[shawnz]

If you don't allow the client to specify the time they sent the messages, then anyone who has a poor connection is going to be subject to an annoying behaviour where their messages are constantly going out of the intended order during busy conversations.

[thomastay]

Obv it depends, but one way to "solve" this it is to show an edit history, or at least the latest edit timestamp along with some visual indicator that the message was edited recently

[shawnz]

I'm not talking about edits, I'm talking about sending new messages which are backdated to appear as part of an older conversation

[thomastay]

icic, yeah that definitely shouldn't be allowed

[glandium]

Matrix allow to edit weeks old messages, already.

[shawnz]

But there's a flag which indicates they've been edited and you can see the edit history, right? So that's not useful for this scenario.