Unexpected message order when system clock set to the past

[DarkCat09]

• Operating System (Linux/Mac/Windows/iOS/Android): Linux
• Core Version: v2.43.0
• Client Version: 2.43.0

Same behavior on latest ArcaneChat v2.46.0

Expected behavior

I'd expect new messages to be displayed at the bottom, after any previous messages in the chat, despite their timestamps.

Actual behavior

When the system time is in the past (because of some misconfiguration), and there are "newer" messages in a chat, Delta clients display new sent messages right where they should've been placed by their timestamp, i.e. above previously written messages that are newer by timestamps but actually were written before.

You may argue this condition is very unusual, but it can happen, it happened, and the client behavior was extremely unobvious and confusing.

Steps to reproduce the problem

1. Send some messages to a chat
2. Set system clock to an hour earlier
3. Send another message
4. That new message is displayed above all the previous

Screenshots

See the screencast showing how to reproduce the bug:

order.mp4

Logs

Nothing useful

[DarkCat09]

I'd also like to ask if there's any advantage of sorting by timestamp instead of sqlite autoincrement id

[DarkCat09]

As i can see, a message timestamp stores when the message was received by this client. So, sorting by timestamp is exactly the same as sorting by ID.

This also does not solve wrong order when fetching from multiple relays -- they're polled for new mail sequentially, so both timestamp and ID are greater for secondary relays.

If we switch to ID message ordering, it shouldn't affect anything, except for being better at handling edge cases with changing system clock.

Please let me know if i'm missing smth (looks like i do)

[link2xt]

So, sorting by timestamp is exactly the same as sorting by ID.

Sorting timestamp is the timestamp of the sender, adjusted to be not in the future and not before the last seen message in the chat. It is not the same if you have a group and receive messages from them reordered because connection between MTA of one of the senders and your MTA was broken for some time. In this case sorting unread messages by timestamp of the sender helps as long as the sender device clock is not wrong.

[link2xt]

It maybe makes sense to sort 1:1 chats by ID, but this will not fix the groups and maybe make them more broken. Having different logic for 1:1 chats and groups will only increase complexity.

See also #8030

I am not sure we should optimize for incorrectly set clock. We also don't want to trust the server clock because then any relay with incorrect clock will break the order. The problem should rather be fixed by telling the user to fix the clock, otherwise we will have no reliable source for message order.

[DarkCat09]

But is it really the sender's timestamp? When my friend accidentally changed system clock to the past (she wanted to change the timezone), on her side, new sent messages were incorrectly ordered above the previous, but on my side, all her new messages were placed at the bottom and even had correct timestamp matching my time

[link2xt]

but on my side, all her new messages were placed at the bottom and even had correct timestamp matching my time

This happens because messages are always sorted below the last seen message. Her messages are from the past and you can see "1 hour ago" in each message bubble, but we always sort them below the last seen message (e.g. your outgoing message). This causes its own problems: #8030 (comment)

[DarkCat09]

Oh, i see, i've carelessly read your reply

Sorting timestamp is the timestamp of the sender, adjusted to be not in the future and not before the last seen message in the chat

Then it makes sense, thank you for clarification <3

[Hocuri]

I think this can be closed, because there is nothing good we can do here, short of a topological ordering of messages by references (which we can experiment with at some point, but there is no need for an open issue)

[iequidoo]

A simple thing than can be done (while we don't have topo-ordering of messages) is adjusting the sort timestamp of a received message so that it's >= one for its parent message. We load the parent message anyway, so it's almost for free.

[link2xt]

A simple thing than can be done (while we don't have topo-ordering of messages) is adjusting the sort timestamp of a received message so that it's >= one for its parent message. We load the parent message anyway, so it's almost for free.

This will make some scenarios better and some worse, if someone sets the clock 1 hour forward then child messages that had the correct timestamp will get the timestamp incorrectly adjusted into the future.

[iequidoo]

This will make some scenarios better and some worse, if someone sets the clock 1 hour forward then child messages that had the correct timestamp will get the timestamp incorrectly adjusted into the future.

The sort timestamp isn't smth user-visible, it only affects sorting. I don't think adjusting it this way will cause sorting issues, we limit the sort timestamp with smeared time anyway, what may happen is we're offline for some time and then we receive a message with a wrong "Date" a bit in the future, and it has replies so all of them will get the sort timestamp shifted, and e.g. also there are other messages which don't reference that message because they were sent by other chat members being offline (or they are replies, but were received earlier by us via another relay), so finally we get messages wrongly ordered, but this isn't much worse than having messages from the member with the wrong clock always sorted to the bottom when we come online.

Moreover, it seems we already have this logic, see receive_imf::tweak_sort_timestamp() -- we should check if it's actually working.

But this issue is about the sender side, so if things work on the receiver side, but don't work for the sender if they have clock in the future, the bug should be reopened. EDIT: And it should:

[...] on her side, new sent messages were incorrectly ordered above the previous, but on my side, all her new messages were placed at the bottom and even had correct timestamp matching my time