MSC2836: Twitter-style Threading

[kegsay]

Rendered

Implementation: https://cerulean.matrix.org/
Server: https://github.com/matrix-org/dendrite/tree/master/setup/mscs/msc2836
Client: https://github.com/matrix-org/cerulean/

[ara4n]

it's more that m.reference is the class of type of relation (i.e. "one event pointing to another somehow" - as opposed to an annotation which can be aggregated like an m.reaction, or an edit which can be aggregated like an m.replace). the question i had is more whether we should have an additional domain-specific field to give the type of the reference. i guess we could default to no explicit relation subtype meaning "threaded reply".

[Sorunome]

m.thread or something seems more appropriate, as m.reference is too general. This, of course, begs the question: what if an event is a reply in the thread? As it can't have multiple relationship types, we can't simulate this.

Perhaps this is reason to look back at event aggregation MSCs and adapt?

[deepbluev7]

m.reference also clashes with MSC2241. They definitely should not look the same.

Regarding the relations format, I commented about that here, mentioning the limitations regarding threads for example: #2674 (comment)

Although if you just want nested threads, replies don't make that much sense anyway.

[kegsay]

I don't care what the rel_type is called, and am happy to go by consensus on this. What's the possible options we have now?

what if an event is a reply in the thread?

Regarding the relations format, I commented about that here, mentioning the limitations regarding threads for example: #2674 (comment)

So after reading this, it comes down to how you see an "event". Is it:

• A single logical piece of JSON with all updates/references contained within.
• A diff on the event DAG.

I see it as the 2nd option, whereas the map/array relations format sees it as the 1st option. Therefore, if you are replying to an event which already has an m.relationship then that's fine, because you are still just referencing the event_id. It makes it impossible to reply to more than one thing - i.e it shouldn't be possible to combine this MSC with in_reply_to and if you do want to do that then you'll need to send two events instead of one. This is partly for simplicity: ensuring threads only have a single parent keeps things simple and fits logically with what clients do - I don't think I've seen any UI which allows a post to merge 2 threads together. Allowing multiple relationships also causes some ambiguity around ordering: if an event has an edit and a reaction, is the reaction for the original message or the edited message?

[deepbluev7]

Thinking of events as diffs is really annoying for clients, since now you need to pull an entire graph out of the database instead of just a single events. For simpler clients, that is simply impossible (which is why weechat doesn't implement proper replies yet) and for others it is just annoying (in Nheko events are rendered using the current event json. Replies are just embedded different events. Having to parse and merge multiple events for edits is not something I want to implement, since it makes a lot of stuff more complicated to handle).

Multiple relations does cause some conflicts in some cases, yes, but I don't think editing a reaction makes any sense anyway. On the other hand there is no order relation, when you edit a reply. It does not matter, if the current event is an edited reply or a replied edit. Forcing a specific order and the client to jump through hoops to have multiple relations, just causes more issues. For example you can't edit a relation away, if edits are just diffs, as there is no "remove that key from the original event" functionality. If events are not diffs, that entire issue goes away.

if an event has an edit and a reaction, is the reaction for the original message or the edited message?

Is that meant as you send one edit to event X and an edit to event X? In that case you have that issue today already in clients not implementing edits. You really should specify that in edits, that you should reply, thread from, react to or edit only the original event id. Nothing to do with diffs vs idempotency.

Which one of those sounds simpler?

• A single logical piece of JSON with all updates/references contained within.
• A diff on the event DAG.

A diff on the event dag sounds horrible from a complexity perspective, even if a single event is simpler. I much prefer having the complexity localized.

[kegsay]

I'm aware that handling diffs is tricky for clients. Clients generally need to be able to handle diffs anyway though if they are listening for incremental updates down /sync. This MSC does not preclude providing some aggregated response in /sync which may include the entire thread of conversation as a single JSON object. However, the ability to handle diffs is still required and necessary. A future MSC could provide a way to aggregate threads on /sync e.g by specifying a filter which enables aggregation.

[deepbluev7]

This is the same m.reference as is being used here. (There is only one!)

It means "this event references is another event". E.g. a verification flow contains events which reference each other.

I thought all relationships are about referencing other events? Replies reference other events, as do edits and reactions. The type is about how the event references the other event. Really, imo m.reference should only be used, if an event references other events loosely, without attaching any meaning. This does not apply to in dm verifications, nor does it apply to threads. While I'm not sure how to call the former, why not call threads m.thread or something similar, that actually conveys the meaning? m.reference and m.replaces are pretty bad names imo, since they are too generic.

Clients generally need to be able to handle diffs anyway though if they are listening for incremental updates down /sync.

Not really, the only thing that really requires it is rendering membership events and a few others nicely, and most clients only do that for memberships.

[ara4n]

I thought all relationships are about referencing other events? Replies reference other events, as do edits and reactions.

I think this is a confusion of terminology. All #1849 style relationships are about relating events together. The relationship can have one of 3 flavours: m.reference (B points to A), m.replace (B replaces A), m.annotate (B annotates A). Yes, they all relate A and B together, but they differ in the semantics of how results are aggregated. References get returned verbatim; Replaces get replaced, Annotations get grouped and summed.

So given that m.reference is the most generic one (it's just saying that B points to A for some reason), the question is whether to say how B points to A. The idea is that you could have different subtypes of reference of B to A (e.g. threaded reply; verification state transitions; voip state transitions etc) and the server will be able to let you navigate through the resulting DAG of references without it understanding semantically what the references actually mean. So that additional typing could be implicit (m.room.message + m.reference = threaded reply, which would then break if you wanted anything but an m.room.message in your thread) or could be explicit, by introducing a ref_type: "m.thread" or perhaps trying to mux the subtype into the rel_type - e.g. rel_type: "m.reference.thread".

Does that make any more sense? I think both @kegsay and @deepbluev7 are failing to grok the intention of m.reference.

[deepbluev7]

Does that make any more sense?

Not really. If "m.reference" is the generic one, why use it at all? Obviously every relations references another event. But why not default to handling events like "m.reference" in you proposal? That way all events get a somewhat reasonable default behaviour, and you can then special case it in the spec. There is not really a problem with m.replace or m.annotate returning all events, it is just much more efficient not to. Munging a special type into the rel_type sounds like a bad idea to me. Every matrix identifier should be opaque, if possible. Either add an additional key like sub_rel_type or just make the reference behaviour the default. I dislike using subparts in the indentifiers, since those are hard to namespace correctly in MSCs, need parsing, etc. (account_data suffers from this too.) The semantic meaning is important in any case. You probably want a subtle difference in how relations are handled for threads, like being able to return a certain level of nesting or only a special label of a thread. Implicit typing sounds like a really bad idea too.

I think using "m.reference" is fine in some cases, but threads are not one of them imo, because clients want to handle them specifically and servers may want to at some point too.

[bwindels]

For better or worse, I've tried to make MSC 2674 somewhat opinionated in this regard in that rel_type is how the server should aggregate things, and accepted values should be kept to a minimum. If clients require further differentiation, another field can be used (e.g the ref_type Matthew alluded to). Would be nice to be consistent between both MSCs.

[ara4n]

This looks really promising, imo. Would be good to mention the swimlane stuff (i.e. "can I have messages in a thread which aren't explicit references? I don't want to have to set an explicit reply whenever I type within a thread - I just want to send a message!"). Also, think the pagination needs a bit more thought.

[deepbluev7]

Wouldn't the swimlane stuff a client side feature? Although it probably makes sense to mention the expectation, that clients implement it like that.

[kegsay]

Swimlanes are entirely a client-side problem imo. It would be up to the clients to determine the 3 swimlanes in a graph like:

A
|
B
| \
C  D
|  |
E  F
|  | \
G  H  I
|  |  |
J  K  L

But at a protocol level it is the most useful to have explicit references back to the previous event in the swimlane. This PR is motivated for more Twitter style use cases which don't really have swimlanes though, hence their omission.

[Pestdoktor]

explicit references back to the previous event in the swimlane

Wouldn't that regularly break when multiple clients try to reply in a swimlane at the same time?

[ara4n]

But at a protocol level it is the most useful to have explicit references back to the previous event in the swimlane.

This would break swimlanes, which is the point i'm trying to make. We should acknowledge that we should support a mix of messages with explicit m.reference replies (an explicit fork in convo) versus ones without them (normal messages in a room, or messages within a thread). (Even though the twitter/reddit/HN-style use cases don't need this).

[Pestdoktor]

To illustrate, given the example conversation from Kegsays comment, this is what a client could do:

(the arrows being m.references)

[ara4n]

What's the advantage an m.label would provide over a subthread?

In the example of your graph (is that yEd?) above, it would tell you whether B is in lane 1 or lane 2. And would let you mix together filter-by-swimlane with filter-by-labels for clients with filter buttons.

[erdnaxeli]

If I may, I like the idea of using m.referencess to define swimlanes. In this graph:

• A is in the room (main swimlane), and a new thread was created from it (and A should also be shown at the top of the thread swimlane)
• B C E G J are in the same swimlane created from A by B.
• D F H K are in the same swimlane. This is another thread, created from B.
• and so I L are in the same swinlane, this is another thread created from F.

From a client perspective, this means you can create threads in threads. You can create a new thread from any message, even a message already in a thread. Contrary to slack for example where you can only create a thread from a message in the channel (i.e. in the main swinlane).

So if you want to post to a swimlane, you have to m.references the same event that the previous event in the swimlane does. Clients that want to have a tree style (like twitter/reddit/...) could just create a new swimlane, i.e they reference the previous message.

[ara4n]

I agree the idea is interesting, although i'm still a bit worried about how the layout algorithm would know that B should be a new swimlane or not. It feels like the only way of telling is whether B has children of its own?

[erdnaxeli]

I think the behavior should be something like:

• all messages sharing the same m.reference are in the same swimlane
• the message referenced should be shown at the top of the swimlane, for context

[kegsay]

I think i'm being thick, but i still don't get how you know whether your result got clipped by max_depth or max_breadth and how you know whether to continue downwards or sideways to grab more of the tree or not?

There is no explicit return value to indicate that the max_depth or max_breadth was hit. If a request reaches the limit, events further away will be retrieved, if you want to walk down a specific path, you need to make an /event_relationships request with that new event ID.

---

              depth
   A           0
   |
   B           1
  / \
 C   D         2
 |  /|\
 E F G H       3
 |   | |
 I   J K       4
     |
     L         5

given event_id=A max_depth=4, max_breadth=2, limit=10, depth_first=false, recent_first=false (and earlier letters are earlier events)
the following items are returned:

A,B,C,D,E,F,G,I,J - 9 items, no next_batch.

This is because the max_ params create a bounded window over the graph, and it is this graph that gets looped over.:

              depth
+-------+              
|   A   |       0
|   |   |
|   B   |       1
|  / \  |
| C    D|       2
| |  / || \
| E F G | H     3
| |   | | |
| I   J | K     4
+-----|-+           max_depth=4
      L         5

Changing the limit to be 5 just returns the first 5 elements in this graph (A,B,C,D,E), and the next_batch returns (F,G,I,J).
It may be desirable to somehow encode the number of clipped items (for UIs which have "see all 97 comments") in the parent event
(in this case D would say it has 3 children), however in practice a lot of UIs just have "see more" which can be easily accomplished
by setting a max_breadth 1 greater than the number of items you wish to display so you know that there are in fact more items to see.
Clicking on "see more" would do a new /event_relationships request to load that part of the DAG.

[nidico]

This is not related to the Threading MSC draft by @timokoesters, which was linked from This week in Matrix two weeks ago, right?

[Sorunome]

so far everything uses m.relates_to (edits, reactions, replies) and if you want to reference accross rooms you should probably also include the room id

[erdnaxeli]

I have the same concern about m.relationship and m.relates_to. Is a standardization planned?

[Sorunome]

The client would have to send it. As of #2848 servers are not expected to be able to map from event ID to a room ID.

[mscbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[mscbot]

The final comment period, with a disposition to close, as per the review above, is now complete.

[ara4n]

I'm not sure we should have closed this, ftr - it provides an entirely different set of threading semantics (i.e. twitter-style threads) to the slack-style ones we've ended up on in #3440, and could well still end up being useful - or as useful as many other open MSCs.

[Sieboldianus]

Does this mean the end of the awesome cerulean.matrix.org? (see Introducing Cerulean)

[ara4n]

I'm taking the liberty of unclosing this, as it's still completely relevant, especially with Twitter's current transformations. I'll rename it to differentiate it from the threading that Element currently implements.

[filipesmedeiros]

Even more relevant when Dendrite (the newer generation of the Matrix server) implements MSC-2836 but not #3440.

Is anyone looking into this? :)

Thanks!

[olivierlambert]

Is it, as of today, a possibility to get this feature?

[anotherdoesnm]

It's abandoned as i asked in matrix group

[olivierlambert]

Darn 😢 Thanks for the info!

[kegsay]

I still have a preference for this model at a protocol level over what we ended up with in practice, but it's an uphill struggle to demonstrate this and then actually work on and ship a change. This came up last year to which my impassioned response on the whole thing is below.

---

Background

Currently at a protocol level, threads are entwined with normal messages in the "main" timeline. This causes numerous problems for the UX we want:

• we want to treat notifications on threaded messages differently
• we want to treat receipts on threaded messages differently
• we want to restrict threaded messages differently to messages in the "main" timeline
• we want to show "XXX is typing" in different places depending on if they are typing in the main timeline or in a thread.

This has led to a proliferation of thread-specific protocol behaviour to fix each of these issues:

• MSC4306 "Thread Subscriptions" to fix notifications.
• threaded read receipts
• MSC3394 "new auth rule that only allows someone to post a message in relation to another message" to restrict threaded messages.

In addition, threads as they are currently implemented, do not work correctly over federation because there is no way to:

• fetch all thread roots in a room
• fetch all the messages for any given thread

..meaning even if we land these protocol changes, threading will remain flakey, with missing threads and missing messages depending on what federation is doing.

Idealistically, the building blocks Matrix provides would be composable enough to not need custom behaviour like this. Taken to the extreme, this could happen again if we wanted to support BBS forums, StackOverflow style Q&A, etc. The net result is a sprawling API which is hard to test, and hard to understand how custom behaviours interact with each other.

Using the room building block

We've used rooms as a building block before when we added Spaces, so we know that rooms can be more than just chat rooms. Spaces added new functionality which
created authorised links between rooms via "restricted" join rules. I didn't realise that this is a powerful feature that can be applied elsewhere until
Half-Shot pointed it out wrt room upgrades (rather than sending invites to the new private room, just make it restricted such that you must be joined to the old tombstoned room to join the new room). What if we did this technique for threads?

Haven't we been here before?

Yes. Cerulean was an MVP pitched to Twitter prior to them doing Bluesky as a way to do decentralised Twitter. Every thread was a room, and the blog post has a diagram explaining how Cerulean did personal timeline rooms and threaded rooms. This was backed by MSC2836. In the end, we went with MSC3440. The Alternatives section of that MSC lists the following:

Advantages to "Threads as rooms" include:

• May be simpler for client implementations
• Restricting events visibility as the room creator
• Ability to create read-only threads

Disadvantages include:

• Access control, membership, history visibility, room versions etc needs to be synced between the thread-room and the parent room
• Harder to control lifetime of threads in the context of the parent room if they're completely split off
• Clients which aren't aware of them are going to fill up with a lot of rooms.
• Bridging to non-threaded chat systems is trickier as you may have to splice together rooms

In particular, the fact that "access control", "membership" and "history visibility" can be different are all listed as disadvantages, but as Element the product
has evolved, it turns out we do want to have different access control ("we want to restrict threaded messages differently to messages in the "main" timeline") and membership ("we want to treat notifications on threaded messages differently"). In addition, it immediately solves the typing and receipts problem as threads are split out into their own timeline. It also fixes the problem of threads not working over federation as "getting earlier threaded messages" is just hitting /messages in the thread room. When coupled with the "restricted" join rule to inherit the access control of the parent room, there's enough advantages here that it's worthwhile revisiting to see if we chose the wrong path.

[clokep]

In particular, the fact that "access control", "membership" and "history visibility" can be different are all listed as disadvantages, but as Element the product has evolved, it turns out we do want to have different access control ("we want to restrict threaded messages differently to messages in the "main" timeline") and membership ("we want to treat notifications on threaded messages differently").

The intent behind listing this as a disadvantage was not "we don't want these things", but "these are primitives that don't exist in the ecosystem today".

In particular with how hard it has been to get notifications and receipts working sanely with threads, I wanted to revisit the "threads as room" when doing receipts, but did not have enough time / motivation to go back to the drawing board. I think it really gives you a lot of advantages / things for free though. And the things it doesn't probably apply to space -> room relationships, as well as room -> thread.

[kegsay]

And the things it doesn't probably apply to space -> room relationships, as well as room -> thread.

Which is a fantastic thing, because it then means any protocol change we add to one benefits the other because the use cases are similar. For example, perhaps we want to automatically cascade kicks/bans from a parent space to children rooms, which we would also want for room to thread rooms. This is composability working at its best imo.