Skip to content

MSC2477: User-defined ephemeral events in rooms #2477

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ananace wants to merge 14 commits into
base: old_master
Choose a base branch
from
Open

MSC2477: User-defined ephemeral events in rooms #2477

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
aad3e26
MSC for user-defined ephemeral events
ananace Mar 28, 2020
4aa82b8
Fix a missing comma in example JSON
ananace Mar 30, 2020
d4930c0
Remove the requirement to forbid sending m.* types
ananace Mar 30, 2020
ba0b657
Relink MSCs to point to their respective PR
ananace Apr 4, 2020
0b27339
Add reference to MSC2487 for appservice filtering
ananace Apr 4, 2020
5dc9372
Namespace the extra key in the room sync example
ananace Apr 4, 2020
dd488d3
Update from some comments
ananace Sep 9, 2020
9d73697
Remove non-existing EDU from fallback list
ananace Sep 9, 2020
2f4dd47
Update proposals/2477-user-defined-ephemeral-events.md
ananace Aug 1, 2021
047d63b
Update according to review comments
ananace Nov 30, 2021
2cae98c
Merge branch 'user-defined-edus' of github.com:ananace/matrix-doc int…
ananace Nov 30, 2021
ff2ddbe
Merge some review suggestions
ananace Jan 10, 2022
64e9460
Some more review comment changes
ananace Feb 9, 2022
d01e95d
Add note on unstable prefix for implementation
ananace Jun 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 44 additions & 26 deletions proposals/2477-user-defined-ephemeral-events.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,28 +23,29 @@ similar manner to the already existing support for users to send their own types
Note though that this proposal does not include any support for sending user-defined ephemeral
events which are not explicitly bound to rooms, like the global `m.presence` event.


Examples of how this feature could be used are; as regular status updates to a user-requested
long-lived task, which a bot might has started for a received event. Or pehaps as a GPS live-location
feature, where participating client would regularly post their current location relative to a
persistent geo-URI event. Perhaps for organizing meetups, or for viewing active tracking of the
locations of vehicles in an autonomous fleet - along with peristent messages posted at a lesser
locations of vehicles in an autonomous fleet - maybe along with peristent messages posted at a lesser
rate for a timeline generation.

The example that will be used througout this proposal is an ephemeral data object that's tracking
the current status of a user-requested 3D print, with some basic printer- and print-status
information being sent every few seconds to a control room, including a reference to the event that
the status is referring to - which the client could use to render a progress bar or other graphic
the status is referring to - which the client could use to render a progress bar or some other graphic
with.

### Addition of an ephemeral event sending endpoint to the Client-Server API

The suggested addition to the CS API is the endpoint
`PUT /_matrix/client/r0/rooms/{roomId}/ephemeral/{eventType}/{txnId}`, which would act in an almost
The addition to the CS API is the endpoint
`PUT /_matrix/client/v3/rooms/{roomId}/ephemeral/{eventType}/{txnId}`, which would act in an almost
identical manner to the event sending endpoint that is already present.
An example of how an update might be posted using the new endpoint;

```
PUT /_matrix/client/r0/rooms/%21636q39766251%3Aexample.com/ephemeral/com.example.3dprint/19914 HTTP/1.1
PUT /_matrix/client/v3/rooms/%21636q39766251%3Aexample.com/ephemeral/com.example.3dprint/19914 HTTP/1.1
Content-Type: application/json

{
Expand Down Expand Up @@ -121,15 +122,23 @@ As it would be possible for the user-defined events to be used to flood a room w
traffic by malicious users - increasing the bandwidth usage for all connected servers, this proposal
also suggests extending the power levels to handle ephemeral types as well.

In any room version implementing this MSC, the auth rules to include in `m.room.power_levels` are;
In any room version implementing this MSC, the auth rules concerning ephemeral events in the
`m.room.power_levels` event are;

- `ephemeral` (`{string: integer}`) - A mapping of EDU types to the power-level required to send them
- `ephemeral_default` (`integer`) - The default power-level required to send any EDU not listed in
the above mapping

These new keys are to function in an identical manner to the already existing `events` and
`events_default` keys, with the suggested default - and fallback - value for `ephemeral_default`
being 50, while the suggested default - and fallback - values for `ephemeral` would be;
`events_default` keys, with the assumed default value for `ephemeral_default` - if there is no
`ephemeral_default` in the `m.room.power_levels` event - being 50, while the default values for
`ephemeral` - if there is no `ephemeral` in the `m.room.power_levels` event - would consider all types
to be `ephemeral_default`, or 0 if there is no `m.room.power_levels` event - which would then not allow
any ephemeral events to be sent.

It is therefore recommended for servers to include at least the following `ephemeral` configuration
for all newly created rooms of any room version implementing this MSC, to allow for the sending of
the default ephemeral events in Matrix;

```json
{
Expand All @@ -138,26 +147,19 @@ being 50, while the suggested default - and fallback - values for `ephemeral` wo
}
```

These defaults are to ensure that all current Matrix EDUs continue to work as intended.

**NB**;
To reduce the complexity of the change, this proposal suggests to - for the time being - only limit
the user-defined types by these power levels changes. The default values for `m.*` specified here in
`ephemeral` would then only be expected to be informational in purpose.

### Extension of the room-specific ephemeral data received in /sync responses

Because the user-defined ephemeral events can't be aggregated and massaged by Synapse in a simple
manner, this then suggests instead adding a few more fields to the room-specific ephemeral events as
they are encoded in a sync response. The suggested additions are;
manner, this MSC instead requires adding a few more fields to the room-specific ephemeral events as
they are encoded in a sync response. The additions in question are;

- `sender` (`string`) - The fully qualified ID of the user that sent the EDU
- `origin_server_ts` (`integer`) - Timestamp in milliseconds on the originating homeserver when this
event was sent
- `origin_server_ts` (`integer`) - The timestamp in milliseconds on the originating homeserver when
- this event was sent

To reduce the risk of breaking existing clients, as well as reducing the scope of change required by
this proposal, the suggestion is to allow the original `m.*` events to skip these keys where no value
could be easily assigned to them. E.g. typing notices, read receipts.
To reduce the scope of changes required by this proposal, the suggestion is to allow the original
`m.*` events to skip these keys where no value could be easily assigned to them. E.g. typing notices,
read receipts.

```json
{
Expand Down Expand Up @@ -249,14 +251,29 @@ could be easily assigned to them. E.g. typing notices, read receipts.
}
```

### Delivery guarantees

The current ephemeral event system in Matrix is built on a sort-of guaranteed delivery - albeit with
mutation/consolidation - of the events. This might not be desirable with custom ephemeral events, as
they could contain volumes of data that's not as easy to keep around for a guaranteed delivery.
Copy link

@TheArcaneBrony TheArcaneBrony Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it might make more sense to remove the mutation/consolidation expectation?

Copy link
Contributor Author

@ananace ananace Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, replacing the built-in m.read/m.presence/etc with proper types that aren't munged by the server is indeed something that could be done, but I'm writing this MSC entirely towards the addition of user-defined EDUs and nothing else - leaving the already existing EDU types as they are.


Therefore I suggest that servers are only required to provide best-effort delivery, with the exact
method in how they propagate EDUs - and store them - left up to implementation.
(Perhaps keeping a max-size ring buffer per room - that will thus remove old data when necessary.
Or maybe only storing per active sync token, per user, per device.)

As the messages in question are ephemeral, I think the only guarantee that should be required is that
all users that are online when the message is sent will receive it. Anything above that should be
Copy link
Member

@kegsay kegsay Sep 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all users that are online when the message is sent will receive it

This is actually a very high bar! When I was designing MSC4354 I thought to provide similar guarantees, but to meet them requires a lot more work than this proposal is doing. There's two main problems with this proposal:

  • the EDUs as described have no reference point in the DAG, meaning servers won't agree on whether the EDU is authorised, as it depends on what their view of the current room state is. You need something like prev_events to anchor the auth rules.
  • Nothing stops a sender from equivocating and sending different EDUs to different servers, intentionally split-braining the room. This is a harder problem to solve, and ultimately requires non-sending servers to talk to each other to see if they have the same data and if not, to deterministically pick one. But once you do that... you need to persist the data so it can be queried later... and you need to have a succinct way of expressing the data like.. a hash, and oh look- we've reinvented PDUs.

commended, but not required.
Copy link

@TheArcaneBrony TheArcaneBrony Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does one define an "online" user?

Copy link
Contributor Author

@ananace ananace Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to avoid putting any specific requirements/definitions on what the best-effort guarantee should mean, hence why I'm using some more vague terms like "online".
I originally wrote the best-effort definition to be for any user that has an active sync request, but it ended up feeling rather arbitrary (not to mention leaving open the question of what would happen to EDUs that arrive while the user's client is busy processing the sync response), and also didn't really match with some of the user stories I had in mind while writing the MSC.

If I were to write my own Matrix server, with custom EDUs, I'd probably consider a size and time gated buffer for them, where all user sessions with an active sync request are guaranteed to receive the EDU, and any user session that starts a new sync request within the time gate - or before the buffer rolls over - may also receive it.
But that's probably quite different from how other servers would do it.

Copy link
Member

@anoadragon453 anoadragon453 Jul 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I was thinking about how to implement this a year or so ago, I had come up with the idea to let the sender of the EDU choose their delivery guarantee from a pre-defined set. The homeserver could either store it indefinitely until the receiver comes online (think to-device messages), or allow the receiver to miss it if they're not currently syncing (i.e. ephemeral live location events). In the latter case, the homeserver may only hold that data for 1m or so before expiring it. The same would apply to federation if the remote homeserver were down.

Copy link
Member

@kegsay kegsay Sep 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the only guarantee that should be required is that
all users that are online when the message is sent will receive it.

This is a weak guarantee which will make a bad UX as if you join a room sending print updates, you need to wait N seconds for it to appear, assuming perfect network connectivity. Having a stronger guarantee of "last write is shown" feels like a better UX for the majority of use cases which want custom EDUs (e.g location sharing). This is bounded data if you replace the EDU based on (sender, type) and have a restricted set of types in your power levels event.


### Extension of the Server-Server spec

As the server-server protocol is currently only designed for transferring the well-defined EDUs that
exist as part of the Matrix communication protocol, this proposal suggests adding additional fields
to the EDU schema in order to let them transmit the user-specified data untouched - while still
exist as part of the Matrix communication protocol, this proposal requires some additional fields to
be added the EDU schema in order to let them transmit the user-specified data untouched - while still
adding source information that is important for the receiving clients.

The suggested fields to add to the EDU schema are;
The fields to add to the EDU schema are;

- `room_id` (`string`) - The fully qualified ID of the room the event was sent in
- `sender` (`string`) - The fully qualified ID of the user that sent the event
Expand Down Expand Up @@ -289,10 +306,11 @@ A user-defined ephemeral event might then look like this when federated;
}
```

To reduce the scope of change required by this proposal, the suggestion is to allow the original
To reduce the scope of the changes required by this proposal, the suggestion is to allow the original
`m.*` events to skip these keys where no value can be easily assigned to them. E.g. aggregated typing
notices, receipt lists.


## Potential issues

Because this change completely redefines how ephemeral events are used, it is likely to be expected
Expand Down