-
Notifications
You must be signed in to change notification settings - Fork 423
MSC3291: Muting in VoIP calls #3291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
78bfa25
a3d1abc
57a6330
7658c4a
2130df9
3a9701d
d809c3b
67168b3
ec24d95
0c01908
cc8b73a
f1ac61f
29e3c6d
5372b7f
00da3d3
fb8f957
c97f650
1aa2cd4
dec84a8
56d3541
8905378
4da6f86
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,19 +8,17 @@ to be quick. | |
|
|
||
| Using pure WebRTC there are two ways to do muting and both have their issues: | ||
|
|
||
| + Disabling the corresponding track - there will be no way to know the track is | ||
| muted from the opponent's perspective. On the other hand, this is almost | ||
| instantaneous. | ||
| + Setting the corresponding track as `recvonly`/`inactive` - this is problematic | ||
| since it leads to re-negotiation which takes time. This could also conflict | ||
| with holding (as defined in | ||
| [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). | ||
| + Disabling the corresponding track | ||
| + Setting the corresponding track as `recvonly`/`inactive` | ||
|
|
||
| The Alternatives section describes the issues with using these alone. | ||
|
|
||
| ## Proposal | ||
|
|
||
| This MSC proposes a solution to this by extending the `SDPStreamMetadata` (see | ||
| [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) by the following | ||
| fields: | ||
| This MSC proposes that clients disable muted tracks (as is almost instantaneous) | ||
| and extends the `SDPStreamMetadata` object (see | ||
| [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) to allow | ||
| indicating the mute state to the other side using the following fields: | ||
|
|
||
| + `audio_muted` - a boolean indicating the current audio mute state | ||
| + `video_muted` - a boolean indicating the current video mute state | ||
|
|
@@ -65,6 +63,40 @@ muted. | |
| When the user mutes their camera, the client will meaningless which will waste | ||
| bandwidth. | ||
|
|
||
| ## Alternatives | ||
|
|
||
| ### Only disabling the corresponding track | ||
|
|
||
| This is the solution that some clients (e.g. Element) use at the moment. While | ||
SimonBrandner marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| this is almost instantaneous, it doesn't allow the other side to know the | ||
| opponent's mute state. This leads to the opponent showing a black screen for a | ||
| muted video track and not doing anything for a muted audio track which is bad | ||
| for UX. | ||
|
|
||
| ### Setting the corresponding track as `recvonly`/`inactive` | ||
|
|
||
| While this would be beneficial for low bandwidth connections, it takes time. The | ||
| delay might be acceptable for video but isn't for audio (with which your would | ||
SimonBrandner marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| assume an instantaneous mute state change). This is also problematic since there | ||
| could be a confusion with holding (as defined in | ||
| [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). | ||
|
|
||
| ### Using a separate event for muting | ||
|
|
||
| While this might feel clearer initially, it doesn't have much real benefit. The | ||
| mute state is in fact a meta information about the stream and using | ||
| `sdp_stream_metadata` is also more flexible for cases where the user joins a | ||
| call already muted. It is also more flexible in general and would be useful if | ||
| we ever decided to do what is described in the next section. | ||
|
|
||
| ### A combination of disabling tracks, `sdp_stream_metadata` and SDP | ||
|
|
||
| An option would be using the current method in combination with setting the | ||
| corresponding track as `recvonly`/`inactive`. Along with this clients would need | ||
| to set the mute state in `sdp_stream_metadata` to avoid conflicts with holding. | ||
SimonBrandner marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| While this solution might be the most flexible solution as it would allow | ||
| clients to choose between bandwidth and a mute state change delay for each | ||
| track, it would be harder to implement and feels generally disjointed. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are we sure we don't want to go for optimal bandwidth usage? I personally often turn off video to have more bandwidth available for audio when the quality is terrible. Feels unfortunate to exclude this. Also, this doesn't feel much harder to implement.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thinking about this further, why not remove the track entirely and send a
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this should be the choice of the client. In general not sending the video when video is off makes sense. The extra latency of starting up the stream is usually quite small and the bandwidth savings is significant which can make a huge difference for someone on a connection that struggle to send audio and video but can support audio reliably. Turning off the camera is less often used because actually getting a new stream from the camera can have very high and unpredictable latency depending on the hardware or the OS. Probably a good client will have a variety of heuristics to decide how far to shut down the camera but the key thing from the spec point of view that completely turning off the camera is possible if desired.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think I stand against going with both options at the same time - let the client choose. @dbkr, would you agree?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FWIW, we do this now: matrix-org/matrix-js-sdk#3028
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, I believe Enrico recently discovered with his experiments that browsers (at least some of them) now send nothing when a video track is disabled, so the bandwidth argument is even less relevant. Turning cameras off is orthogonal and can be done whether or not you remove the video track. The negotiate mechanism still exists either way.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Is this documented anywhere?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the MSC need to be updated given the conclusions drawn here?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm going to assume not and resolve this thread. Please re-open if you believe otherwise. |
||
|
|
||
| ## Security considerations | ||
|
|
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.