[WIP] Add screensharing support.

[fancycode]

This PR adds support for screensharing (#31).

Currently more a tech demo 🙈 various parts still need implementation:

• Chrome support (see https://github.com/nextcloud/spreed-screensharing-chrome-extension for the required extension)
• Firefox support (see https://github.com/nextcloud/spreed-screensharing-firefox-addon for the required extension)
• Add custom icon to toggle screensharing (currently shows a play icon below the avatar)
• Better layout of the streams (currently only renders stacked below the other videos)
• Feedback to the user about why it could not be started (currently logs through console.log in app.js)

cc @jancborchardt for ui, @nickvergessen @LukasReschke, @Ivansss for general review

[nickvergessen]

😒

[fancycode]

This one actually would allow wildcard hostnames, it's the matches below that doesn't allow it.
I'll look if there is another way to communicate between website and the extension, or if we have to deal with the "one extension per hostname".

[fancycode]

Updated extension to work with all domains 🎉

[fancycode]

I think we should develop the extensions in separate repositories as their release cycle (and versioning) will be different from the app itself. What do you think?

[nickvergessen]

missing ,

[nickvergessen]

chrome-screensharing-extension ?

[codecov-io]

Codecov Report

Merging #227 into master will decrease coverage by 0.39%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #227     +/-   ##
===========================================
- Coverage     17.31%   16.92%   -0.4%     
- Complexity      240      291     +51     
===========================================
  Files            16       16             
  Lines          1155     1312    +157     
===========================================
+ Hits            200      222     +22     
- Misses          955     1090    +135

Impacted Files	Coverage Δ	Complexity Δ
lib/Controller/ApiController.php	0% <0%> (ø)	113% <0%> (+41%)	✅
lib/Notification/Notifier.php	95.19% <0%> (+1.28%)	27% <0%> (+10%)	✅

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c29495a...ae9d60d. Read the comment docs.

[nickvergessen]

Created https://github.com/nextcloud/chrome-screensharing-extension

[jancborchardt]

Interface-wise what about:

• »Share screen« and the current »Fullscreen« function should be in a (to be created) popover menu of a 3-dot-menu next to the audio and video buttons of your own video.
• If someone shares a screen then that screen should be shown as the promoted (big) video permanently as long as it’s shared. Sized so that it’s not overlayed by the participant videos below though.
• Should we limit screensharing to one screen at a time? Otherwise we need to reconsider the interface.
• Anything else?

[fancycode]

Firefox support is now blocked on #234 (no PeerConnection is being established, so also no screensharing possible).

[nickvergessen]

log is only done in debug mode, I guess it would be fine to keep.

[fancycode]

@Ivansss did you make sure to keep my screensharing-related changes (and any other changes that might have happened in the past) in your simplewebrtc upgrade? Maybe it would make sense to upgrade in a separate PR.

[Ivansss]

@fancycode Yes, I applied all the changes we did in simplewebrtc.js since the initial commit on top.

[Ivansss]

Reverted SimpleWebRTC update, doing it in a different PR #242 .

[nickvergessen]

Rebased and simply dropped the commit and the revert.

[fancycode]

FYI, you could have rebased only up to my last commit to avoid getting new hashes for all old commits.

[jancborchardt]

Talked more with @Ivansss how to proceed:

• clicking the shared screen increases the size and breaks it
• when sharing a screen, we need to change the layout to have the screenshare max height not be overlapped by the videos of the other people. Also both videos need to be visible. So we need to switch to the 3-person-layout as soon as screensharing begins, with your own video staying in the bottom right and the video of the other person in the bottom left.
• screensharing will be a third icon next to audio and video
• by default the icon should be showed slightly transparent, just like the audio and video icons when muted / disabled
• fullscreen will move to the top right of the #app-content because it relates to the whole call, not only to your view or your things. Shouldn’t be shown in the emptycontent view because then the sidebar isn’t shown and you can’t join a call.
• I’ll do a proper icon for screensharing and add it to core
• the icons should get tooltips (the dark JS ones) saying – depending on state: »Mute audio« / »Unmute audio«, »Disable video« / »Enable video«, »Share screen« / »Unshare screen« and »Fullscreen« / »Exit fullscreen«
• when clicking the screensharing icon and the extension is not installed yet, show a popover prompting to install the extension, with a link fitting for the browser. If it’s Safari or something else, have the popover say »For screensharing, please use Firefox or Chrome and install the Nextcloud screensharing extension.« – with Firefox and Chrome being linked to the respective pages. This should not be a showNotification which is very far away from the icons but rather an arrowed popover over the icon, just like it looks for the 3-dot-menu
• in a similar vein, the error message »Screensharing is not supported by your browser.« shows when using Firefox and a HTTP instance. Correct would be »Screensharing is only possible over https«. Again, as a popover over the icon instead of a showNotification on the other edge of the screen.
• publish the extensions of course, and give it a Nextcloud icon (like the favicon)

[jancborchardt]

Icon PR at nextcloud/server#3480 and stable11 backport at nextcloud/server#3481

Let me know if you need anything else here. :)

[Ivansss]

Rebased to use SimpleWebRTC v2.2.3.

[Ivansss]

@jancborchardt Could you please take a look to the current state? I would like to know your opinion about the layout when a screen is being shared and discuss how to improve it 😄

[nickvergessen]

Addons are now published:

• Firefox Addon
• Chrome Extension

[jancborchardt]

@Ivansss I’m using Google Chrome 55 with the extension on Ubuntu 16.04 and on clicking the »Share screen« it says »Screensharing is not supported by your browser«. :\

But generally judging from the screenshots it looks great! :) There’s just two things:

• I would change the background of the content from this grey to black. (Below the shared screen)
• When video of one participant is deactivated, the layout of the current speaker avatar is wrong:
  Master (correct):
  
  This branch (see the avatar on the bottom and a bit to the right):
  

I can look into both, but for that I have to get the actual screensharing working :\ we could also merge this first and polish it forward?

[nickvergessen]

Works fine here, are you having your netflix-cheat in place again?

[jancborchardt]

@nickvergessen no, I also just disabled all extensions and it’s the same issue. We are also trying on @gsambrotta’s setup and couldn’t get a video call to run at all yet :\ (@gsambrotta best open an issue about it).

I’ll try if it works with Firefox.

[jancborchardt]

Oh and screensharing doesn’t work for public call guests, does it? Because the icon is shown and nothing happens on clicking it.

[Ivansss]

It should work for guests too, but I noticed that notifications are not shown for guests.

So probably in your case »Screensharing is not supported by your browser« notification is being triggered, but you can't see it because of the problem with notifications for guests.

[gsambrotta]

no problem from my side, just wrong branch (oooops!)

[jancborchardt]

@Ivansss ah ok, can you then make sure the notification – if it’s not supported – is also shown for guests?

Will need to see what’s wrong with my setup then 😕

[jancborchardt]

@Ivansss @nickvergessen ok, I changed the grey background to black in the CSS.

If there’s no layout issues for you, and the rest of the issues in my other comment at #227 (comment) are fixed (or should be postponed) and the notification about screensharing not being available is also shown for guests → we can merge this! :)