Migrate video player from video.js to shaka-player

[absidue]

Migrate video player from video.js to shaka-player

Pull Request Type

• Bugfix
• Feature Implementation

Related issue

• closes [Bug]: Same video can start playing twice, as separate instances #5187
• closes 'Back to video player' doesnt work when not on video page #624
• closes Videos that aren't 16:9 are cut off by the bottom of the screen. #959
• closes Title not visible on white background videos #1348
• closes [Feature Request]: 1440p and 2160p qualities when using the local API #2013
• closes [Bug]: you cant exit out of fullwindowmode when there is a network error and the video stops working #2098
• closes [Bug]: Video starts playing from the beginning when the format is changed #2365
• closes Keybindings for the videoplayer are not successfully overriden #637
• closes [Bug]: Picture in Picture - Background Continue to Play Unable to Stop #4039
• closes [Bug]: Rewinding, entering/exiting Full Window/Fullscreen mode, and turning CC on/off reduces the captions font size #3920
• closes [Bug]: Screen tearing when switching qualities #3560
• closes [Bug]: SponsorBlock segment isnt skipped #3534
• closes [Bug]: Clicking fast through video qualities creates visual bug #3163
• closes [Bug]: Problem with subtitle centering with local language option #2931
• closes [Bug]: Attempting to pause a video in a popout window only pauses the main window - the popout window cannot be paused anymore #2455
• closes Long videos causes some problems #845
• closes Resolution qualities appear twice in quality selector #1344
• closes [Feature Request] Hide Progress Bar / Old Progress Bar Option #920
• closes [Feature Request]: hard to select between the timeline bar and video options buttons #3049
• closes Move Auto from top to bottom in player quality menu #1803
• closes [Bug]: Video is shifted and cropped when pausing in full screen after controls disappear #4983
• closes [Bug]: Player title getting stuck on fullscreen mode #5253
• closes [Bug]: Videos don't open in Theatre Mode even if "Enable Theatre Mode by Default" is checked #5252
• closes Display all characters of the title in one line. #857
• closes Limit resolution to what what is actually displayed #1129
• closes Minor change in labeling player buttons #1682
• closes Keyboard controls don't work after popup window is opened and closed #1606
• closes Caption settings background does not appear #1343
• closes [Bug]: Broken subtitle #2073
• closes [Bug]: Subtitle sizing are applied to current size, not original size #5615
• closes [Bug]: Subtitle sizing are applied every action #5616
• closes [Feature Request]: Buffer video further when connection comes back #4159
• closes Keeps on Spinning #5166 (adding it here for reference purposes, Discussions cant be closed by PR's)
• partially adresses (only some VR videos are supported) 360゜Camera Video Support #1460

Description

After almost a years worth of work, here is the shaka-player migration.

Significant improvements and new features

• vp9 and opus during DASH playback. vp9 only works on the local API and Invidious instances that have a version number of 2024.07.21 or newer.
• The video player will now match the users display language when possible (41/45 locales) and custom player elements added by FreeTube are now translatable. shaka-player doesn't support Estonian et (eesti keel), Basque eu (Euskera), Galician gl (galego) and Icelandic is (Íslenska), for those languages it will use English (US) instead.
• Stats are now supported for all 3 Media Formats, not just DASH.
• Switching formats will keep your playback rate, selected subtitle, audio language and video quality settings, additionally the player will no longer flash while switching.
• The player should be a lot more usuable on mobile screens, as player elements get moved into an overflow menu when they don't fit.
• You can now watch just finished live streams (Post-Live-DVR) from the start with the local API and you'll get thumbnails on the seek bar when they are available (the thumbnails seem to be missing on about 50% of Post-Live-DVR streams).
• The video player will pick up most settings changes without having to leave the watch page, such as changing the display language or the playback rate interval.
• Subtitles will now work on live streams that have them (very rare but they do exist).
• Support for VR videos that use equirectangular projection, shaka-player doesn't support YouTube's other projection types yet, local API only (The ANDROID_TESTSUITE client that Invidious uses, claims that all video streams use normal rectangular projection, so we don't have a way to identify VR videos through Invidious).

Significant bug fixes

• The subtitle translation feature now works on all videos, not just ones that are in English local API only.
• Starting at a certain timestamp from the video URL or resuming a partially watched video, should now work all the time, instead of just randomly.
• Long videos start straight away and no longer freeze FreeTube for almost a minute.

Missing/Removed features

• shaka-player doesn't have any subtitle settings. Maybe in the future we could implement our own ones, but it's not-trivial to implement, so creating a replacement is out-of-scope for this pull request.
• I removed the ability to seek into sponsor block segments that are set to skip, as it was causing the skipping to not work properly, due to the existing implementation being fundamentally flawed. If someone would like to add something like that back in the future, it should require a specific user-interaction to unskip the segment, e.g. a button in the skip message, instead of allowing users to manually seek into the segment.

Known issues

• Focus is not restored properly from exiting video player menus using keyboard navigation (will not be addressed in this pull request).

Screenshots

DASH

DASH with controls faded away

DASH in full screen mode

DASH in full window mode

DASH stats

Audio only with seek bar thumbnails and stats visible

Audio only with quality selector visible

DASH with a vertical video/short

Audio only with a vertical video/short and the FreeTube locale preference set to French

Mobile audio only with the the overflow menu closed

Mobile audio only with the overflow menu open

Mobile DASH with the overflow menu open on a video with subtitles and multiple audio languages

VR video that uses equirectangular projection

Testing

• HDR: https://youtu.be/njX2bu-_Vw4
• Dubbed and descriptive audio tracks: https://youtu.be/Kn56bMZ9OE8
• Audio and subtitle tracks in multiple languages: https://youtu.be/pU9sHwNKc2c
• Live: https://youtu.be/jfKfPfyJRdk
• 21:9 aspect ratio: https://youtu.be/Ttl8Gg-P-Ao
• YouTube short: https://youtu.be/hxQwWEOOyU8
• Chapters: https://youtu.be/Rb0k9vUCEno
• 100 hours long (has no seek bar thumbnails): https://youtu.be/56lkofpjOAs
• Subtitle formatting and colors: https://youtu.be/OwQwz07-DFc (The video stream has some burned in subtitles too, which we can't do anything about.)
• Non-seekable live stream: https://youtu.be/8M0AvPvPg0A
• Live streams with subtitles: https://www.youtube.com/@NBCNews/streams
• Live with auto-generated subtitles: https://youtu.be/Ap-UM1O9RBU
• VR video that uses equirectangular projection: https://youtu.be/H9r8CTsSf4U

Desktop

• OS: Windows
• OS Version: 10
• FreeTube version: 0.20.0

[absidue]

Relevant Invidious pull requests:

• Fix proxying live DASH streams iv-org/invidious#4589
  Current workaround: Use HLS manifests when proxying is enabled, which means no seeking but at least you can still watch the live stream.
• API: Return actual stream height, width and fps iv-org/invidious#4586
  Current workaround: Fetch the DASH manifest in addition to making the API request, as their DASH manifest doesn't use harcoded values. However as Invidious' player doesn't support WebM DASH, they only include the h264 and AV1 streams in the DASH manifests, so supporting vp9 when Invidious is used as the backend or proxying is enabled is not currently possible.

[PikachuEXE]

Tested live streams
Re-tested auto generated caption for 4.0.11
No idea what else or if anything else should be tested

[PikachuEXE]

Player settings > Display play button in video player

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[PikachuEXE]

No further tests from me (except daily test via custom build) until something else popups up

[ChunkyProgrammer]

I'm approving under the assumption that we will do a Release Candidate sometime before the next full release

[absidue]

A release candidate sounds good to me, as this pull request contains a lot of changes.

[PikachuEXE]

Let's RC