improve TTML rendering

[valotvince]

This PR adds:

• a notion of nested cue, that renders inside the parent cue
• a first support of spans inside paragraphs

How to test:

• Live MPD w/ TTML subtitles https://irtdashreference-i.akamaihd.net/dash/live/901161/bfs/manifestARD.mpd
• VoD w/ French subtitles https://e213.cdn.m6web.fr/videos/test_pixel/usp/mb_hd2/subtitle/NCIS-TEARS2_chromecast_unp_sd_dash_v1.ism/Manifest.mpd (the subtitles are not in sync w/ the video, this was only made for debugging purpose)

TODO

• Unit tests: text_displayer(s), ttml_text_parser
• Rebase branch on master
• Follow the Contributing guidelines fully

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[valotvince]

I signed it!

[googlebot]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[ismena]

Hi @valotvince, thanks for the PR!
Before we get deep into code review, could you help me better understand what this change is doing?
Is it coloring some parts of the subtitles based on a TTML attribute?
Could you give me an example of a TTML that specifies this (doesn't have to be real, made up is fine) and what you expect to happen.

[valotvince]

Hi @ismena, sorry for the wait, I am on vacation :)

Lets suggest we have the following TTML extract:

<p region="..." begin="..." end="...">
  <span style="style01">A text</span>
  <br />
  <span>Another text</span>
</p>
<p region="..." begin="..." end="...">Text</p>

As of today, Shaka won't read styles in each span of paragraphs, but will merge the spans in one and only cue: A text\nAnother text (only printed with the paragraph color)

That PR is makes Shaka read each span within paragraphs. The spans are read as independent cues with their own region/begin/end attributes (based on the closest ancestor who possesses them), so two cues will be printed out (with their own style definitions based on the text displayer in use):
A text (maybe printed with a yellow color)
Another text (maybe printed with a red color)

The counter-part of doing so, is that the paragraph style section won't be used in the span styles... But as I've seen in issues on Github, the text displayers and TTML parser would need a little refactoring to be able to meet the TTML specs.

Hope I was clear enough :)

Best,

[ismena]

No worries, hope you had a great vacation :)

Ah, gotcha. Let's get to it! I left a few questions on the CL, here's another one: do you have content you're testing with? I'd love to cherry-pick your change and try it.
Feel free to send it privately to shaka-player-issues@ if you'd like.

[valotvince]

Hi @ismena,

I have some private streams that I can't share but I was also testing alongside the streams found in that article: https://subtitling.irt.de

• https://irtdashreference-i.akamaihd.net/dash/live/901161/bfs/manifestARD.mpd
• https://irtdashreference-i.akamaihd.net/dash/live/901161/bfs/manifestBR.mpd

[valotvince]

Hi @ismena,

I've committed to add some features for TTML rendering (I've followed some of the ideas of #1080).

My company's video team is working to give you a public manifest with some TTML features like positioning and colours, I'll update here as soon as I have it :)

Best,

[googlebot]

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of all the commit author(s), set the cla label to yes (if enabled on your project), and then merge this pull request when appropriate.

ℹ️ Googlers: Go here for more info.

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[ismena]

Why?

[valotvince]

I've changed the flex-direction for the cues to display on a column so I had to change the flex rules on the container.

That was mainly because I've tested prior to the support of span of p, I'll revert this to see if it still works ok !

[ismena]

Thanks for replying to my comments! I'll try your changes on the content you provided and will get back to you early next week.
Please ping me on this mid next week if I get swamped with other stuff and forget (I do that sometimes :( )

[ismena]

I tried the changes out - looks good! The (hopefully) last round of nitpicks, mostly style.

[valotvince]

@ismena Hi :) Just a ping to let you know I've taken account of you review in the last commit :)

[ismena]

@valotvince Awesome and thank for pinging.
Please rebase the commit on top of the latest master and I will run the tests and approve the commit if the tests are ok.

[valotvince]

@ismena Done :)

[shaka-bot]

Test Failure:

Generating Closure dependencies...
Linting JavaScript...

/var/lib/jenkins/workspace/Manual PR Test (local-tests)/test/text/ttml_text_parser_unit.js
853:1   error  Line 853 exceeds the maximum line length of 80       max-len
853:45  error  Expected parentheses around arrow function argument  arrow-parens

✖ 2 problems (2 errors, 0 warnings)
1 error and 0 warnings potentially fixable with the `--fix` option.

END-BUILD: FAILURE
Build step 'Execute shell' marked build as failure

[ismena]

Looks like there are some linter problems. Please run build/all.py locally to see what needs resolving.

[valotvince]

@ismena Sorry, forgot to run it after the rebase 💯 Should be okay now !

[shaka-bot]

All tests passed!

[ismena]

@valotvince Cool! Thanks for contributing :)

[avelad]

@ismena after this PR, CE608 and SMPTE-TT subtitles are broken. Please review it! (Asset: Live sim TTML Image Subtitles embedded (VoD) )

[avelad]

The issue is related with the next if: c670b55#diff-226b4f50d0392c977a71a42816ecdad1R246

[valotvince]

Hello @avelad, could you provide an example source so I could work on a fix ? Got it ;)
Sorry for that !

[ismena]

Ah, damn it! We should look into adding unit tests for this so it doesn't happen again. Thanks for spotting, @avelad and @valotvince for the fix. I'll get right on to reviewing it.

[valotvince]

Hi there ✋Any idea when that diff will be released ? Thanks !

[ismena]

We should've released this in 2.5.5 that just went out :( @joeyparrish @TheModMaker can we include this in the next minor release?

[joeyparrish]

I'll delegate this to @TheModMaker. If it makes sense to cherry-pick it, please cherry-pick to v2.5.x now, and it will show up in the next release, v2.5.6.

[TheModMaker]

I've cherry-picked these commits to v2.5.x, and they'll appear in the v2.5.6 release.