Skip to content

Dynamic source tracks.#4128

Merged
toots merged 7 commits intomainfrom
dynamic-tracks
Sep 13, 2024
Merged

Dynamic source tracks.#4128
toots merged 7 commits intomainfrom
dynamic-tracks

Conversation

@toots
Copy link
Member

@toots toots commented Sep 11, 2024

There still are reports of issues with source.tracks forcing sources to compute their content_type before we had a chance to assign their frame_type.

For reference:

  • frame_type is a type representing the source's content as computed during type unification, e.g. {audio=pcm(stereo)}, {video=canvas, audio=pcm(5.1)} etc. This value can be incomplete if no full restrictions are put on the source content.

For instance, in:

s = blank()
output.dummy(s)

s can be any content because output.dummy does not care about the source's content. So, its frame_type is 'a

  • content_type is a fully realized content value. It is calculated from the frame_type, adding default values.

For instance, a source with PCM audio without a specified number of channels, {audio=pcm('a)} is assigned a content type of {audio=pcm(stereo)}.

The problem is that there can be a competition between requesting the source's content_type and assigning the frame_type. If content type is requested before all the frame_type have been assigned then the result can be incompatible, e.g. assigning a stereo channel when the source should have had 5.1 channels.

The most typical case is when computing source.tracks. In the current implementation, the tracks are all extracted immediately at evaluation time which forces the computation of the source's content type right away.

In this PR, we introduce a delayed Source_tracks runtime value. This value can be passed in-lieu of the actual record of source tracks and evaluated only when needed, e.g. in the muxer operator.

In order to make things work, we need to also introduce a notion of dynamic_methods so that we can do the following:

# s = blank();;
s :
  source('A)
  .{
...
  } where 'A is a set of internal tracks =
  <source(id=blank, frame_type=something that is a set of internal tracks>.{...}
# t = source.tracks(s);;
t :
  'A.{metadata : metadata, track_marks : track_marks}
  where 'A is a set of internal tracks = source.tracks(source=blank)
# s' = source({audio=t.audio});;
s' :
  source(audio='A)
  .{
...
  }
  where
    'A is a track and a track of type: pcm, pcm_s16, pcm_f32, canvas, metadata or track_marks =
  <source(id=source, frame_type={audio : 'A}
where
  'A is a track of type: pcm, pcm_s16, pcm_f32, canvas, metadata or track_marks and a track>.{...}
# t;;
- : 'A.{audio : 'B, metadata : metadata, track_marks : track_marks}
    where 'A is a set of internal tracks,
      'B is a track and a track of type: pcm, pcm_s16, pcm_f32, canvas, metadata or track_marks = source.tracks(source=blank)

It's a bit hacky but it seems to work fine.

@toots toots force-pushed the dynamic-tracks branch 2 times, most recently from 7c6935e to 0025859 Compare September 12, 2024 02:11
@toots toots changed the title Dynamic tracks. Dynamic source tracks. Sep 12, 2024
@toots toots marked this pull request as ready for review September 12, 2024 02:25
@toots toots requested a review from smimram September 12, 2024 02:28
@toots toots added this pull request to the merge queue Sep 13, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 13, 2024
@toots toots merged commit 310b5fc into main Sep 13, 2024
@toots toots deleted the dynamic-tracks branch September 13, 2024 17:56
toots added a commit that referenced this pull request Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant