Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve VFR support #393

Merged
merged 2 commits into from
May 13, 2024
Merged

Improve VFR support #393

merged 2 commits into from
May 13, 2024

Conversation

j0sh
Copy link
Collaborator

@j0sh j0sh commented Apr 11, 2024

Improve VFR support ( 55a5d80 )

Manually calculate the duration of each frame and set
the PTS to that before submitting to the filtergraph.

This allows us to better support variable frame rates,
and is also better aligned with how ffmpeg does it.

This may change the number of frames output by the FPS
filter by +/- 1 frame. These aren't issues in themselves
but breaks a lot of test cases which will need to be updated.

Update testcases for VFR ( 1986cd4 )

Notes

Previously we were depending on a calculated frame rate (r_frame_rate)
but this isn't set to a sensible value for VFR streams. Packet duration also
can't be relied on (see the new VFR test case where each duration is N/A),
so we need to do some manual bookkeeping to calculate the timestamp
difference between frames ourselves.

I haven't yet fully understood what happens with the FPS filter during the
transition between segments but the behavior seems okay and the
timestamps don't change too much, aside from the +/- 1 frame difference.

Most unit tests pass on my machine, except I did not test GPU or non-H264
codecs. I don't think things would be much different there, but someone with
a GPU should check. The tests in ffmpeg/sign_test.go all timed out for me
and there is a segmenter test that seems flaky, but these are probably unrelated.

Manually calculate the duration of each frame and set
the PTS to that before submitting to the filtergraph.

This allows us to better support variable frame rates,
and is also better aligned with how ffmpeg does it.

This may change the number of frames output by the FPS
filter by +/- 1 frame. These aren't issues in themselves
but breaks a lot of test cases which will need to be updated.
Copy link

codecov bot commented Apr 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 50.20216%. Comparing base (684658a) to head (1986cd4).
Report is 4 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@                 Coverage Diff                 @@
##              master        #393         +/-   ##
===================================================
+ Coverage   49.88901%   50.20216%   +0.31315%     
===================================================
  Files             12          12                 
  Lines           1802        1484        -318     
===================================================
- Hits             899         745        -154     
+ Misses           852         689        -163     
+ Partials          51          50          -1     

see 12 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ced71c4...1986cd4. Read the comment docs.

see 12 files with indirect coverage changes

@j0sh
Copy link
Collaborator Author

j0sh commented Apr 12, 2024

Another quick note that I forgot to mention. I am not sure how this fix would be rolled-out operationally, but because this will produce different frame counts (and thus different timings), it is important that streams be "pinned" to the same version of LPMS if some transcoders are slow to update themselves. Otherwise there might be some breakage in between segment intervals, eg timestamp overlaps or gaps.

@leszko
Copy link
Contributor

leszko commented Apr 15, 2024

Another quick note that I forgot to mention. I am not sure how this fix would be rolled-out operationally, but because this will produce different frame counts (and thus different timings), it is important that streams be "pinned" to the same version of LPMS if some transcoders are slow to update themselves. Otherwise there might be some breakage in between segment intervals, eg timestamp overlaps or gaps.

Good point, thanks for mentioning it!

@leszko leszko merged commit 11a5584 into livepeer:master May 13, 2024
5 checks passed
j0sh added a commit to j0sh/lpms that referenced this pull request Jun 19, 2024
The nvidia test suite was never run after livepeer#393 so this breakage
was not noticed.
j0sh added a commit that referenced this pull request Jul 10, 2024
The nvidia test suite was never run after #393 so this breakage
was not noticed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants