-
Notifications
You must be signed in to change notification settings - Fork 179
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
QPA causes significant detail loss in static parts of video #369
Comments
I encoded the source in x264@CRF21 and h266@QP28 files but couldn't see any difference when playing back the files at the same time in single step mode: The video is a bit odd with alternative bright and dark frames in places. The clouds seem to move on every frame but the anime moves on alternative frames - maybe this is normal. |
Sure, there's no difference... if you don't count the texture and lineart loss. |
Thanks, we'll have a look. |
POC0IDR will add orphan IDR frame sometimes, quite often actually. Amend: the statement above is not correct. SAO in help message is default to 0 but actually not. There's a roughly +5 offset in qp/bitrate relation for qpa in this setting. Amend: QPA can cause unwanted quality deterioration in some frames under its current situation fraunhoferhhi/vvenc#369
Hi joletb,
Can you clarify what you mean by 'static'? The video scene features quite some motion, but 'static' to me means not moving.
So, is (to your eyes) the QPA version of the video still better overall than the non-QPA version when played at normal speed? I have to admit that, when comparing the 'qpa/noqpa' bitstreams you shared (thanks), I cannot reliably spot any differences. Are your observations based on watching the video during playback, or just on inspecting isolated frames (i.e. screenshots)? Note that VVenC's QP adaptation makes use of spatial as well as temporal psycho-visual modelling. It's based on the XPSNR visual model, details of which you can read up on in this open-access paper: http://handle.itu.int/11.1002/pub/8153d78b-en This means that VVenC's QPA is solely intended, and optimized, for videos at typical frame rates, NOT for still image coding. Or, in other words, screenshots of individual frames don't tell the whole (perceptual) story. That said, I'll take another look at the issue with the first frame in the video. Christian |
The "static" part here refers to the to the duplicate parts of the building in the video, there is practically no motion between them there and as such there should be no detail loss. (If we're going this route, should the detail loss be extended to the duplicate parts to make the video more consistent?) That being said, I don't think this is entirely a QPA issue as this does not happen when you remove the first frame of the sample. (Both the original sample and this one mentioned below are perceptually the same) |
I'm an absolute newbie in video encoding but to me this issue looks similar to #212. |
It's not. |
Maybe the problem is the opposite: the subsequent frame of the "problematic" frame is actually getting too many quality. Tested with the y4m clip provided by joletb ,arguments used: With QPA:
Without QPA:
Notice how frame 17, 19, 21 and so on jumped from QP 27-ish to 19-ish between two samples while other frames are less significantly changed. |
Thanks, L4cache, for the detailed analysis! In case you have a hex editor available and know how to use it: could you please
This effectively changes the frame rate from 23.976 to 59.94 fps for testing and should help to isolate the problem. Thanks, Christian |
To make things more clear, I'm using the original file The 59.94 fps reports as follows
But... it decodes with an error (both ffvvc and vvdec), in frame 32, it seems like the error is caused by
|
Great, thanks! With those statistics, I can confirm that there are 2 separate issues here: one at frame 0 (reported by joletb), and one at frames 17, 19, 21, ... (reported by L4cache). The second one is relatively easy to fix and I'll start working on that soon. The first one is a bit harder to fix, so will take some time. However, the fix to the second issue will require MCTF to be enabled (to have access to certain picture statistics). @joletb, is there a particular reason why you chose to set Christian |
Heya, Christian, great to hear progress is being made on this particular issue! To address MCTF, it being set to 0 is mostly a remnant from my previous encoding tests. (Whose videos were far more noisy) Unfortunately, I am not able to produce any bitstreams until later next week, so @L4cache if you could help out a bit more with this, it'd be greatly appreciated! |
Apologies for the delays, life has been getting busy. Here are the stats for the MCTF encode, command line is:
And here are the stats of the original command line in case they are necessary:
I've additionally uploaded the MCTF encode in the GDrive folder in the first post. |
Is this the part of the fix merged? |
While QPA gives quite good increases in fidelity in intra coding, it can cause static parts of a video to lose all detail in inter coding, even when there is a good reference frame available.
Furthermore, if we remove the first frame from our sample below, this detail loss is remedied and the encoder properly uses info from previous/next frames, signifying a possible issue with the GOP structure when used with QPA.
This issue does not happen at all in non-QPA encoding, but then detail preservation in general is quite a bit worse.
The following command line was used for the encoding:
vvencffapp -c config/slow --y4m 1 -q 20 -t -1 --PrintFrameMSE 1 --PrintSequenceMSE 1 --ALF 1 -ip 0 -rs 11 --CCALF 1 --STA 0 --WppBitEqual -1 --MaxParallelFrames -1 --SAO 0 --qpa 1 --BIM 0 --MCTF 0 --InputFile - -b slow.266
Here are a couple of comparisons: https://slow.pics/c/koHLpziy
Note also that this issue is significantly more prevalent in anime as there tend to be a lot of duplicate frames.
The encoded bitstreams and the video source used for encoding can be found here: https://drive.google.com/drive/folders/1OAsQxfd6y-Xl8u1syeXkZ9u22scQD0YG?usp=sharing
The text was updated successfully, but these errors were encountered: