Problem with AVX2 SIMD

[Jamaika1]

When do I use SIMD AVX then codec works.
When do I use SIMD AVX2 then codec doesn't work.

Compiled: INTEL, flags: MMX SSE SSE2 SSE3 SSSE3 SSE41 SSE42 ???
Detected: INTEL, flags: MMX SSE SSE2 SSE3 SSSE3 SSE41 SSE42 AVX AVX2
Available: avx2(5)
In use: avx2(5)
--owf=auto value set to 3.
--threads=auto value set to 20.
Output #0, matroska, to 'output_kvazaar.mkv':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf61.5.101
Stream #0:0(und): Video: hevc, 1 reference frame, yuv420p(progressive, left), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 3000 kb/s, 29.97 fps, 1k tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
encoder : Lavc61.10.100 libkvazaar
Stream #0:1(und): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
encoder : Lavc61.10.100 aac
[out#0/matroska @ 000001e63341b2c0] Starting thread...
Assertion failed: cur_cu->type != CU_NOTSET, file ../../src/search.c, line 1044its/s speed= 0.6x

[Jovasa]

Hello, could you please provide some additional information, for example:

• What is the used version of Kvazaar?
• Compiler?
• Were some non-default compile flags used?
• Build system?
• Used CPU?
• Operating system?
• You appear to be using kvazaar in FFmpeg? What is the used version?
• What are the options provided for Kvazaar?

[Jamaika1]

https://forum.doom9.org/showthread.php?p=2004554#post2004554

[Jovasa]

Yeah, that does not really, help. I can see that you are using FFmpeg 7.1.0 (what is this even? the latest FFmpeg release is 7.0.1), compiled with GCC 11.3.1, but this is not nearly enough that I could start to debug where the issue is.

[Jamaika1]

https://github.com/FFmpeg/FFmpeg/releases/tag/n7.1-dev

I use:
Windows 11 Pro 23H2
http://msystem.waw.pl/x265/mingw-gcc1131-20221227.7z
13th Gen Intel(R) Core(TM) i5-13600K 3.50 GHz
AVX2: gcc.exe -std=gnu11 -static -march=x86-64-v3 -ftree-vectorize -g0 -O3 -fPIC -mavx2 -mpclmul -maes -mbmi2 -mfma -mf16c -Wcomment -Wformat -DWINVER=0x0602 -D_WIN32_WINNT=0x0602 -DWIN32_LEAN_AND_MEAN=/"/" -DARCH_X86_64=1 -DSRM_REPORT=1 -c xxx.c -o xxx.o
AVX: gcc.exe -std=gnu11 -static -march=x86-64-v2 -ftree-vectorize -g0 -O3 -fPIC -mavx -mxsave -mpclmul -maes -Wcomment -Wformat -DWINVER=0x0602 -D_WIN32_WINNT=0x0602 -DWIN32_LEAN_AND_MEAN=/"/" -DARCH_X86_64=1 -DSRM_REPORT=1 -c %%f -o %%~nf.o

ffmpeg_avx2.exe -v verbose -i "input.mp4" -y -c:v libkvazaar -vb 3000k -c:a aac -ac 2 -ar 48000 -ab 128k -vf scale=1920:1080 -frames:v 1000 -pix_fmt yuv420p output_kvazaar.mkv

https://www.sendspace.com/file/ehtanv