[Bug]: GPU Hang when decoding HEVC 4K UHD 10-bit 4:4:4

[MartinPulec]

Which component impacted?

Decode

Is it regression? Good in old configuration?

None

What happened?

Hi, I am encouraging "device failed" and subsequent "GPU Hang" when decoding 10-bit 4K UHD 4:4:4 (YCbCr) video with cartwheel-ffmpeg. I've also described the issue here.

HW I am using - Intel NUC12WSKi7 with i7-1260P (so Alder Lake-P GPU), Ubuntu 22.04 and Intel Media Driver 2.6.6. Steps to reproduce:

$ ffmpeg -f lavfi -i testsrc=size=3840x2160 -t 1 -pix_fmt yuv444p10le -c:v libx265 testsrc.mp4
$ ffmpeg -c:v hevc_qsv -i testsrc.mp4 -f null -

Produce couple of GPU Hang errors. No output is produced if I replace -f null - with output file name (4 empty/green YCbCr frames; input has 25). Adding -hwaccel qsv+-vf hwdownload doesn't make any difference.

Anyways, the decoding pipeline works correctly with different properties, changing either:

1. resolution - even 3832x2160 [sic!] works correctly. Which is very interesting, because I assume that 3833 (not working) rounds up to 3840 because of macroblock alignment. So it looks like there is some threshold
2. subsampling - both yuv422p and yuv422p10le are ok, yuv444 not, so bit depth perhaps doesn't play a role

It doesn't seem to be a wrong peace of HW - IIRC one of our users reported this also with a different NUC (but most likely the same type). Also, the error message 'GPU Hang' doesn't mean that the graphics hung except the decode, it is normally operable after that.

ffmpeg.log

What's the usage scenario when you are seeing the problem?

Others

What impacted?

We use QuickSync as a HW acceleration API for UltraGrid.

Debug Information

1. libva is 2.16 (it is from this repo), Intel Media Driver is 22.6.6 from GH
2. ls--dev-dri.log
3. vainfo.log
4. libva_trace.log.144857.thd-0x00003e1a.log, libva_trace.log.144857.thd-0x00003e15.log
5. dmesg.log

Do you want to contribute a patch to fix the issue?

None

[Jexu]

It seems not real GPU hang here since it dosn't show Hang in dmesg. But something in SW like copy may cause it. Can you share your bitstream for further analysis.

[MartinPulec]

It was the bitstream generated by FFmpeg and libx265 by the command above. It was reproducible also with stream generated similarly by hevc_qsv encoder.

However, I'd have an update on that. We have been solving something with your colleague who suggested us to enable HuC. After doing that (and using newer kernel 6.0, because ADL-P requires at least 5.17), the problem seems to have disappeared. Previous kernel was 5.15 (Ubuntu 22.04 default).

The question is we want to solve this further when it works in this setup. If so, would you still like to have the bitstream attached (even though generated by FFmpeg)?

[Jexu]

You could confirm firstly that if your problem is related to HUC enabling or kernel incompatible, while ADL-P released version is kernel5.17+ with HUC enabled default. So incompatible version or confirguration may have protential unknown issues that was not verified ever. It may have low priority if you still want to find out what causes that on this setup.

[intel-mediadev]

Auto Created VSMGWL-61281 for further analysis.

[MartinPulec]

@Jexu it's kernel version dependent. After removing enable_guc=2, HuC remained still enabled (as you write), with enable_guc=0 not. But the described problem didn't occur in neither case with the new kernel. If 5.15 is unsupported we could perhaps close it.

[Jexu]

Yeah, 5.15 doesn't fully support adl-p and I will close this issue now. You can re-open it agasin if still having other questions after updating kernel.