libva lacks a way for user/app to get the actual parameter sets supported by the HW before encoding begins

[nyanmisaka]

As the title suggests, libva currently assumes that the VA_ENC_PACKED_HEADER_SEQUENCE written by the user or application is fully compatible with the actual bitstream encoded by the hardware. However, in practice, this assumption does not hold true for hardware other than Intel QuickSync.

Starting with FFmpeg 8.0, the MP4 muxer strips in-band parameter sets from videos using the HEVC codec_tag hvc1, leaving only the incompatible VA_ENC_PACKED_HEADER_SEQUENCE written by FFmpeg.
FFmpeg/FFmpeg@f005160

This causes decoding failures for the encoded video. Affected players include, but are not limited to, Safari, Chrome, Firefox, FFplay, and MPC-BE. The affected videos cannot be used in HLS fMP4 segments (which is why hvc1 is needed).

• https://gitlab.freedesktop.org/mesa/mesa/-/work_items/15529

Matroska/MKV files also require AV_CODEC_FLAG_GLOBAL_HEADER, so similar issues arise there as well.

• https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/22801
• Advertise packed sequence headers for encode radxa-pkg/libva-v4l2#7

Mesa/AMD developer have provided a suggestion:

What would be a good solution is to have an option to only get the encoded headers, eg. a new flag that will disable slice output. App would do the exact same calls as when encoding a frame, except set this new flag. But then again, this is just like a dummy encode which apps can do today and it will work with all drivers. Since it's only needed once at the start of the session, the overhead of one encode job is probably not a big deal.

Could libva consider adding a method to obtain HEVC parameter sets from the vaapi driver before encoding the first frame?

Thanks in advance! @XinfengZhang @nowrep