615.71.09: HF-VSDB DSC_MaxSlices=7 now parsed as 12 slices @ 600 MHz, forcing YCbCr 4:2:2 limited on HDMI 2.1 (7680x2160@240, Samsung Odyssey G95NC, RTX 5090)

[Masorian06]

NVIDIA Open GPU Kernel Modules Version

615.71.09 (tag 615.71.09; installed from NVIDIA's Ubuntu 24.04 local repo, DKMS package nvidia-kernel-source-open 615.71.09-1ubuntu1). Regression from 610.57.04, which works.

Please confirm this issue does not happen with the proprietary driver (of the same version)

• Not applicable: the GPU is Blackwell (RTX 5090), which is only supported by the open kernel modules. The affected code is part of this repository (src/common/modeset/timing/nvt_edidext_861.c and src/common/modeset/hdmipacket/nvhdmipkt_C671.c).

Operating System and Version

Ubuntu 24.04.5 LTS

Kernel Release

Linux mario-linux 7.0.0-31-generic #31~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Aug 10 09:38:02 UTC 2 x86_64 GNU/Linux (Ubuntu stock kernel, not self-built)

Please confirm you are running a stable release kernel

• I am running on a stable kernel release.

Hardware: GPU

GPU 0: NVIDIA GeForce RTX 5090 (GB202, PCI 10de:2b85, VBIOS 98.02.2E.40.E1)

Describe the bug

After upgrading from 610.57.04 to 615.71.09, my RTX 5090 drives a Samsung Odyssey G95NC over HDMI 2.1 (7680x2160@240) in YCbCr 4:2:2, limited range instead of RGB 4:4:4 full range. nvidia-settings still shows the requested Color Space = RGB and Color Range = Full, but "Current Color Space" reports YCbCr422 and "Current Color Range" reports Limited. On 610.57.04 the same setup runs RGB full range.

I traced it to a change in the HDMI Forum VSDB parser in 615.71.09. The monitor advertises DSC_MaxSlices = 7. Up to 610.57.04 this was decoded as 16 slices at 400 MHz per slice; 615.71.09 decodes it as 12 slices at 600 MHz. The HDMI FRL code only treats 400 MHz as the higher DSC throughput class, so the sink's DSC decoder capacity is now computed as 12 x 340 = 4080 MPix/s, which is below the 4233.6 MPix/s this mode needs in RGB/4:4:4. The mode-set code then silently downgrades to 4:2:2. Restoring the 610 mapping for case 7 fixes it.

Root cause

The monitor's HDMI Forum VSDB (EDID offset 168, 14 bytes):

6d d8 5d c4 01 78 80 6b 02 00 00 c3 67 1d
                                    ^^ byte 12: DSC_MaxSlices = 7, DSC_Max_FRL_Rate = 6 (12G x4)

src/common/modeset/timing/nvt_edidext_861.c, DSC_MaxSlices decoding, 610.57.04:

case 7: pHdmiInfo->dsc_MaxSlices = 16; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 400; break;
case 6: pHdmiInfo->dsc_MaxSlices = 12; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 400; break;

615.71.09 (line 3913):

case 8: pHdmiInfo->dsc_MaxSlices = 12; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 900; break;
case 7: pHdmiInfo->dsc_MaxSlices = 12; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 600; break;
case 6: pHdmiInfo->dsc_MaxSlices = 12; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 400; break;

The consumer in src/common/modeset/hdmipacket/nvhdmipkt_C671.c (line 519) only knows two throughput classes, so the new 600 MHz value falls into the 340 MHz bucket:

NvU32 peakThroughput = (pSinkCaps->pHdmiForumInfo->dsc_MaxPclkPerSliceMHz == 400) ?
                            DSC_DECODER_PEAK_THROUGHPUT_MODE0_400 :
                            DSC_DECODER_PEAK_THROUGHPUT_MODE0_340;

and dsc.maxSliceCount is capped by dsc_MaxSlices (line 633). Net effect for this sink:

Driver	dsc_MaxSlices	peak throughput used	DSC budget	7680x2160@240 RGB needs
610.57.04	16	400	6400 MPix/s	4233.6 MPix/s, fits
615.71.09	12	340 (600 is not recognised)	4080 MPix/s	4233.6 MPix/s, fails

Because native 4:2:2 uses the mode1 throughput (half the pixel rate), the 4:2:2 DSC configuration still passes, and nvDowngradeColorFormatAndBpc() in nvkms-evo.c falls back to YCbCr422 + limited range without any log message.

Proposed fix

Restore the previous decoding of code 7 (16 slices, 400 MHz), which matches the HF-VSDB definition used by every release up to 610.57.04 and matches what this monitor actually supports:

--- a/src/common/modeset/timing/nvt_edidext_861.c
+++ b/src/common/modeset/timing/nvt_edidext_861.c
@@ -3911,7 +3911,7 @@
             switch(pHdmiForum->DSC_MaxSlices)
             {
             case 8: pHdmiInfo->dsc_MaxSlices = 12; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 900; break;
-            case 7: pHdmiInfo->dsc_MaxSlices = 12; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 600; break;
+            case 7: pHdmiInfo->dsc_MaxSlices = 16; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 400; break;
             case 6: pHdmiInfo->dsc_MaxSlices = 12; pHdmiInfo->dsc_MaxPclkPerSliceMHz = 400; break;

Two things suggest the case 7 change is unintended rather than a spec update:

• Independent implementations decode code 7 the old way. The Linux kernel's drivers/gpu/drm/drm_edid.c (HF-VSDB parsing) maps DSC_MaxSlices 7 to max_slices = 16, clk_per_slice = 400, the same as every NVIDIA release up to 610.57.04. Redefining an existing code would break every HDMI 2.1 sink that advertises it, which are exactly the highest-end 16-slice decoders.
• Nothing in the 615.71.09 tree consumes the new 600 / 900 MHz values. The only reader of dsc_MaxPclkPerSliceMHz is the == 400 test in nvhdmipkt_C671.c, so both new values silently degrade the sink to the 340 MHz class, below what it had before. The DSC library does define DSC_DECODER_PEAK_THROUGHPUT_MODE0_600 and _900, but the HDMI path never selects them.

If the 600 MHz / 900 MHz values are intentional for newer sinks (code 8 was added at the same time), nvhdmipkt_C671.c also needs to map them to a throughput class at least as high as DSC_DECODER_PEAK_THROUGHPUT_MODE0_400 instead of degrading to 340. Either way, code 7 should keep its HDMI 2.1 meaning.

Verification

I rebuilt nv-modeset-kernel.o from the 615.71.09 tag with only the line above changed, dropped it into the DKMS tree and rebuilt the modules. Compared with an unpatched build from the same toolchain, the only difference in the object is one instruction in parseEdidHdmiForumVSDB:

< or     $0x2580030,%r8d     (12 slices, 600 MHz)
> or     $0x1900040,%r8d     (16 slices, 400 MHz)

With the patched module, the same mode comes up as RGB, full range, 10 bpc at 7680x2160@240:

$ nvidia-settings -q "[dpy:DPY-7]/CurrentColorSpace" -q "[dpy:DPY-7]/CurrentColorRange" -t
0
0

Note: a CustomEDID with DSC_MaxSlices=6 also restores RGB, but it is not a usable workaround because this monitor's 7680x2160 timings come from its native DisplayID, which the driver no longer reads when CustomEDID is set.

To Reproduce

Preconditions: Blackwell GPU, HDMI 2.1 sink whose HDMI Forum VSDB advertises DSC_MaxSlices = 7 (here: Samsung Odyssey G95NC, 7680x2160@240, pixel clock 4233.6 MHz, FRL 12G x4, DSC 1.2), X11 session (GNOME 46) with nvidia-drm modeset=1.

1. Connect a G95NC over HDMI 2.1 to a Blackwell GPU and select 7680x2160@240.
2. Leave nvidia-settings at Color Space = RGB, Color Range = Full.
3. Query the actual output:

$ nvidia-settings -q "[dpy:DPY-7]/CurrentColorSpace" -q "[dpy:DPY-7]/CurrentColorRange" -t
1      # YCbCr422   (expected 0 = RGB)
1      # Limited    (expected 0 = Full)

On 610.57.04 both return 0.

Bug Incidence

Always. Every boot and every mode set on 615.71.09; never on 610.57.04 with the same hardware, cable and settings.

nvidia-bug-report.log.gz

Can be provided on request.

More Info

Samsung Odyssey G95NC EDID as read by the driver over HDMI (256 bytes)

000: 00 ff ff ff ff ff ff 00 4c 2d ce 73 53 51 51 30
010: 2a 21 01 03 80 8c 28 78 2a 4e d5 ae 4e 45 aa 27
020: 0e 50 54 25 cf 00 71 4f 81 c0 81 00 81 80 95 00
030: a9 c0 b3 00 d1 c0 1a 68 00 a0 f0 38 1f 40 30 20
040: 3a 00 78 90 51 00 00 1a 00 00 00 fd 00 1e f0 1e
050: ff ff 00 0a 20 20 20 20 20 20 00 00 00 fc 00 4f
060: 64 79 73 73 65 79 20 47 39 35 4e 43 00 00 00 ff
070: 00 48 4e 54 57 41 30 30 31 33 37 0a 20 20 01 b2
080: 02 03 55 f0 e2 78 03 45 61 10 04 5f 03 23 09 07
090: 07 83 01 00 00 e2 00 4f e3 05 c0 00 6b 03 0c 00
0a0: 30 00 b8 44 28 00 20 01 6d d8 5d c4 01 78 80 6b
0b0: 02 00 00 c3 67 1d 6d 1a 00 00 02 0f 60 f0 00 04
0c0: 8b 06 73 08 e6 06 05 01 8b 73 00 e3 0f 03 00 e5
0d0: 01 8b 84 90 79 56 5e 00 a0 a0 a0 29 50 30 20 35
0e0: 00 b9 88 21 00 00 1a 00 00 00 00 00 00 00 00 00
0f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 bb

Disclosure: the source analysis and the patch in this report were prepared with the help of an AI assistant (Claude). The diagnosis and the fix were verified on my hardware as described above.