nvkms: select 2Heads1OR for DP modes wider than the DSC 5120 limit

[mickr]

On RTX 3060 Ti (GA104) with open 610.57.04, Pro Display XDR native 6016x3384@60 stays on one head: its 1286010 kHz pixel clock is below the 1340000 kHz per-head cap. DSC_PpsCalcBase then rejects the picture width and NVKMS reports DP bandwidth check failed. The 5120 limit is the non-multi_tile, non-dual PPS limit, not a limit on every GPU's DSC path.

The same clock-only selector is present in 615.71.09. This preserves its existing eligibility and high-clock behavior. For eligible DP modes wider than 5120, with DSC not disabled and no hardware YUV420 packing, it tries existing single-head validation before selecting two heads. That preserves wide modes that work uncompressed, including permitted color fallbacks. The probe uses the caller's explicit format/BPC flags without changing the caller's color. Final DP and IMP validation still apply.

Hardware verification, RTX 3060 Ti + Pro Display XDR, patched 610.57.04:

• 5120x2880@60 validates on one head.
• Native 6016x3384@60 validates with automatic DSC and forced DSC at 8 bpp. Physical 6K60 output and the desktop were verified, including reboot.
• DSC-disabled and maximum-one-head requests still reject native 6K60.
• Synthetic 6016x3384@30 validates uncompressed on one head, with automatic or disabled DSC. Validation only, never modeset at 30 Hz.

615.71.09 port, offline verification:

• make -j6 -C src/nvidia-modeset succeeds at 1a62581567f644a8fad1e2ca0d55a8a22cffa92c.
• 23 scenarios exercise the compiled selector and existing color negotiation with simulated DP transport. They cover explicit versus automatic format/BPC, wide uncompressed modes, width boundaries, original eligibility/clock behavior, and final rejection when the simulated link cannot support DSC.
• Negative controls fail as expected: the upstream selector rejects the 6K case; the initial port with hard-coded false color flags rejects the explicit-BPC case.

The 615 port has not been loaded on hardware. Other GPUs and combined MST/multi-display configurations are not covered. I did not A/B the proprietary driver on this machine. A related RTX 3070 report reported the same 6K rejection with both nvidia and nvidia-open 565.77.

[CLAassistant]

All committers have signed the CLA.

[mickr]

I missed that 615 only allows color/BPC fallback when those flags are false. The single-head probe now uses the caller specified flags so it cannot pick a forbidden lower-color path and then fail real validation.

[mickr]

Compiled 1a62581567f644a8fad1e2ca0d55a8a22cffa92c with make -j6 -C src/nvidia-modeset. No errors; one unused-variable warning at the unmodified nvEvoAttachConnector line 370. Modeset core SHA-256: 1453513a3bf51903d5d920253ffa8cf3507a271c6e71bb43b0992c8c2bd6de52.

Removed the unused scratch DSC output after checking that the PRE_IMP path accepts NULL. The compiled selector/color-negotiation harness passes 23 scenarios with simulated DP transport; both historical negative controls fail as expected. Offline verification only. Hardware evidence remains on 610.57.04.