Decompress: optimize sequence decompression (fast-path inlining, ABI scalarization, branch elision)

[JoeAzar]

Verified Google Fleet Production Impact (CPU)

Summary & Motivation

Sequence decoding and execution form the primary CPU hotspot during Zstandard block decompression. In high-throughput workloads, modern out-of-order execution pipelines frequently stall on avoidable microarchitectural bottlenecks within lib/decompress/zstd_decompress_block.c:

1. System V ABI stack spills: Passing the 24-byte seq_t struct by value forces compilers to classify it as MEMORY class under the x86-64 System V ABI (and exceeds register budgets on other ABIs), resulting in defensive stack spills and store-to-load forwarding (STLF) stalls.
2. Escaped literal pointer: Passing const BYTE** litPtr by reference breaks compiler escape analysis. Because litPtr's address is exposed across function boundaries, compilers defensively spill and reload the pointer to/from memory on every loop iteration.
3. Redundant external dictionary checks: Standard decompression paths (the vast majority of production workloads) repeatedly execute branch checks and address arithmetic for external dictionaries, even though hasExtDict is known at block start.
4. Deep ternary dependency chains in offset decoding: RepCode updates in small offset decoding evaluate serialized ternary cascades, generating complex dependency chains.
5. Function call boundaries on the fast path: >90% of sequences fit comfortably within wildcopy limits without crossing dictionary or buffer boundaries. Dispatching these sequences through an external function call forces bitstream states (seqState) to be spilled.

This PR introduces an orchestrated set of microarchitectural optimizations addressing all five bottlenecks. Across microbenchmarks and multi-corpus decompression suites, these changes deliver a 1.19x – 1.23x speedup (+23% decompression throughput) with zero changes to the on-disk format or external APIs.

---

Detailed Architectural Changes

1. Scalar Parameter Flattening & Calling Convention Register Compliance

• Problem: seq_t is a 24-byte structure (litLength, matchLength, offset). Under x86-64 System V ABI, structures larger than 16 bytes cannot be passed in registers and must be pushed onto the stack. Combined with 7 other parameters passed to ZSTD_execSequence*, callers exceeded the 6-register passing limit.
• Solution:
  • Flattened seq_t at function boundaries into three scalar size_t arguments: litLength, matchLength, and offset.
  • Consolidated loop-invariant dictionary boundaries (prefixStart, virtualStart, dictEnd) into a single const ZSTD_DCtx* dctx pointer.
  • Reduced total parameter pressure to fit strictly within hardware register calling conventions, completely eliminating stack spills and STLF stalls on sequence dispatch.

2. Escapeless Literal Pointer Promotion (Value Semantics for litPtr)

• Problem: ZSTD_execSequence* accepted const BYTE** litPtr so that the callee could advance the pointer. Taking &litPtr in the caller loop caused litPtr to escape, preventing the compiler from keeping it pinned to a CPU register across iterations.
• Solution:
  • Refactored ZSTD_execSequence* to take const BYTE* litPtr strictly by value.
  • Hoisted the predictable pointer advance (litPtr += sequence.litLength) directly into the caller loops.
  • With its address never taken, litPtr achieves permanent register residency throughout the entire decompression loop.

3. External Dictionary Elision via Template Specialization (hasExtDict)

• Problem: The execution fast-path evaluated external dictionary fallback conditions (offset > oLitEnd - prefixStart) on every sequence, even when no external dictionary was loaded.
• Solution:
  • Added a const int hasExtDict specialization parameter to ZSTD_decompressSequences_body, ZSTD_decompressSequences_bodySplitLitBuffer, and ZSTD_decompressSequencesLong_body.
  • Top-level wrappers evaluate hasExtDict = (dctx->prefixStart != dctx->virtualStart) once per block and dispatch the constant (0 or 1) to the body functions.
  • For standard decompression (hasExtDict == 0), the compiler dead-code-eliminates the external dictionary branch and its pointer arithmetic out of the inner loop, while retaining the essential corruption check (offset > oLitEnd - virtualStart).

4. Branch-Separated Small Offset & RepCode Decoding (ZSTD_decodeSmallOffset)

• Problem: ZSTD_decodeSequence and ZSTD_decodeSequenceLong decoded small offsets and RepCodes via nested ternary operators handling ofBits == 0 and ofBits == 1, serializing dependency chains.
• Solution:
  • Extracted the small offset decoding into an inlined helper function ZSTD_decodeSmallOffset.
  • Handled the highly predictable ofBits == 0 case with an early return branch, completely bypassing deeper logic.
  • For ofBits == 1, replaced nested ternary expressions with sequential conditional assignments (if (offset_eq1 >= 2) ...), allowing compilers to emit optimal, zero-spill register cmov instructions.

5. Inlined Sequence Execution Fast-Path in ZSTD_decompressSequences_body

• Problem: Calling ZSTD_execSequence on every iteration causes heavily mutated bitstream decoder states (such as seqState.DStream and FSE states) to be preserved across call boundaries.
• Solution:
  • Inlined the sequence execution fast path directly inside the core decoding loop in ZSTD_decompressSequences_body.
  • Sequences that satisfy safety guards (literals fit in buffer, match + literals fit within oend - WILDCOPY_OVERLENGTH, offset >= WILDCOPY_VECLEN, match remains within prefix) execute direct 16-byte copy + wildcopy for literals and wildcopy for match directly in the loop.
  • Edge cases (small offsets < WILDCOPY_VECLEN, external dictionaries, tail sequences near oend_w) cleanly fall back to ZSTD_execSequence.

---

Safety & Invariants Preserved

• Memory safety: Full window limit corruption checks (RETURN_ERROR_IF(UNLIKELY(offset > (size_t)(oLitEnd - virtualStart)), corruption_detected, "")) remain strictly enforced on all paths.
• 32-bit architectures: Retained address wrap-around guards (!MEM_32bits() || (size_t)(oend - op) >= sequence.litLength + sequence.matchLength + WILDCOPY_OVERLENGTH).
• Split literal buffers: ZSTD_split modes in ZSTD_decompressSequences_bodySplitLitBuffer and ZSTD_decompressSequencesLong_body are fully updated and tested with identical scalar register semantics.
• Fuzzing & Assertions: FUZZING_ASSERT_VALID_SEQUENCE and FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION test guards remain intact.

---

Benchmark Results

Benchmark 1: Core Decompression (BM_COMPRESSION_ZSTD_DECOMPRESS) (x86-64, --runs=15)

Metric	Baseline	Current	Delta	Speedup
CPU / op	0.6120 ns	0.5127 ns	-16.22%	1.19x
Time / op	0.6132 ns	0.5139 ns	-16.20%	1.19x
Instructions / op	7.443	6.373	-14.38%	1.17x
Cycles / op	2.149	1.804	-16.09%	1.19x

Benchmark 2: Google Fleetbench (BM_DecompressSink) (x86-64, --runs=10)

Metric	Baseline	Current	Delta	Speedup
Throughput (B/s)	1.118 GiB/s	1.374 GiB/s	+22.95%	1.23x
CPU / op	35.99 µs	29.27 µs	-18.67%	1.23x
Time / op	36.07 µs	29.33 µs	-18.67%	1.23x
Instructions / op	445.0k	389.4k	-12.50%	1.14x
Cycles / op	126.5k	102.9k	-18.70%	1.23x
Memory Allocs	80.98m	80.98m	+0.00%	1.00x
Peak Memory	201.0k	201.0k	+0.00%	1.00x

[meta-cla]

Hi @JoeAzar!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

[meta-cla]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[JoeAzar]

cc @danlark1