Skip to content

quantize_4bit fails with "invalid configuration argument" (ops.cu:54) for tensors of exactly >= 2^31 elements #2067

Description

@TomMoeras

Versions: bitsandbytes 0.49.1, torch 2.11.0+cu128, CUDA 12.8, NVIDIA H200 (141 GB).

Repro (isolated, one process per size, fp32 flattened input, compress_statistics=True — mirroring axolotl 0.18's _simulate_nf4_roundtrip call):

n = 2**31 - 65536   -> OK
n = 2**31           -> Error invalid configuration argument at line 54 in file /src/csrc/ops.cu
n = 2**31 + 65536   -> same failure

A clean int32 boundary in quantize_4bit's kernel-launch arithmetic.

Real-world trigger: axolotl 0.18's merge CLI enables NF4 round-trip simulation for every 4-bit config; any model carrying a >= 2^31-element tensor (e.g. google/gemma-4-E4B-it's embedding) cannot merge. Smaller-vocab models (Mistral-7B, Qwen3.5-4B) pass, which hides the boundary.

Expected: chunked launch or int64 indexing above 2^31 elements, or a clear size-limit error.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions