Skip to content

Fix CPU Hub kernel backend selection - #2065

Open
SiluPanda wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
SiluPanda:fix/issue-2064
Open

Fix CPU Hub kernel backend selection#2065
SiluPanda wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
SiluPanda:fix/issue-2064

Conversation

@SiluPanda

Copy link
Copy Markdown

Description

The fused CPU 4-bit GEMM loader knows it is registering a CPU kernel, but it let kernels infer the backend from the installed PyTorch build. With a CUDA PyTorch wheel, that selects CUDA even though kernels-community/quantization-bitsandbytes publishes a CPU build, so the fused kernel is rejected and bitsandbytes falls back after logging a warning.

Pass backend="cpu" explicitly when loading the Hub kernel. The existing best-effort warning and fallback behavior remains unchanged.

Tests

  • Added regression coverage that asserts the loader requests version=1 and backend="cpu".
  • Added coverage for the existing warning-and-fallback path.
  • PYTHONPATH=. .venv/bin/pytest tests/test_cpu_ops.py -v --tb=short (2 passed)
  • .venv/bin/pre-commit run --all-files (all hooks passed)

Fixes #2064

AI assistance: This change was implemented with OpenAI Codex and independently reviewed and validated before submission.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CPU gemm_4bit_forward kernel is requested without backend="cpu", so it never loads on a CUDA torch build

1 participant