Skip to content

Commit c0219ee

Browse files
fix(backend/kernel): make package importable without the kernel wheel
The +PyArrow CI matrix installs pyarrow but not the databricks-sql-kernel wheel (the wheel isn't on PyPI yet, and the [kernel] extra is deferred — see commit 31ca581). The previous fix gated unit tests on `pytest.importorskip("pyarrow")` but test_kernel_auth_bridge.py was still pulled into a kernel-wheel ImportError because: src/databricks/sql/backend/kernel/__init__.py -> from databricks.sql.backend.kernel.client import KernelDatabricksClient -> import databricks_sql_kernel # ImportError on +PyArrow CI The eager re-export from `__init__.py` was a convenience that broke every consumer that only needed a submodule (type_mapping, result_set, auth_bridge) — they all triggered the kernel wheel import for no reason. Fix: - Drop the eager re-export from `kernel/__init__.py`. Comment documents why and points callers (= session.py::_create_backend, already this shape) at the direct `from .client import ...`. - Drop the no-longer-needed `pytest.importorskip("pyarrow")` / `importorskip("databricks_sql_kernel")` from test_kernel_auth_bridge.py — auth_bridge.py itself has neither dep, so the test now runs on every CI matrix variant. - test_kernel_result_set.py and test_kernel_type_mapping.py keep the pyarrow importorskip because they themselves use pyarrow. Verified locally across the three matrix shapes: - both pyarrow + kernel installed: 39 pass. - pyarrow only (no kernel wheel — the +PyArrow CI shape): 39 pass. - neither: 9 pass (auth_bridge only), 2 modules skip (the others use pyarrow). Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
1 parent 823c416 commit c0219ee

2 files changed

Lines changed: 19 additions & 11 deletions

File tree

src/databricks/sql/backend/kernel/__init__.py

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,20 @@
66
switch its default transport (SEA REST → SEA gRPC → …) without
77
renaming this module.
88
9+
This ``__init__`` deliberately does **not** re-export
10+
``KernelDatabricksClient`` from ``.client``. Importing ``.client``
11+
loads the ``databricks_sql_kernel`` PyO3 extension at module-import
12+
time; doing that eagerly here would make ``import
13+
databricks.sql.backend.kernel.type_mapping`` (used by tests / by
14+
``KernelResultSet`` consumers) require the kernel wheel even when
15+
the caller never plans to open a kernel-backed session. Callers
16+
that need the client import it directly:
17+
18+
from databricks.sql.backend.kernel.client import KernelDatabricksClient
19+
20+
``session.py::_create_backend`` already does this lazy import under
21+
the ``use_sea=True`` branch.
22+
923
See ``docs/designs/pysql-kernel-integration.md`` in
1024
``databricks-sql-kernel`` for the full integration design.
1125
"""
12-
13-
from databricks.sql.backend.kernel.client import KernelDatabricksClient
14-
15-
__all__ = ["KernelDatabricksClient"]

tests/unit/test_kernel_auth_bridge.py

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,11 @@
1515

1616
import pytest
1717

18-
# The kernel backend's result_set + type_mapping modules transitively
19-
# import pyarrow; the connector's default-deps test job doesn't
20-
# install pyarrow, so importing the auth_bridge in that environment
21-
# would fail at module-collection time. Gate the whole module on
22-
# pyarrow availability — matches the convention the connector uses
23-
# for pyarrow-dependent tests.
24-
pytest.importorskip("pyarrow")
18+
# auth_bridge.py itself has no pyarrow or kernel-wheel deps. The
19+
# `databricks.sql.backend.kernel` package's __init__.py deliberately
20+
# does *not* eagerly re-export from .client either (which would
21+
# require the kernel wheel). So this test can run on the
22+
# default-deps CI matrix without any extras. No importorskip needed.
2523

2624
from databricks.sql.auth.authenticators import (
2725
AccessTokenAuthProvider,

0 commit comments

Comments
 (0)