Skip to content

Enable Unity Catalog (and Docs) in Databricks Remote SSH mode - #2016

Open
misha-db wants to merge 10 commits into
mainfrom
unity-catalog-remote
Open

Enable Unity Catalog (and Docs) in Databricks Remote SSH mode#2016
misha-db wants to merge 10 commits into
mainfrom
unity-catalog-remote

Conversation

@misha-db

Copy link
Copy Markdown
Contributor

Changes

When the extension runs inside a Databricks Remote SSH session ("remote mode"), it previously short-circuited activation and surfaced no data views. This PR surfaces the Unity Catalog browser (and the Documentation view) in remote mode, connecting Unity Catalog directly from the ambient environment credentials — no bundle/config project (host + target) required.

Tests

ConnectionManager.test.ts

@rugpanov

rugpanov commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

🤖 Integration tests ❌ 5 of 35 test jobs failed for 5f72aa0e (30 passed).
View run

@rugpanov rugpanov added the databricks-team Authored by a Databricks team member label Jul 17, 2026
@misha-db
misha-db temporarily deployed to test-trigger-is July 29, 2026 07:57 — with GitHub Actions Inactive
@misha-db
misha-db temporarily deployed to test-trigger-is July 29, 2026 07:57 — with GitHub Actions Inactive
@misha-db
misha-db requested a review from rclarey July 29, 2026 08:03
@rugpanov

rugpanov commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

🤖 Integration tests ❌ 1 of 35 test jobs failed for 4ac9f0fe (34 passed).
View run

@rugpanov rugpanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review via Codex + Claude review agent + a manual pass. Core mechanism (env-based PAT connect, inlined mutex logic to avoid the non-reentrant loginLogoutMutex deadlock, status-node rendering, and the registerUnityCatalog/registerDocsView refactor) is correct and well-tested — this-binding in the refactored refresh command is preserved, there's no double registration (remote branch returns early), and no init-render race (the provider subscribes to onDidChangeState). when-clause changes are additive/backward-compatible.

Every comment below was verified against the code/SDK before posting; one candidate finding (that the connectRemote .catch might log the PAT) was checked and dropped — the SDK only logs request headers to its own internal context logger, and the thrown ApiError carries just a message, so there's no leak.

CI: unit tests + packaging pass. The two red checks are not PR-caused: Trigger Tests failed with "exceeded max execution time while awaiting a runner for 24h" (infra), and Integration Tests failed only on the Windows run_dbconnect.*.e2e.ts shard (databricks-connect/serverless-setup — the known flake bucket; this PR touches none of those files). Worth a re-run.

"id": "unityCatalogView",
"name": "Unity Catalog",
"when": "databricks.context.activated && databricks.context.loggedIn"
"when": "databricks.context.activated && (databricks.context.loggedIn || databricks.context.remoteMode)"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must-address: the remote reconnect/refresh button is disabled in exactly the failure state it exists to recover from.

The new reconnect wiring exists so the refresh command re-establishes a failed connection in remote mode (extension.ts connectRemote). But the databricks.unityCatalog.refresh command keeps "enablement": "databricks.context.activated && databricks.context.loggedIn" (package.json:259). In remote mode loggedIn is set true only on updateState("CONNECTED"); while CONNECTING and after a failed connect (DISCONNECTED) it's false. So when the initial connectRemote() fails and the tree shows the Not connected: <error> node, the toolbar refresh button is greyed out. I confirmed there is no other trigger — no commandPalette override and no viewsWelcome entry for unityCatalogView — so the user's only recovery is a full window reload.

Suggest relaxing the command enablement at package.json:259 to mirror this view's when, e.g. databricks.context.activated && (databricks.context.loggedIn || databricks.context.remoteMode).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// The authProvider is only injected by tests; in production it
// is resolved from the SDK's default credential chain.
if (authProvider === undefined) {
const config = new Config({});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must-address (please confirm intent): new Config({}) resolves the full SDK credential chain, not only the injected environment.

ensureResolved() runs the default loaders [EnvironmentLoader, KnownConfigLoader]. EnvironmentLoader wins when DATABRICKS_HOST/DATABRICKS_TOKEN are set, but if they're absent KnownConfigLoader silently falls back to the DEFAULT profile in ~/.databrickscfg. So the config.token !== undefined check below doesn't prove the token came from the remote-injected env — a stray local DEFAULT PAT profile would satisfy it and connect to a possibly-wrong workspace instead of failing fast, which contradicts the method's doc-comment ("the remote environment always provides a token, so we fail fast if one isn't present").

If only the injected env should be honored, read DATABRICKS_HOST/DATABRICKS_TOKEN directly (or pass explicit values into Config) rather than relying on default resolution.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

);
context.subscriptions.push(
remoteBundleFileWatcher,
remoteConfigModel,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The four models passed to remoteConfigModel (BundleValidateModel, OverrideableConfigModel, BundlePreValidateModel, BundleRemoteStateModel) are constructed above but not pushed to context.subscriptions — only the watcher, remoteConfigModel, and remoteConnectionManager are. ConfigModel.dispose() disposes only its own listeners, not its child models (the normal flow pushes all four individually). Impact is bounded (one activation, no CLI work since target stays undefined), but it's an inconsistency. Better still: since connectFromEnvironment() never reads the ConfigModel (per the comment above), consider not building the bundle stack here at all and just satisfying the constructor.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// sync/cluster/config-project machinery they run, since remote mode
// only needs a workspace client for the Unity Catalog view.
this._connectionError = undefined;
this.updateState("CONNECTING");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] updateState("CONNECTING") here doesn't clear _workspaceClient/_databricksWorkspace first (only the catch clears them). On a refresh-triggered reconnect, a concurrent getChildren() — which checks only workspaceClient — could briefly use the old client while re-authenticating. Low impact in remote mode since credentials don't change, but clearing both before entering CONNECTING (or having the provider treat non-CONNECTED as unavailable) would make it airtight.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

});

it("returns undefined when not connected", async () => {
it("shows a status node at the root when not connected", async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Nice coverage of the DISCONNECTED empty/error branches. The state === "CONNECTING" branch of getDisconnectedRootChildren (the Connecting… node) is still untested — worth a case stubbing state = "CONNECTING" and asserting a single empty node with message Connecting….

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

reset(mockConfigModel);
});

it("connectFromEnvironment connects using the injected auth provider", async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] All three tests inject an AuthProvider, so the production credential-resolution path (new Config({}), missing host/token, PAT-only enforcement, normalizeHost) is never exercised — which is where the credential-fallback concern above lives. Consider a test around the no-authProvider branch. An activation-level test covering "initial connect fails → refresh reconnects" would also lock down the recovery flow.

@misha-db
misha-db temporarily deployed to test-trigger-is July 30, 2026 19:49 — with GitHub Actions Inactive
@misha-db
misha-db temporarily deployed to test-trigger-is July 30, 2026 19:49 — with GitHub Actions Inactive
@rugpanov

rugpanov commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

🤖 Integration tests ✅ all 35 test jobs passed for 36e88d90.
View run

@misha-db
misha-db temporarily deployed to test-trigger-is July 31, 2026 18:06 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

  • PR number: 2016
  • Commit SHA: 04216ebd58050a9e8da99a4ee4c689630343707d

Checks will be approved automatically on success.

@misha-db
misha-db temporarily deployed to test-trigger-is July 31, 2026 18:06 — with GitHub Actions Inactive
@misha-db
misha-db requested a review from rugpanov July 31, 2026 18:17
@rugpanov

Copy link
Copy Markdown
Contributor

🤖 Integration tests triggered for 04216ebd — ⏳ running.
View run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

databricks-team Authored by a Databricks team member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants