Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions acceptance/air/convert-to-dabs/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ resources:
environment_version: "6"

=== the current SDK recognizes B300 so bundle validation is clean

>>> [CLI] bundle validate
Name: b300-smoke-test
Target: dev
Expand Down
9 changes: 8 additions & 1 deletion acceptance/air/image-push/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ Pushed image.
Now you can set `environment.unity_catalog_image = my-catalog.my-schema.cuda:13.4.1-cudnn-devel-ubuntu26.04` in your YAML config and create a workload with `databricks air run -f config.yaml`.

=== inferred region and workspace ID requests

>>> print_requests.py --get //api
{
"method": "GET",
Expand All @@ -35,6 +34,7 @@ tag nvidia/cuda:13.4.1 [NUMID].container.us-west-2.localhost/my-catalog.my-schem
push [NUMID].container.us-west-2.localhost/my-catalog.my-schema.cuda:13.4.1-cudnn-devel-ubuntu26.04

=== missing image is pulled for AIR; destination defaults to source

>>> databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Pushing example/image:v1 as main.training.image:v1 using profile docker-test
Expand All @@ -50,6 +50,7 @@ Pushed image.
Now you can set `environment.unity_catalog_image = main.training.image:v1` in your YAML config and create a workload with `databricks air run -f config.yaml`.

=== forced pull and JSON output

>>> databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training --pull -o json
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Pushing example/image:v1 as main.training.image:v1 using profile docker-test
Expand All @@ -73,6 +74,7 @@ Pushed image.
}

=== daemon failure must not pull

>>> musterr databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Pushing example/image:v1 as main.training.image:v1 using profile docker-test
Expand All @@ -83,6 +85,7 @@ Cannot connect to the Docker daemon
Error: check local source image example/image:v1 (ensure Docker is running and accessible): exit status 1

=== inspect failure must not pull

>>> musterr databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Pushing example/image:v1 as main.training.image:v1 using profile docker-test
Expand All @@ -94,6 +97,7 @@ Docker inspect failed
Error: inspect source image example/image:v1: exit status 1

=== incompatible local image must not be pushed

>>> musterr databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Pushing example/image:v1 as main.training.image:v1 using profile docker-test
Expand All @@ -115,6 +119,7 @@ Docker pull progress
Error: source image example/image:v1 has platform "linux/arm64"; AIR requires linux/amd64: use --pull to fetch a compatible image, or rebuild the source for linux/amd64

=== pull failure must not be pushed

>>> musterr databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training --pull
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Pushing example/image:v1 as main.training.image:v1 using profile docker-test
Expand All @@ -126,6 +131,7 @@ Docker pull failed
Error: pull source image example/image:v1: exit status 1

=== invalid destination and removed flags

>>> musterr databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training --artifact invalid.name
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Error: invalid artifact name "invalid.name": use lowercase letters and digits separated by one or two underscores or one or more hyphens
Expand Down Expand Up @@ -177,6 +183,7 @@ Global Flags:


=== reject oversized catalog.schema.artifact before Docker operations

>>> musterr databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training --artifact aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
Error: invalid image repository name: catalog.schema.artifact must be 255 characters or less
Expand Down
6 changes: 3 additions & 3 deletions acceptance/air/image-push/script
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ trace databricks air images push \
--schema my-schema \
--artifact cuda:13.4.1-cudnn-devel-ubuntu26.04

title "inferred region and workspace ID requests\n"
title "inferred region and workspace ID requests"
trace print_requests.py --get //api

title "docker commands\n"
title "docker commands"
cat "$DOCKER_LOG"

title "missing image is pulled for AIR; destination defaults to source"
Expand Down Expand Up @@ -59,5 +59,5 @@ cp "$DOCKER_LOG" "$HOME/docker.before-length.log"
trace musterr databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training --artifact "$(cat oversized-artifact.txt)"
cmp "$DOCKER_LOG" "$HOME/docker.before-length.log"

title "all docker commands\n"
title "all docker commands"
cat "$DOCKER_LOG"
4 changes: 0 additions & 4 deletions acceptance/auth/bundle_default_profile/output.txt
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@

=== Bundle without workspace.host: default_profile is honored

>>> [CLI] bundle validate -o json
{
"host": null,
"profile": "default-target"
}

=== --profile overrides default_profile (negative case)

>>> errcode [CLI] bundle validate -p other -o json
Warn: [hostmetadata] failed to fetch host metadata for https://other.test, will skip for 1m0s
Error: Get "https://other.test/api/2.0/preview/scim/v2/Me": (redacted)
Expand All @@ -21,15 +19,13 @@ Exit code: 1
}

=== Bundle with workspace.host: default_profile is NOT applied

>>> errcode [CLI] bundle validate -o json
{
"host": "[DATABRICKS_URL]",
"profile": null
}

=== Bundle with workspace.profile: pinned profile wins over default_profile

>>> errcode [CLI] bundle validate -o json
Error: Get "https://other.test/api/2.0/preview/scim/v2/Me": (redacted)

Expand Down
8 changes: 4 additions & 4 deletions acceptance/auth/bundle_default_profile/script
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,24 @@ unset DATABRICKS_TOKEN
unset DATABRICKS_CONFIG_PROFILE

cp databricks.yml.no-host.tmpl databricks.yml
title "Bundle without workspace.host: default_profile is honored\n"
title "Bundle without workspace.host: default_profile is honored"
trace $CLI bundle validate -o json | jq '{host: .workspace.host, profile: .workspace.profile}'

cp databricks.yml.no-host.tmpl databricks.yml
title "--profile overrides default_profile (negative case)\n"
title "--profile overrides default_profile (negative case)"
trace errcode $CLI bundle validate -p other -o json | jq '{host: .workspace.host, profile: .workspace.profile}'

# Switch to a bundle that pins workspace.host. The default_profile guard in
# configureProfile must NOT apply default_profile here, because that would
# silently route the user to a profile pointing at a different host than the
# bundle declares.
DATABRICKS_HOST="$host_value" envsubst < databricks.yml.with-host.tmpl > databricks.yml
title "Bundle with workspace.host: default_profile is NOT applied\n"
title "Bundle with workspace.host: default_profile is NOT applied"
trace errcode $CLI bundle validate -o json | jq '{host: .workspace.host, profile: .workspace.profile}'

# Switch to a bundle that pins workspace.profile but no host. The pinned
# profile must win over default_profile — configureProfile's guard skips
# default_profile when workspace.profile is already set.
cp databricks.yml.with-profile.tmpl databricks.yml
title "Bundle with workspace.profile: pinned profile wins over default_profile\n"
title "Bundle with workspace.profile: pinned profile wins over default_profile"
trace errcode $CLI bundle validate -o json | jq '{host: .workspace.host, profile: .workspace.profile}'
6 changes: 3 additions & 3 deletions acceptance/auth/host-metadata-cache/script
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ export DATABRICKS_CACHE_DIR="$TEST_TMP_DIR/cache"
# never fetch or populate the cache.
envsubst < "./home/.databrickscfg.tmpl" > "./home/.databrickscfg"

title "First invocation populates the cache\n"
title "First invocation populates the cache"
$CLI auth profiles --output json

title "Second invocation should read from the cache\n"
title "Second invocation should read from the cache"
$CLI auth profiles --output json

title "Only one /.well-known/databricks-config request recorded\n"
title "Only one /.well-known/databricks-config request recorded"
print_requests.py //.well-known/databricks-config --get --sort
1 change: 1 addition & 0 deletions acceptance/bundle/apps/compute_size/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Files: 4 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged

=== Update compute size and redeploy

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete resources.apps.my_app
Expand Down
4 changes: 2 additions & 2 deletions acceptance/bundle/apps/delete_deleting/script
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ cleanup() {
}
trap cleanup EXIT

title "Deploy the app\n"
title "Deploy the app"
trace $CLI bundle deploy &> LOG.deploy
cat LOG.deploy | contains.py '!panic' '!internal error' > /dev/null

Expand All @@ -31,7 +31,7 @@ cat LOG.plan_delete.err | contains.py '!panic' '!internal error' > /dev/null
trace $CLI apps delete "app-$UNIQUE_NAME" &> LOG.apps_delete
cat LOG.apps_delete | contains.py '!panic' '!internal error' > /dev/null

title "Apply the stale delete plan against the DELETING app\n"
title "Apply the stale delete plan against the DELETING app"
trace $CLI bundle deploy --auto-approve --plan plan_delete.json &> LOG.deploy_delete
cat LOG.deploy_delete | contains.py '!panic' '!internal error' > /dev/null

Expand Down
3 changes: 3 additions & 0 deletions acceptance/bundle/apps/git_source/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
You can access the app at [APP_NAME]-123.cloud.databricksapps.com

=== Get app details and verify git_source configuration

>>> [CLI] bundle summary --output json

>>> [CLI] apps get [APP_NAME] --output json
Expand All @@ -51,6 +52,7 @@ You can access the app at [APP_NAME]-123.cloud.databricksapps.com
Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged

=== Run the app to verify it works

>>> cat tmp.app-run
✓ Getting the status of the app [APP_NAME]
✓ App is in RUNNING state
Expand All @@ -59,6 +61,7 @@ Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
You can access the app at [APP_NAME]-123.cloud.databricksapps.com

=== Update git_source branch and redeploy

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/app-git-source-[UNIQUE_NAME]/files...
Files: 3 uploaded, 0 deleted
Expand Down
1 change: 1 addition & 0 deletions acceptance/bundle/apps/job_permissions/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Add the following section to the jobs permissions:
false

=== Apply fix and redeploy

>>> [CLI] bundle deploy -qq

>>> has_manage_run
Expand Down
2 changes: 2 additions & 0 deletions acceptance/bundle/apps/source_code_path_update/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ Files: 4 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged

=== Run app to redeploy (reproduces the issue precondition of having an active deployment)

=== Redeploy from saved plan after description change

>>> [CLI] bundle deploy --plan plan.json
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/app-scp-update-[UNIQUE_NAME]/files...
Updated apps.my_app
Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/artifacts/whl_dynamic/script
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ trace jq .path < "$OUT_REQUESTS" | grep import | grep whl | sort

rm "$OUT_REQUESTS"

title "Updating the local wheel and deploying again\n"
title "Updating the local wheel and deploying again"
touch my_test_code/src/new_module.py
$CLI bundle plan -o json > tmp.plan.json 2>>LOG.plan
nostamp < tmp.plan.json > out.plan_update.json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ Resource: resources.pipelines.pipeline1


=== Configuration changes

>>> diff.py databricks.yml.backup databricks.yml
--- databricks.yml.backup
+++ databricks.yml
Expand Down
4 changes: 0 additions & 4 deletions acceptance/bundle/config-remote-sync/cli_defaults/script
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ job2_id="$(read_id.py job2)"
pipeline1_id="$(read_id.py pipeline1)"

title "Modify jobs"
echo
echo "Change max_concurrent_runs, name, change default tag"
edit_resource.py jobs $job1_id <<EOF
r["max_concurrent_runs"] = 5
Expand All @@ -36,19 +35,16 @@ EOF
# EOF

title "Modify pipeline"
echo
echo "Change edition and channel"
edit_resource.py pipelines $pipeline1_id <<EOF
r["edition"] = "CORE"
r["channel"] = "PREVIEW"
EOF

title "Detect and save all changes"
echo
cp databricks.yml databricks.yml.backup
$CLI bundle config-remote-sync --save

title "Configuration changes"
echo
trace diff.py databricks.yml.backup databricks.yml
rm databricks.yml.backup
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ Resource: resources.jobs.my_job


=== Configuration changes

>>> diff.py databricks.yml.backup databricks.yml
--- databricks.yml.backup
+++ databricks.yml
Expand Down
9 changes: 0 additions & 9 deletions acceptance/bundle/config-remote-sync/config_edits/script
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ $CLI bundle deploy
job_id="$(read_id.py my_job)"

title "Case 1: Added locally, updated remotely with different value"
echo
old=$(cat <<'EOF'
email_notifications:
on_success:
Expand All @@ -33,7 +32,6 @@ r["email_notifications"]["on_failure"] = ["remote-failure@example.com"]
EOF

title "Case 2: Added locally, removed remotely"
echo
old=$(cat <<'EOF'
max_concurrent_runs: 1
EOF
Expand All @@ -49,7 +47,6 @@ r.pop("timeout_seconds", None)
EOF

title "Case 3: Removed locally, removed remotely"
echo
old=$(cat <<'EOF'
tags:
env: dev
Expand All @@ -69,21 +66,18 @@ r["tags"].pop("version", None)
EOF

title "Case 4: Updated locally, updated remotely with different value"
echo
update_file.py databricks.yml 'max_concurrent_runs: 1' 'max_concurrent_runs: 3'
read -r -d '' case4 <<'EOF' || true
r["max_concurrent_runs"] = 5
EOF

title "Case 5: Updated locally, removed remotely"
echo
update_file.py databricks.yml 'env: dev' 'env: config-production'
read -r -d '' case5 <<'EOF' || true
r["tags"].pop("env", None)
EOF

title "Case 6: Added locally and remotely with same value (no drift expected)"
echo
old=$(cat <<'EOF'
my_job:
email_notifications:
Expand All @@ -101,7 +95,6 @@ r["description"] = "A test job"
EOF

title "Edit job remotely"
echo
edit_resource.py jobs $job_id <<EOF
$case1

Expand All @@ -117,11 +110,9 @@ $case6
EOF

title "Detect and save changes"
echo
cp databricks.yml databricks.yml.backup
$CLI bundle config-remote-sync --save

title "Configuration changes"
echo
trace diff.py databricks.yml.backup databricks.yml
rm databricks.yml.backup
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
}

=== Detect changes: etag drift must not be reported

>>> [CLI] bundle config-remote-sync
No changes detected.

Expand Down
3 changes: 0 additions & 3 deletions acceptance/bundle/config-remote-sync/dashboard_etag/script
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,14 @@ $CLI bundle deploy
dashboard_id=$(read_id.py my_dashboard)

title "Modify the dashboard out of band to bump its remote etag"
echo
# Keep only lifecycle_state: the full response carries a nondeterministic etag
# and update_time that would churn the snapshot.
$CLI lakeview update "$dashboard_id" --json '{"serialized_dashboard": "{}", "warehouse_id": "'$TEST_DEFAULT_WAREHOUSE_ID'"}' | jq '{lifecycle_state}'

title "Detect changes: etag drift must not be reported"
echo
trace $CLI bundle config-remote-sync

title "Save changes: config must not be modified"
echo
cp databricks.yml databricks.yml.backup
trace $CLI bundle config-remote-sync --save
trace diff.py databricks.yml.backup databricks.yml
Expand Down
Loading