Skip to content

Skip whole-block references above the key when setting defaults - #7338

Open
pietern wants to merge 3 commits into
mainfrom
pietern/f08-7225-set-default-ref-parent
Open

pietern wants to merge 3 commits into
mainfrom
pietern/f08-7225-set-default-ref-parent

Conversation

@pietern

@pietern pietern commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Changes

A ${...} reference to a whole block one level above a default key, such as new_cluster: ${resources.jobs.base.job_clusters[0].new_cluster} or a cluster workload_type, failed bundle validate with "expected a map to index". SetDefault now skips such references at any level, and errors it can't skip report the full config path.

Why

Follow-up to #7225, which handled references at the leaf only.

Tests

  • Unit tests for both reference cases and the error path.
  • Acceptance test bundle/validate/references/job_cluster_whole_reference.

This pull request and its description were written by Isaac.

@github-actions github-actions Bot added the DABs DABs related issues label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Commit: af29e37cf | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

2 tests: regression=2 unreleased=0 coverage=0 failing=0
test branch main (7adc80d) latest (v1.24.0)
TestAccept/bundle/validate/references/job_cluster_whole_reference/DMS= ✅ ❌ ❌
TestAccept/bundle/validate/references/job_cluster_whole_reference/DMS=true ✅ ❌ ❌
TestAccept/bundle/validate/references/job_cluster_whole_reference/DMS=: branch ✅ | main (7adc80d) ❌ | latest (v1.24.0) ❌

main (7adc80d):

acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate\nName: test-bundle\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default\n\nValidation OK!\n\n>>> [CLI] bundle plan -o json\n{\n  \"clients\": {\n    \"jobs\": false,\n    \"notebooks\": true\n  }\n}\n\n>>> [CLI] bundle plan -o json\n{\n  \"clients\": {\n    \"jobs\": false,\n    \"notebooks\": true\n  }\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate\nError: expected a map to index \"resources.jobs.viaref.job_clusters[0].new_cluster.workload_type\", found string\n\nName: test-bundle\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default\n\nFound 1 error\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -2,2 +2,4 @@
        	            	 >>> [CLI] bundle validate
        	            	+Error: expected a map to index "resources.jobs.viaref.job_clusters[0].new_cluster.workload_type", found string
        	            	+
        	            	 Name: test-bundle
        	            	@@ -8,19 +10,5 @@
        	            	 
        	            	-Validation OK!
        	            	+Found 1 error
        	            	 
        	            	->>> [CLI] bundle plan -o json
        	            	-{
        	            	-  "clients": {
        	            	-    "jobs": false,
        	            	-    "notebooks": true
        	            	-  }
        	            	-}
        	            	+Exit code: 1
        	            	 
        	            	->>> [CLI] bundle plan -o json
        	            	-{
        	            	-  "clients": {
        	            	-    "jobs": false,
        	            	-    "notebooks": true
        	            	-  }
        	            	-}
        	            	-
        	Test:       	TestAccept/bundle/validate/references/job_cluster_whole_reference/DMS=
        	Messages:   	bundle/validate/references/job_cluster_whole_reference/output.txt vs /tmp/TestAcceptbundlevalidatereferencesjob_cluster_whole_referenc1925119852/002/output.txt

latest (v1.24.0): same output as main (7adc80d)

TestAccept/bundle/validate/references/job_cluster_whole_reference/DMS=true: branch ✅ | main (7adc80d) ❌ | latest (v1.24.0) ❌

main (7adc80d):

acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate\nName: test-bundle\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default\n\nValidation OK!\n\n>>> [CLI] bundle plan -o json\n{\n  \"clients\": {\n    \"jobs\": false,\n    \"notebooks\": true\n  }\n}\n\n>>> [CLI] bundle plan -o json\n{\n  \"clients\": {\n    \"jobs\": false,\n    \"notebooks\": true\n  }\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate\nError: expected a map to index \"resources.jobs.viaref.job_clusters[0].new_cluster.workload_type\", found string\n\nName: test-bundle\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default\n\nFound 1 error\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -2,2 +2,4 @@
        	            	 >>> [CLI] bundle validate
        	            	+Error: expected a map to index "resources.jobs.viaref.job_clusters[0].new_cluster.workload_type", found string
        	            	+
        	            	 Name: test-bundle
        	            	@@ -8,19 +10,5 @@
        	            	 
        	            	-Validation OK!
        	            	+Found 1 error
        	            	 
        	            	->>> [CLI] bundle plan -o json
        	            	-{
        	            	-  "clients": {
        	            	-    "jobs": false,
        	            	-    "notebooks": true
        	            	-  }
        	            	-}
        	            	+Exit code: 1
        	            	 
        	            	->>> [CLI] bundle plan -o json
        	            	-{
        	            	-  "clients": {
        	            	-    "jobs": false,
        	            	-    "notebooks": true
        	            	-  }
        	            	-}
        	            	-
        	Test:       	TestAccept/bundle/validate/references/job_cluster_whole_reference/DMS=true
        	Messages:   	bundle/validate/references/job_cluster_whole_reference/output.txt vs /tmp/TestAcceptbundlevalidatereferencesjob_cluster_whole_referenc3395663508/002/output.txt
FAIL

latest (v1.24.0): same output as main (7adc80d)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: af29e37

Run: 42001908825

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 271 16 4:08
✅​ aws-windows-latest-4core-16gb 273 14 3:19
✅​ azure linux-2core-8gb 270 16 4:11
✅​ azure-windows-latest-4core-16gb 272 14 3:09
✅​ gcp linux-2core-8gb 271 16 3:57
✅​ gcp-windows-latest-4core-16gb 273 14 3:20
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:13 gcp-windows-latest-4core-16gb TestAccept
3:11 aws-windows-latest-4core-16gb TestAccept
3:03 azure-windows-latest-4core-16gb TestAccept

SetDefault only skipped a pure ${...} reference where the pattern ends.
A reference higher up, such as
new_cluster: ${resources.jobs.base.job_clusters[0].new_cluster}, still
failed with "expected a map to index ...workload_type, found string",
because fixed path components do not treat a non-map as a no-match.

Walk the pattern one component at a time and skip a pure reference at
any level; the referenced block gets the default itself.

Co-authored-by: Isaac <no-reply@databricks.com>
@pietern
pietern force-pushed the pietern/f08-7225-set-default-ref-parent branch from 45a3fff to 5fa9cd3 Compare November 4, 2026 10:41
An error from the key lookup at the end of the pattern is relative to the
matched value, but it was prefixed with the parent's path, so the last
pattern component was missing from the message. Prefix it with the path of
the matched value instead.

Also cover the cluster workload_type whole-block reference in the
acceptance test.

Co-authored-by: Isaac <no-reply@databricks.com>
@pietern
pietern requested a review from denik November 5, 2026 09:05
@pietern
pietern marked this pull request as ready for review November 5, 2026 09:05
@pietern
pietern requested review from a team as code owners November 5, 2026 09:05
Co-authored-by: Isaac <no-reply@databricks.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants