Skip to content

dynvar: allow trailing underscores in reference path segments - #6711

Closed
denik wants to merge 3 commits into
mainfrom
denik/reference-nonalnum-key
Closed

denik wants to merge 3 commits into
mainfrom
denik/reference-nonalnum-key

Conversation

@denik

@denik denik commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Changes

Allow a trailing underscore (but not a trailing hyphen) in ${...} reference path segments, in both the Go regex (libs/dyn/dynvar/ref.go) and Python's _base_var_def, kept in sync via reference_vectors.json.

Why

A resource key ending in an underscore (e.g. my_job_) deploys fine, but the reference regex rejected it, so ${resources.jobs.my_job_.id} was silently dropped (a warning, then an unset field) rather than resolving. Trailing hyphens and leading digits stay rejected — the former to keep the dot-separated grammar unambiguous, the latter because Terraform rejects such keys anyway.

Tests

acceptance/bundle/resource_deps/trailing_underscore_key_reference now shows the reference resolving; added parity vectors for the allowed trailing-underscore and still-rejected trailing-hyphen cases.

This pull request and its description were written by Isaac.

A job resource key like `producer_` deploys fine, but the reference regex
(libs/dyn/dynvar/ref.go) requires every path segment to end in a letter or
digit, so `${resources.jobs.producer_.id}` is not recognized as a reference.
In an int field it is silently dropped with only a warning rather than
resolving or erroring. Documented as a Badness test for now.

Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: b212705

Run: 35215027971

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux 276 16 4:46
✅​ aws windows 278 14 4:03
✅​ azure linux 275 16 6:23
🔄​ azure windows 2 275 14 4:30
✅​ gcp linux 276 16 6:42
✅​ gcp windows 278 14 3:59
Test Name azure windows
🔄​ TestFsCpDirToDirFileNotOverwritten 🔄​f
🔄​ TestFsCpDirToDirFileNotOverwritten/uc-volumes_to_dbfs 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:01 aws windows TestAccept
3:44 gcp windows TestAccept
3:27 azure windows TestAccept

A resource key ending in an underscore (e.g. `my_job_`) is valid and
deployable, but the reference regex rejected it, so `${resources.jobs.my_job_.id}`
was silently dropped instead of resolving. Allow a trailing underscore (not a
trailing hyphen) in both the Go and Python regexes, kept in sync via
reference_vectors.json.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik denik changed the title acc: reproduce reference to a resource key ending in a separator dynvar: allow trailing underscores in reference path segments Sep 17, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
@denik denik closed this Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants