Skip to content

validate: check required and enum fields with generated typed walkers - #7143

Draft
denik wants to merge 9 commits into
mainfrom
denik/required-enum-typed-walkers
Draft

denik wants to merge 9 commits into
mainfrom
denik/required-enum-typed-walkers

Conversation

@denik

@denik denik commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Changes

validate:required and validate:enum now run generated typed walkers (bundle/config/validate/{required,enum}_walk.go) instead of matching every node of the dyn tree against a trie built from the generated pattern tables. The walkers only go down the pattern paths of the typed config.

The typed config can't tell unset from null, zero or a pure ${...} reference, and it has no locations. For those, each walker node also follows the dyn.Value at its path. This bridge lives in one file, walk_node.go, and gets replaced once config no longer uses dyn.

This is a slice of #7098 (#7098). libs/dyn stays in place.

Behavior

No change: same diagnostics, wording, paths, locations and ordering. Enum diagnostics are first put in dyn-walk order so that ties in the existing sort come out as before.

Tests

  • New unit tests in walk_test.go.
  • All TestAccept/bundle acceptance tests pass with unchanged goldens. During that run a temporary check (not committed) compared every result with the old trie implementation and found no mismatch.
  • The same check passed on 16k randomly mutated acceptance configs (null/""/0/false/{}/[]/refs).

Benchmarks

1000 jobs (bundletest.Bundle), median of 6 runs, macOS M-series, load ~5:

main this PR
Required().Apply 41.0 ms 20.7 ms
Enum().Apply 23.0 ms 4.0 ms
BenchmarkValidateRequired1000 (via bundle.Apply) 133 ms 113 ms
BenchmarkValidateEnum1000 (via bundle.Apply) 114 ms 95 ms
bundle validate end to end, acceptance/bundle/benchmarks --jobs 1000 11.12 s 11.04 / 11.18 s

The required check is now dominated by the unchanged sortDiagnostics (it calls fmt.Sprintf in the comparator). End-to-end bundle validate shows no measurable difference: these two mutators are under 1% of it.

This pull request and its description were written by Isaac.

@github-actions github-actions Bot added the DABs DABs related issues label Oct 21, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 229ccd4

Run: 41434155626

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 281 15 4:20
✅​ aws-windows-latest-4core-16gb 283 13 3:54
🔄​ azure linux-2core-8gb 2 278 15 4:19
✅​ azure-windows-latest-4core-16gb 282 13 3:41
✅​ gcp linux-2core-8gb 281 15 4:40
✅​ gcp-windows-latest-4core-16gb 283 13 3:48
Test Name azure linux-2core-8gb
🔄​ TestFsCpDirToDirWithOverwriteFlag 🔄​f
🔄​ TestFsCpDirToDirWithOverwriteFlag/local_to_uc-volumes 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:47 aws-windows-latest-4core-16gb TestAccept
3:41 gcp-windows-latest-4core-16gb TestAccept
3:35 azure-windows-latest-4core-16gb TestAccept

@denik denik changed the title Generate typed walkers for the required and enum field validators validate: check required and enum fields with generated typed walkers Oct 21, 2026
@denik
denik force-pushed the denik/required-enum-typed-walkers branch 2 times, most recently from 8df35af to 50e9336 Compare October 21, 2026 15:50
@denik
denik changed the base branch from main to denik/typed-mutators-simple October 21, 2026 15:50
@denik
denik added this pull request to stack #7150 October 21, 2026 15:50
@denik
denik force-pushed the denik/required-enum-typed-walkers branch 2 times, most recently from ab6c0ac to 2f2e8ac Compare October 21, 2026 16:12
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from d7f2171 to 83666bc Compare October 22, 2026 09:41
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from 83666bc to e8a5ecc Compare October 22, 2026 09:54
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from e8a5ecc to 23bdaae Compare October 27, 2026 11:12
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from 23bdaae to 9a552ef Compare October 27, 2026 11:29
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from 9a552ef to b3c20fe Compare October 27, 2026 12:12
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from b3c20fe to e5b6084 Compare October 27, 2026 13:00
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from e5b6084 to e35d9f0 Compare October 27, 2026 17:17
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from e35d9f0 to 38a5b50 Compare October 27, 2026 17:51
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from 38a5b50 to 653dde2 Compare October 27, 2026 20:35
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from 653dde2 to dec69ae Compare October 28, 2026 12:41
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from dec69ae to 338e9fe Compare October 28, 2026 13:14
@denik
denik force-pushed the denik/required-enum-typed-walkers branch 2 times, most recently from fd697a1 to e669436 Compare October 28, 2026 15:00
Base automatically changed from denik/typed-mutators-simple to main October 28, 2026 15:00
@denik
denik force-pushed the denik/required-enum-typed-walkers branch 4 times, most recently from 510d877 to 0104737 Compare October 29, 2026 11:45
denik and others added 9 commits November 2, 2026 12:03
Replace the generated pattern tables, matched against every node of the
dynamic configuration with a trie, by generated walkers that descend
only along the patterns' paths of the typed configuration. Null, zero,
pure-reference values and locations still come from the dyn.Value at
each node (walk_node.go), so the diagnostics are unchanged.

Slice of #7098.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…res resource

Co-authored-by: Isaac <no-reply@databricks.com>
…ttern trie

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Port of #7226 to the generated walker: a value that is a pure reference has
no fields of its own, the referenced value is checked where it is defined.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik force-pushed the denik/required-enum-typed-walkers branch from 0104737 to 229ccd4 Compare November 2, 2026 11:06

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants