Skip to content

testserver: use 16-digit workspace object ids to fix flaky id replacements - #7253

Open
denik wants to merge 1 commit into
mainfrom
denik/acc-default-python-root-dir-repl
Open

denik wants to merge 1 commit into
mainfrom
denik/acc-default-python-root-dir-repl

Conversation

@denik

@denik denik commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Changes

The testserver now gives workspace objects 16-digit ids (4000000000000000 + counter) instead of 1000 + counter.

Why

Scripts register these ids as literal replacements (add_repl "$id" DEPLOYMENT_ID), and user replacements are applied before the harness's patterns. A 4-digit id also matched inside other numbers in the output and cut them up — timestamps (no longer matching [UNIX_TIME_NANOS]), lineage UUIDs, resource_ids, ports, cli_version hashes:

-  "whl": ".../my_default_python-0.0.1+[UNIX_TIME_NANOS][0]-py3-none-any.whl"
+  "whl": ".../my_default_python-0.0.1+1[PROD_ROOT_DIR_ID]23385490265124-py3-none-any.whl"

Fixing it at the source covers every test that registers a tree-node id, rather than anchoring each script's replacement. 16 digits matches real workspace object ids; a leading 4 never matches the 1[78]... timestamp patterns, and a specific 16-digit run practically never occurs inside a UUID chunk or a timestamp.

Dashboard errors from local runs, Oct 1 – Nov 2 (default-python/classic and bundle/dms/*). Each snippet is the first changed line with the id cut into another value:

Error Test Snippet Why the 16-digit id fixes it
1307 bundle/templates/default-python/classic/DMS=/READPLAN= (macos) + "cli_version": "[CLI_VERSION]+1[PROD_ROOT_DIR_ID]9783df6", inside a run shorter than 16 chars
1313 bundle/templates/default-python/classic/DMS=/READPLAN=1 (macos) + "created_time": 1[PROD_ROOT_DIR_ID]42371540, inside a run shorter than 16 chars
1314 bundle/templates/default-python/classic/DMS=/READPLAN= (linux) + "created_time": 179266[PROD_ROOT_DIR_ID]051, inside a run shorter than 16 chars
1315 bundle/templates/default-python/classic/DMS=/READPLAN=1 (linux) + "created_time": 1792130[PROD_ROOT_DIR_ID]45, inside a run shorter than 16 chars
1316 bundle/templates/default-python/classic/DMS=/READPLAN= (windows) + "created_time": 179242[PROD_ROOT_DIR_ID]583, inside a run shorter than 16 chars
1317 bundle/templates/default-python/classic/DMS=/READPLAN=1 (windows) + "created_time": [PROD_ROOT_DIR_ID]766033611, inside a run shorter than 16 chars
1353 bundle/templates/default-python/classic/DMS=/READPLAN= (linux) + "nuid": "5ch3m4-w1dg-3t98-7654-3210[PROD_ROOT_DIR_ID]5432", inside a run shorter than 16 chars
1354 bundle/templates/default-python/classic/DMS=/READPLAN= (linux) + "nuid": "5ch3m4-w1dg-3t98-7654-32[PROD_ROOT_DIR_ID]765432", inside a run shorter than 16 chars
1355 bundle/templates/default-python/classic/DMS=/READPLAN=1 (linux) + "nuid": "5ch3m4-w1dg-3t98-7654-3210987[PROD_ROOT_DIR_ID]2", inside a run shorter than 16 chars
1392 bundle/templates/default-python/classic/DMS=/READPLAN=1 (linux) Unexpected output "rowLimit": [PROD_ROOT_DIR_ID] whole value equals the 4-digit id
1079 bundle/templates/default-python/classic/DMS=/READPLAN= (linux) + Note that http://127.0.0.1:[PROD_ROOT_DIR_ID]5 is used for initialization. inside a run shorter than 16 chars
2182 bundle/templates/default-python/classic/DMS=/READPLAN= (macos) + uuid: 7[PROD_ROOT_DIR_ID]faf-1ebc-4fb9-8a70-33338ab9a01d inside a run shorter than 16 chars
2200 bundle/templates/default-python/classic/DMS=/READPLAN=1 (macos) + "lineage": "a68d3430-dc71-[PROD_ROOT_DIR_ID]-9991-be16930f9243", inside a run shorter than 16 chars
2522 bundle/templates/default-python/classic/DMS=/READPLAN=1 (macos) + "lineage": "5b11bdde-a8c2-4a02-93d4-c39232[PROD_ROOT_DIR_ID]77", inside a run shorter than 16 chars
2528 bundle/templates/default-python/classic/DMS=/READPLAN= (linux) + "cli_version": "[CLI_VERSION]+[PROD_ROOT_DIR_ID]44069a29", inside a run shorter than 16 chars
2592 bundle/templates/default-python/classic/DMS=/READPLAN=1 (macos) + "lineage": "bf681176-596e-43aa-9c1e-7e39a9d6[PROD_ROOT_DIR_ID]", inside a run shorter than 16 chars
3036 bundle/templates/default-python/classic/DMS=/READPLAN=1 (macos) + "last_modified": 17933761[PROD_ROOT_DIR_ID]6, inside a run shorter than 16 chars
3066 bundle/templates/default-python/classic/DMS=/READPLAN=1 (macos) + Note that http://127.0.0.1:[PROD_ROOT_DIR_ID]8 is used for initialization. inside a run shorter than 16 chars
3070 bundle/templates/default-python/classic/DMS=/READPLAN=1 (macos) + uuid: 33c2e4b1-1077-[PROD_ROOT_DIR_ID]-b7ea-7c2055eae54a inside a run shorter than 16 chars
93 bundle/dms/record-failure/DMS=true (linux) + Endpoint: POST http://127.0.0.1:[DEPLOYMENT_ID]1/api/2.2/jobs/create inside a run shorter than 16 chars
1340 bundle/dms/successful-recreate/DMS=true (linux) + "lineage": "8037e727-4e27-4cc4-8636-c1c8[DEPLOYMENT_ID]d21", inside a run shorter than 16 chars
1386 bundle/dms/depends-on/DMS=true (macos) + "resource_id": "879[DEPLOYMENT_ID]41474890000", inside a 19-digit testserver id; would need it to contain 4000000000000…
1388 bundle/dms/depends-on/DMS=true (linux) + "resource_id": "879204843836[DEPLOYMENT_ID]941", inside a 19-digit testserver id; would need it to contain 4000000000000…
2042 bundle/dms/no-drift/DMS=true (macos) + "cli_version": "[CLI_VERSION]+e3a74b[DEPLOYMENT_ID]02", inside a run shorter than 16 chars
2154 bundle/dms/stale-plan/DMS=true/READPLAN=1 (windows) + "resource_id": "8791993254222[DEPLOYMENT_ID]00", inside a 19-digit testserver id; would need it to contain 4000000000000…
2155 bundle/dms/stale-plan/DMS=true/READPLAN= (macos) + "resource_id": "87924034[DEPLOYMENT_ID]1413000", inside a 19-digit testserver id; would need it to contain 4000000000000…
2245 bundle/dms/stale-plan/DMS=true/READPLAN= (windows) + "__id__": "87930[DEPLOYMENT_ID]7322207800", inside a 19-digit testserver id; would need it to contain 4000000000000…
3018 bundle/dms/failed-delete/DMS=true (linux) + "resource_id": "8793[DEPLOYMENT_ID]0179872414", inside a 19-digit testserver id; would need it to contain 4000000000000…
3077 bundle/dms/depends-on/DMS=true (linux) + "resource_id": "8793612[DEPLOYMENT_ID]22543488", inside a 19-digit testserver id; would need it to contain 4000000000000…

Tests

Regenerated the whole local acceptance suite with -update: no golden output changes (every printed id is already replaced). All unit-test packages that use the testserver pass.

This pull request and its description were written by Isaac.

@github-actions github-actions Bot added the DABs DABs related issues label Oct 29, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 8075de6

Run: 41440891932

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 281 15 3:27
✅​ aws-windows-latest-4core-16gb 283 13 3:07
✅​ azure linux-2core-8gb 280 15 3:21
✅​ azure-windows-latest-4core-16gb 282 13 3:23
✅​ gcp linux-2core-8gb 281 15 3:39
✅​ gcp-windows-latest-4core-16gb 283 13 3:33
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:25 gcp-windows-latest-4core-16gb TestAccept
3:01 azure-windows-latest-4core-16gb TestAccept
2:40 aws-windows-latest-4core-16gb TestAccept

@denik
denik force-pushed the denik/acc-default-python-root-dir-repl branch from 9f38514 to fd4fe22 Compare November 2, 2026 11:17
Scripts register these ids as literal replacements, which run before the
harness patterns. A 4-digit id also matched inside other numbers in the
output (timestamps, UUIDs, ports, resource ids) and cut them up, making
default-python/classic and the bundle/dms tests flaky.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik force-pushed the denik/acc-default-python-root-dir-repl branch from fd4fe22 to 8075de6 Compare November 2, 2026 11:57
@denik denik changed the title acc: fix flaky timestamp replacement in default-python/classic testserver: use 16-digit workspace object ids to fix flaky id replacements Nov 2, 2026
@denik
denik marked this pull request as ready for review November 2, 2026 12:41
@denik
denik requested a review from a team as a code owner November 2, 2026 12:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants