Skip to content

[air] Clarify node 0 log streaming for multinode --watch - #6719

Merged
caroline-db merged 1 commit into
mainfrom
air/watch-node-log-streaming
Sep 17, 2026
Merged

caroline-db merged 1 commit into
mainfrom
air/watch-node-log-streaming

Conversation

@caroline-db

Copy link
Copy Markdown
Contributor

Changes

  • Update air run --watch to identify node 0 as the log source and show the total node count for multinode workloads.
  • Preserve the existing single-node message and JSON output.

Why

air run --watch streams logs only from node 0, but the existing message does not make that limitation clear for multinode workloads.

Tests

  • go test ./experimental/air/cmd -run TestRunWatch -count=1
  • ./task checks
  • Full Go lint and Ruff 0.15.17 checks
  • ./task test
  • Live two-node GPU_1xA10 submission verified the watch message; the run was canceled while pending capacity.

This PR was written by Codex.

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 2c9b190

Run: 35151004422

Env ✅​pass 🙈​skip Time
✅​ aws linux 276 15 4:00
✅​ aws windows 278 13 4:36
✅​ azure linux 275 15 3:56
✅​ azure windows 277 13 3:35
✅​ gcp linux 276 15 4:46
✅​ gcp windows 278 13 4:27
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:34 aws windows TestAccept
4:25 gcp windows TestAccept
3:33 azure windows TestAccept

@caroline-db
caroline-db added this pull request to the merge queue Sep 17, 2026
Merged via the queue into main with commit 76603c3 Sep 17, 2026
34 checks passed
@caroline-db
caroline-db deleted the air/watch-node-log-streaming branch September 17, 2026 01:47
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 76603c3

Run: 35171957767

Env 🔄​flaky ✅​pass 🙈​skip Time
🔄​ aws linux 12 1621 1076 244:16
🔄​ azure linux 4 1442 1125 162:38
🔄​ gcp linux 2 1430 1129 192:09
16 interesting tests: 16 flaky
Test Name aws linux azure linux gcp linux
🔄​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS= ✅​p 🔄​f 🔄​f
🔄​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true ✅​p 🔄​f 🔄​f
🔄​ TestAccept/bundle/resources/mcp_services/lifecycle 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/mcp_services/lifecycle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/model_provider_services/lifecycle 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/model_provider_services/lifecycle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/model_services/lifecycle 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/model_services/lifecycle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/postgres_branches/recreate 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/postgres_branches/recreate/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🔄​f
🔄​ TestAccept/bundle/resources/postgres_databases/live_errors/bad_role_ref 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/postgres_databases/live_errors/bad_role_ref/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🔄​f
🔄​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name ✅​p 🔄​f ✅​p
🔄​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true ✅​p 🔄​f ✅​p
🔄​ TestAccept/ssh/bulk-transfer 🔄​f ✅​p ✅​p
🔄​ TestAccept/ssh/bulk-transfer/DATABRICKS_BUNDLE_ENGINE=direct 🔄​f ✅​p ✅​p
Top 50 slowest tests (at least 2 minutes):
duration env testname
40:36 aws linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
26:22 aws linux TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
23:21 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
22:55 aws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
22:35 aws linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
16:09 aws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
15:46 aws linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
15:30 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
13:19 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
12:40 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
12:33 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
12:28 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:46 aws linux TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:53 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:45 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:41 aws linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:12 azure linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:09 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
7:54 aws linux TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:49 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:40 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:29 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:07 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:01 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:00 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
6:42 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:39 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
6:30 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
6:29 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
6:29 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
6:12 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:54 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:49 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:45 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:42 aws linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:41 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
5:34 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:23 azure linux TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:17 aws linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
5:13 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
5:12 gcp linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
5:01 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
5:00 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
4:52 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
4:52 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
4:42 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
4:41 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
4:34 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
4:30 aws linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
4:26 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants