Skip to content

acc: Collapse the test table in the regression report comment - #7181

Merged
janniklasrose merged 1 commit into
mainfrom
janniklasrose/regr-collapse-table
Nov 3, 2026
Merged

janniklasrose merged 1 commit into
mainfrom
janniklasrose/regr-collapse-table

Conversation

@janniklasrose

Copy link
Copy Markdown
Member

Changes

The regression report comment now puts the per-test table in a <details> block, with the counts line (659 tests: regression=15 ...) as its summary.

The workflow trims an oversized report by splitting it at <details>, so it now always keeps the first block (the table) and drops only failure details.

Why

The table has a row per test variant, often hundreds (e.g. #7094), which buries the failure details below it.

Tests

  • Extended TestRenderRegressionReport to assert the table is wrapped.
  • Ran the workflow's trimming logic in node on a synthetic oversized report (700-row table, 20 failures): the table is kept, 8 failures fit, 12 are reported as omitted, and <details> tags stay balanced.

This pull request and its description were written by Isaac.

The table has a row per test variant, often hundreds, which buries the
failure details below it. It now sits in a <details> block summarized by
the counts line.

The workflow trims an oversized report by splitting it at <details>, so
it now always keeps the first block, the table.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik

denik commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

can you add before/after as a comment there?

@denik

denik commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

The table has a row per test variant, often hundreds (e.g. #7094 (comment)), which buries the failure details below it.

This example seems to have an issue with test selection in general, it should not have selected all auto-migrate tests. (Does not invalidate this PR, it could still be useful)

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 5d0fedc

Run: 40309614348

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 281 15 6:24
✅​ aws-windows-latest-4core-16gb 283 13 5:17
✅​ azure linux-2core-8gb 280 15 6:41
✅​ azure-windows-latest-4core-16gb 282 13 4:34
✅​ gcp linux-2core-8gb 281 15 7:11
✅​ gcp-windows-latest-4core-16gb 283 13 6:45
Top 7 slowest tests (at least 2 minutes):
duration env testname
4:42 aws-windows-latest-4core-16gb TestAccept
3:55 gcp linux-2core-8gb TestAccept
3:48 azure linux-2core-8gb TestAccept
3:34 aws linux-2core-8gb TestAccept
3:16 gcp-windows-latest-4core-16gb TestTelemetryEndpoint
3:10 azure-windows-latest-4core-16gb TestAccept
3:04 gcp-windows-latest-4core-16gb TestAccept

@janniklasrose

Copy link
Copy Markdown
Member Author

Before/after, using the first rows of the #7094 report (660 rows in the real one).

Before

Regression test report

Selected 14 changed tests (limit=20, 0 not selected)

660 tests: regression=16 unreleased=0 coverage=644 failing=0

test branch main (aed6702) latest (v1.23.0)
TestAccept/bundle/data_quality_monitor/DMS= ✅ ❌ ❌
TestAccept/bundle/data_quality_monitor/DMS=true ✅ ❌ ❌
TestAccept/bundle/deployment/bind/data_quality_monitor/DMS= ✅ ❌ ❌
TestAccept/bundle/deployment/bind/data_quality_monitor/DMS=true ✅ ❌ ❌
TestAccept/bundle/invariant/auto-migrate/DMS=/INPUT_CONFIG=alert.yml.tmpl ✅ ✅ ✅
… (655 more rows)
TestAccept/bundle/data_quality_monitor/DMS=: branch ✅ | main (aed6702) ❌ | latest (v1.23.0) ❌

main (aed6702):

(failure output)

latest (v1.23.0): same output as main (aed6702)


After

Regression test report

Selected 14 changed tests (limit=20, 0 not selected)

660 tests: regression=16 unreleased=0 coverage=644 failing=0
test branch main (aed6702) latest (v1.23.0)
TestAccept/bundle/data_quality_monitor/DMS= ✅ ❌ ❌
TestAccept/bundle/data_quality_monitor/DMS=true ✅ ❌ ❌
TestAccept/bundle/deployment/bind/data_quality_monitor/DMS= ✅ ❌ ❌
TestAccept/bundle/deployment/bind/data_quality_monitor/DMS=true ✅ ❌ ❌
TestAccept/bundle/invariant/auto-migrate/DMS=/INPUT_CONFIG=alert.yml.tmpl ✅ ✅ ✅
… (655 more rows)
TestAccept/bundle/data_quality_monitor/DMS=: branch ✅ | main (aed6702) ❌ | latest (v1.23.0) ❌

main (aed6702):

(failure output)

latest (v1.23.0): same output as main (aed6702)

@janniklasrose
janniklasrose marked this pull request as ready for review November 3, 2026 10:59
@janniklasrose
janniklasrose requested a review from a team as a code owner November 3, 2026 10:59
@janniklasrose
janniklasrose requested review from denik and renaudhartert-db and removed request for renaudhartert-db November 3, 2026 10:59
@janniklasrose

Copy link
Copy Markdown
Member Author

@denik

can you add before/after as a comment there?

done in #7181 (comment)

This example seems to have an issue with test selection in general, it should not have selected all auto-migrate tests. (Does not invalidate this PR, it could still be useful)

Investigating separately

@janniklasrose
janniklasrose added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 5be5a4c Nov 3, 2026
29 checks passed
@janniklasrose
janniklasrose deleted the janniklasrose/regr-collapse-table branch November 3, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants