Skip to content

Investigate slow test root causes: OS vs solver#8868

Draft
tautschnig wants to merge 1 commit intodiffblue:developfrom
tautschnig:investigate-slow-tests
Draft

Investigate slow test root causes: OS vs solver#8868
tautschnig wants to merge 1 commit intodiffblue:developfrom
tautschnig:investigate-slow-tests

Conversation

@tautschnig
Copy link
Collaborator

Add a focused CI workflow that runs the 4 tests marked THOROUGH in 6e616e9 across different OS × solver combinations to determine whether the slowness is inherent to the operating system or caused by the choice of SAT/SMT solver.

Test matrix:

  • Tests 1-3 (StringSubstring, pid/C11, assigns-local-composite): {Linux, macOS-14} × {minisat2, cadical}
  • Test 4 (large_array_of_struct_nondet_index): {Linux, Windows} × incremental SMT2

The 4 tests are temporarily changed back to CORE so CTest picks them up. All other PR-triggered workflows are disabled to minimize CI load.

  • Each commit message has a non-empty body, explaining why the change was made.
  • n/a Methods or procedures I have added are documented, following the guidelines provided in CODING_STANDARD.md.
  • n/a The feature or user visible behaviour I have added or modified has been documented in the User Guide in doc/cprover-manual/
  • Regression or unit tests are included, or existing tests cover the modified code (in this case I have detailed which ones those are in the commit message).
  • My commit message includes data points confirming performance improvements (if claimed).
  • My PR is restricted to a single feature or bugfix.
  • n/a White-space or formatting changes outside the feature-related changed lines are in commits of their own.

@tautschnig tautschnig self-assigned this Mar 16, 2026
@tautschnig tautschnig force-pushed the investigate-slow-tests branch 3 times, most recently from e706251 to bf7da15 Compare March 16, 2026 19:20
Add a focused CI workflow that runs the 4 tests marked THOROUGH in
6e616e9 across different OS × solver combinations to determine whether
the slowness is inherent to the operating system or caused by the
choice of SAT/SMT solver.

Test matrix:
- Tests 1-3 (StringSubstring, pid/C11, assigns-local-composite):
  {Linux, macOS-14} × {minisat2, cadical}
- Test 4 (large_array_of_struct_nondet_index):
  {Linux, Windows} × incremental SMT2

The 4 tests are temporarily changed back to CORE so CTest picks them
up. All other PR-triggered workflows are disabled to minimize CI load.

Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
@tautschnig tautschnig force-pushed the investigate-slow-tests branch from bf7da15 to 2d1bc66 Compare March 16, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant