Conversation
There was a problem hiding this comment.
Pull request overview
This PR reorganizes the repository layout by moving system-test infrastructure from tools/tests/ to a top-level tests/ directory (per #703), updating documentation and CI workflows accordingly so system tests remain runnable in CI and locally.
Changes:
- Moved system test scripts/config/assets to
tests/and updated path references across Python, shell scripts, and docs. - Updated CI workflows to install dependencies from
tests/requirements.txtand run system tests fromtests/. - Adjusted pre-commit permalink checks to skip the new
tests/directory.
Reviewed changes
Copilot reviewed 9 out of 48 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tools/tests/paths.py | Removed legacy paths module from the old tools/tests/ location. |
| tools/check.sh | Updated tutorial-scan ignore pattern to exclude tests/. |
| tests/tests.yaml | Added system test suite definitions under tests/. |
| tests/systemtests/TestSuite.py | Added/relocated YAML parsing and suite construction logic for system tests. |
| tests/systemtests/SystemtestArguments.py | Added/relocated build-arg parsing utilities for system tests. |
| tests/systemtests/Systemtest.py | Updated system test runner to reference tests/ assets and copy tests/ into run workspaces. |
| tests/systemtests/sources.py | Added support for external tutorial sources (git/archive) used by system tests. |
| tests/systemtests/init.py | Package marker for systemtests. |
| tests/systemtests.py | New/relocated main entry point for running system tests from tests/. |
| tests/rerun-system-test.sh | New/relocated helper script to rerun a test from an extracted artifact. |
| tests/requirements.txt | New/relocated Python requirements for system tests. |
| tests/requirements-reference.txt | Updated reference requirements metadata to point at tests/requirements.txt. |
| tests/reference-results-metadata.md.template | Added/relocated template for reference-results metadata. |
| tests/reference_versions.yaml | Added/relocated default version pins used by reference generation/system tests. |
| tests/README.md | Updated documentation links and instructions to use tests/ paths. |
| tests/print_test_suites.py | Added/relocated CLI helper to print test suites. |
| tests/print_metadata.py | Added/relocated CLI helper to print tutorial metadata. |
| tests/print_case_combinations.py | Added/relocated CLI helper to print available case combinations. |
| tests/paths.py | Added new canonical paths module for the tests/ layout. |
| tests/metadata_parser/metdata.py | Added/relocated tutorial metadata parsing and schema objects. |
| tests/metadata_parser/init.py | Package marker for metadata_parser. |
| tests/generate_reference_results.py | Added/relocated script to generate reference result archives. |
| tests/dockerfiles/ubuntu_2404/Dockerfile | Added/relocated Docker build context for system test components. |
| tests/docker-compose.template.yaml | Added/relocated compose template for tutorial runs. |
| tests/docker-compose.field_compare.template.yaml | Added/relocated compose template for field comparison runs. |
| tests/components.yaml | Added/relocated component definitions and build arguments for system tests. |
| tests/component-templates/su2-adapter.yaml | Added/relocated SU2 adapter service template. |
| tests/component-templates/solids4foam.yaml | Added/relocated solids4foam service template. |
| tests/component-templates/rust-bindings.yaml | Added/relocated rust bindings service template. |
| tests/component-templates/python-bindings.yaml | Added/relocated python bindings service template. |
| tests/component-templates/openfoam-adapter.yaml | Added/relocated OpenFOAM adapter service template. |
| tests/component-templates/nutils-adapter.yaml | Added/relocated Nutils adapter service template. |
| tests/component-templates/micro-manager.yaml | Added/relocated micro-manager service template. |
| tests/component-templates/mercurydpm-adapter.yaml | Added/relocated MercuryDPM service template. |
| tests/component-templates/julia-bindings.yaml | Added/relocated Julia bindings service template. |
| tests/component-templates/fmi-runner.yaml | Added/relocated FMI runner service template. |
| tests/component-templates/fenicsx-adapter.yaml | Added/relocated FEniCSx adapter service template. |
| tests/component-templates/fenics-adapter.yaml | Added/relocated FEniCS adapter service template. |
| tests/component-templates/dune-adapter.yaml | Added/relocated DUNE adapter service template. |
| tests/component-templates/dumux-adapter.yaml | Added/relocated DuMux adapter service template. |
| tests/component-templates/dealii-adapter.yaml | Added/relocated deal.II adapter service template. |
| tests/component-templates/calculix-adapter.yaml | Added/relocated CalculiX adapter service template. |
| tests/component-templates/bare.yaml | Added/relocated bare (preCICE-only) service template. |
| tests/component-templates/aste.yaml | Added/relocated ASTE service template. |
| tests/build_docker_images.py | Added/relocated script to build component images for selected suites. |
| .github/workflows/system-tests.yml | Updated workflow steps to run from tests/ and install tests/requirements.txt. |
| .github/workflows/generate-reference-results.yml | Updated workflow steps to run from tests/ and install tests/requirements.txt. |
| .github/pull_request_template.md | Updated checklist path from tools/tests/tests.yaml to tests/tests.yaml. |
Suppressed comments (1)
tests/systemtests/Systemtest.py:338
self.params_to_use.get("PLATFORM")never raises, so the try/except doesn’t enforce the presence of PLATFORM; when missing it becomesNoneand the laterPath(None)fails with a less clear error. Also,plaform_requestedlooks like a typo and is easy to misread/propagate.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@PranjalManhgaye FYI, this might affect your PRs. Anything I missed, or any protests? |
|
I am merging this to continue with more cleanup work. Happy to help with resolving any merge conflicts. |
|
thanks @MakisH let me resolve conflicts in my pr due to this. |
…ons. Use point sprites for dense meshes, parallelize rendering, raise the visualizer timeout to 900s, and stream system-tests-compare-diff.log so CI no longer times out mid-run on flap Solid Force diffs.
Related to #703.
I only did:
git mv tools/tests/ tests/tools/tests/withtests/check.shto ignore thetests/for the permalink check.Looking at the diff, I also updated a few paths in the
Systemtest.py, where"tools","tests"appeared, and a few more similar cases.Note that
PRECICE_TOOLS_DIR(pointing to../..) was removed and replaced by the already existingPRECICE_TESTS_DIR(pointing to..). All these path variables have now one layer less (andPRECICE_TESTS_DIRpoints to..).Testing:
rerun-system-test.shfrom the archive, it works.I assume that other open PRs could be updated in a mostly fast-forward way. Only the
paths.pydoes not seem to be recognized as a rename.