Render fieldcompare diff visualizations

.github/workflows/system-tests.yml

@@ -83,14 +83,36 @@ jobs:
         cd ../../
     - name: Archive system test logs
       if: ${{ always() }}
+      id: archive-logs
       uses: actions/upload-artifact@v7
       with:
         name: system_tests_run_${{ github.run_id }}_${{ github.run_attempt }}_logs
         path: |
           runs/*/system-tests-build.log
           runs/*/system-tests-run.log
           runs/*/system-tests-compare.log
+          runs/*/system-tests-compare-diff.log
           runs/*/*/system-tests_*.log
+        if-no-files-found: warn
+    - name: Archive fieldcompare diff visualizations
+      if: ${{ always() }}
+      id: archive-diffs
+      uses: actions/upload-artifact@v7
+      with:
+        name: system_tests_run_${{ github.run_id }}_${{ github.run_attempt }}_diffs
+        path: |
+          runs/*/diff-results/visualizations/**/*.png
+        if-no-files-found: warn
+    - name: Link diff visualizations in the job summary
+      if: ${{ always() && steps.archive-diffs.outputs.artifact-url != '' }}
+      run: |
+        {
+          echo ""
+          echo "## Diff visualizations"
+          echo ""
+          echo "When fieldcompare fails, PNG renders of the archived diff VTK fields are included in the [\`_diffs\` artifact](${{ steps.archive-diffs.outputs.artifact-url }})."
+          echo "Look under \`runs/*/diff-results/visualizations/\`."
+        } >> "$GITHUB_STEP_SUMMARY"
     - name: Archive run files
       if: ${{ failure() || inputs.upload_artifacts == 'TRUE' }}
       uses: actions/upload-artifact@v7

changelog-entries/441.md

@@ -1 +1 @@
-- Archive fieldcompare diff VTK files into a `diff-results/` folder in each systemtest run directory on failure so they are easy to find in CI artifacts when investigating comparison failures (fixes [#441](https://github.com/precice/tutorials/issues/441)). Nested paths under `precice-exports/` are preserved under `diff-results/`.
+- Archive fieldcompare diff VTK files into a `diff-results/` folder in each systemtest run directory on failure so they are easy to find in CI artifacts when investigating comparison failures (fixes [#441](https://github.com/precice/tutorials/issues/441), [#740](https://github.com/precice/tutorials/pull/740), [#883](https://github.com/precice/tutorials/pull/883)).

tests/README.md

@@ -90,6 +90,7 @@ Each of these directories includes the usual tutorial case files and logs, as we
 1. `system-tests-build.log`: The logs of building the respective components.
 2. `system-tests-run.log`: The logs of running the simulation (intermixed, from all participants).
 3. `system-tests-compare.log`: The logs for the comparison to the reference results.
+4. `system-tests-compare-diff.log`: Progress and errors from rendering fieldcompare diff visualizations. Only present when comparison fails and visualization ran.
 
 In addition, in the directories of the cases executed, you can find `system-tests-<case>.log` files.
 
@@ -99,7 +100,7 @@ When the tests fail at the results comparison step, this typically means that th
 
 - `precice-exports/`: The coupling meshes of the test run.
 - `reference-results/`: The coupling meshes of the reference run, as stored on Git LFS, expanded into `reference-results-unpacked`. For test cases using implicit coupling, the reference `.tar.gz` also contains the reference `precice-*-iterations.log` files.
-- `diff-results/`: Numerical difference of the results in the two directories (computed with `fieldcompare dir --diff precice-exports/ reference/`). These are only present on failed comparisons.
+- `diff-results/`: Numerical difference of the results in the two directories (computed with `fieldcompare dir --diff precice-exports/ reference/`). These are only present on failed comparisons and accompanied by visualization in `diff-results/visualizations/`.
 - `iterations-logs/`: The `precice-*-iterations.log` files of the test run. Only present in test cases using implicit coupling. The comparisons to references only take into account the file SHA-256 checksums.
 
 To reproduce the comparison locally, use the [same fieldcompare command](https://github.com/precice/tutorials/blob/develop/tests/docker-compose.field_compare.template.yaml):
@@ -120,13 +121,25 @@ The differences are only shown per file, and there is no global metric or other
 
 Alternatively, [visualize the `precice-exports/diff_*.vtu` in ParaView](https://precice.org/configuration-export.html#visualization-with-paraview).
 
+To regenerate the PNG visualizations locally from an archived `diff-results/` folder (for example after downloading a CI artifact):
+
+```bash
+python3 visualize_fieldcompare_diffs.py /path/to/diff-results
+```
+
+The script prints progress per file (`[3/40] Rendering ...`) and uses several worker processes. Sparse meshes keep sphere glyphs; denser clouds use point sprites so large cases finish in minutes rather than tens of minutes.
+
+The default image size is set by `WINDOW_SIZE` in `visualize_fieldcompare_diffs.py` (currently `1024 x 768`). Increase it for higher-resolution PNGs, e.g. `WINDOW_SIZE = (1920, 1080)`.
+
+For PDF output instead of PNG, replace the `plotter.show(screenshot=...)` call in `render_field()` with `plotter.render()` followed by `plotter.save_graphic(str(output_file.with_suffix(".pdf")))`.
+
 ### Re-running from CI artifacts
 
 When a system test fails in CI, download the **full** artifact:
 
 `system_tests_run_<run_id>_<run_attempt>_full`
 
-(a smaller `_logs` archive contains only log files). The archive contains a shared `runs/` directory:
+(a smaller `_logs` archive contains the stage log files; on comparison failures, difference visualizations are in a separate `_diffs` archive. The archives contain a shared `runs/` directory:
 
 ```text
 runs/
@@ -138,6 +151,7 @@ runs/
     ├── system-tests-build.log
     ├── system-tests-run.log
     ├── system-tests-compare.log
+    ├── system-tests-compare-diff.log   # on comparison failures, when visualization ran
     └── …
 ```
 
@@ -290,6 +304,7 @@ Metadata and workflow/script files:
     - Multi-stage build Dockerfiles that define how to build each component, in a layered approach
   - `docker-compose.template.yaml`: Describes how to prepare each test (Docker Compose service template)
   - `docker-compose.field_compare.template.yaml`: Describes how to compare results with fieldcompare (Docker Compose service template)
+  - `docker-compose.diff_visualizer.template.yaml`: Describes how to render fieldcompare diff VTK files to PNG images on failure
   - `components.yaml`: Declares the available components and their parameters/options
   - `reference-results-metadata.md.template`: Template for reporting the versions and machine used to generate each reference results archive
   - `reference_versions.yaml`: List of arguments to use for generating the reference results
@@ -310,7 +325,8 @@ Implementation scripts:
 
 - `tests/`
   - `systemtests.py`: Main entry point
-  - `requirements.txt`: Dependencies (jinja2, pyyaml)
+  - `requirements.txt`: Dependencies (jinja2, pyyaml, pyvista for optional local use of the visualizer script)
+  - `visualize_fieldcompare_diffs.py`: Renders archived fieldcompare diff VTK files to PNG images (normally run via the `diff_visualizer` Docker stage)
   - `metadata_parser/`: Reads the YAML files into Python objects (defines the schema)
   - `systemtests/`: Main implementation classes
     - `Systemtest.py`

tests/docker-compose.diff_visualizer.template.yaml

@@ -0,0 +1,21 @@
+services:
+  diff-visualizer:
+    build:
+      context: {{ dockerfile_context }}
+      dockerfile: Dockerfile
+      target: diff_visualizer
+      args:
+        {% for key, value in build_arguments.items() %}
+        - {{ key }}={{ value }}
+        {% endfor %}
+    volumes:
+      - ./{{ diff_results_folder }}:/diff-results
+      - ../tests/visualize_fieldcompare_diffs.py:/home/precice/visualize_fieldcompare_diffs.py:ro
+    environment:
+      VTK_DEFAULT_OPENGL_WINDOW: vtkOSOpenGLRenderWindow
+      PYVISTA_OFF_SCREEN: "true"
+      PYTHONUNBUFFERED: "1"
+    command:
+      - /home/precice/venv/bin/python
+      - /home/precice/visualize_fieldcompare_diffs.py
+      - /diff-results

tests/dockerfiles/ubuntu_2404/Dockerfile

@@ -60,7 +60,8 @@ RUN apt-get -qq update && \
         python3-venv \
         pkg-config \
         wget \
-        inotify-tools
+        inotify-tools \
+        libosmesa6
 # The following are dependencies of gmsh, needed by some tutorials
 RUN apt-get -qq update && \
     apt-get -qq install \
@@ -73,6 +74,18 @@ USER precice
 ### end of precice_dependencies stage ###
 
 
+FROM precice_dependencies AS diff_visualizer
+# Headless PNG rendering of fieldcompare diff VTK files (CPU/OSMesa, no display server).
+USER precice
+WORKDIR /home/precice
+ENV VTK_DEFAULT_OPENGL_WINDOW=vtkOSOpenGLRenderWindow
+ENV PYVISTA_OFF_SCREEN=true
+RUN python3 -m venv /home/precice/venv && \
+    . /home/precice/venv/bin/activate && \
+    pip3 install --no-cache-dir pyvista
+### end of diff_visualizer stage ###
+
+
 FROM precice_dependencies AS precice
 # Build & install precice into /home/precice/precice
 ARG PRECICE_PR

tests/requirements-reference.txt

@@ -2,7 +2,8 @@
 # Reference only: run scripts keep using loose requirements.txt constraints.
 # Generated by tools/releasing/update-requirements-reference.py — do not edit manually.
 # Source: tests/requirements.txt
-# Generated: 2026-07-13T16:48:40Z
+# Generated: 2026-07-19T05:18:03Z
 
 jinja2==3.1.6
+pyvista==0.48.4
 pyyaml==6.0.3

tests/requirements.txt

@@ -1,2 +1,3 @@
 jinja2
+pyvista
 pyyaml

tests/systemtests/Systemtest.py

@@ -30,6 +30,10 @@
 
 DIFF_RESULTS_DIR = "diff-results"
 ITERATIONS_LOGS_DIR = "iterations-logs"
+DIFF_VISUALIZER_LOG = "system-tests-compare-diff.log"
+DIFF_VISUALIZER_TIMEOUT = int(
+    os.environ.get("PRECICE_SYSTEMTESTS_DIFF_VISUALIZER_TIMEOUT", 900)
+)
 
 STAGE_LOG_FILES = {
     "build": "system-tests-build.log",
@@ -214,7 +218,7 @@ def _get_length_of_name(results: List[SystemtestResult]) -> int:
         with open(os.environ["GITHUB_STEP_SUMMARY"], "a") as f:
             print("\n\n", file=f)
             print(
-                "In case a test fails, download the archive from the bottom of this page and inspect the per-stage logs (`system-tests-build.log`, `system-tests-run.log`, `system-tests-compare.log`). The stage runtimes might already give useful hints.",
+                "In case a test fails, download the archive from the bottom of this page and inspect the per-stage logs (`system-tests-build.log`, `system-tests-run.log`, `system-tests-compare.log`, and `system-tests-compare-diff.log` when visualizations ran). The stage runtimes might already give useful hints.",
                 file=f)
             print(
                 "See the [documentation](https://precice.org/dev-docs-system-tests.html#understanding-what-went-wrong).",
@@ -805,6 +809,153 @@ def __archive_fieldcompare_diffs(self) -> None:
                 self,
             )
 
+    def __get_diff_visualizer_compose_file(self) -> str:
+        platform = self.params_to_use.get("PLATFORM")
+        render_dict = {
+            'dockerfile_context': (
+                Path("..") / "tests" / "dockerfiles" / Path(platform)
+            ),
+            'build_arguments': self.params_to_use,
+            'diff_results_folder': DIFF_RESULTS_DIR,
+        }
+        jinja_env = Environment(loader=FileSystemLoader(PRECICE_TESTS_DIR))
+        template = jinja_env.get_template(
+            "docker-compose.diff_visualizer.template.yaml")
+        return template.render(render_dict)
+
+    def __append_diff_visualizer_status(self, status: str, elapsed_s: float) -> None:
+        log_path = self.system_test_dir / DIFF_VISUALIZER_LOG
+        with log_path.open("a", encoding="utf-8") as log_file:
+            log_file.write(f"\nstatus: {status}\nelapsed_s: {elapsed_s:.1f}\n")
+
+    def __visualize_fieldcompare_diffs(self) -> None:
+        """Best-effort rendering of archived fieldcompare diff VTK files via Docker."""
+        diff_results_dir = self.system_test_dir / DIFF_RESULTS_DIR
+        if not diff_results_dir.is_dir():
+            return
+
+        compose_path = self.system_test_dir / "docker-compose.diff_visualizer.yaml"
+        log_path = self.system_test_dir / DIFF_VISUALIZER_LOG
+        log_path.write_text("=== compare-diff ===\n", encoding="utf-8")
+        log_lock = threading.Lock()
+        time_start = time.perf_counter()
+
+        try:
+            compose_path.write_text(
+                self.__get_diff_visualizer_compose_file(), encoding="utf-8")
+        except OSError as error:
+            elapsed_s = time.perf_counter() - time_start
+            self.__append_diff_visualizer_status(f"error: {error}", elapsed_s)
+            logging.warning(
+                "Could not render fieldcompare diff visualizations for %s: %s",
+                self,
+                error,
+            )
+            return
+
+        logging.info(
+            "Rendering fieldcompare diff visualizations for %s "
+            "(timeout %ss)",
+            self,
+            DIFF_VISUALIZER_TIMEOUT,
+        )
+        try:
+            process = subprocess.Popen(
+                [
+                    "docker",
+                    "compose",
+                    "--file",
+                    compose_path.name,
+                    "up",
+                    "--exit-code-from",
+                    "diff-visualizer",
+                    "--abort-on-container-exit",
+                ],
+                cwd=self.system_test_dir,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+                text=True,
+                bufsize=1,
+            )
+        except OSError as error:
+            elapsed_s = time.perf_counter() - time_start
+            self.__append_diff_visualizer_status(f"error: {error}", elapsed_s)
+            logging.warning(
+                "Could not render fieldcompare diff visualizations for %s: %s",
+                self,
+                error,
+            )
+            return
+
+        def read_stream(stream, prefix: str) -> None:
+            if stream is None:
+                return
+            for line in stream:
+                line = line.rstrip("\n\r")
+                with log_lock:
+                    with log_path.open("a", encoding="utf-8") as log_file:
+                        log_file.write(f"{prefix}{line}\n")
+            stream.close()
+
+        stdout_thread = threading.Thread(
+            target=read_stream, args=(process.stdout, ""), daemon=True)
+        stderr_thread = threading.Thread(
+            target=read_stream, args=(process.stderr, "[stderr] "), daemon=True)
+        stdout_thread.start()
+        stderr_thread.start()
+
+        timed_out = False
+        try:
+            exit_code = process.wait(timeout=DIFF_VISUALIZER_TIMEOUT)
+        except subprocess.TimeoutExpired:
+            timed_out = True
+            process.kill()
+            try:
+                process.wait(timeout=SHORT_TIMEOUT)
+            except subprocess.TimeoutExpired:
+                pass
+            exit_code = process.returncode if process.returncode is not None else 1
+
+        stdout_thread.join(timeout=SHORT_TIMEOUT)
+        stderr_thread.join(timeout=SHORT_TIMEOUT)
+        elapsed_s = time.perf_counter() - time_start
+
+        if timed_out:
+            self.__append_diff_visualizer_status(
+                f"timed out after {DIFF_VISUALIZER_TIMEOUT}s", elapsed_s
+            )
+            logging.warning(
+                "Could not render fieldcompare diff visualizations for %s: "
+                "timed out after %ss (visualizer ran %.1fs). "
+                "See %s",
+                self,
+                DIFF_VISUALIZER_TIMEOUT,
+                elapsed_s,
+                DIFF_VISUALIZER_LOG,
+            )
+            return
+
+        if exit_code != 0:
+            self.__append_diff_visualizer_status(
+                f"failed (exit {exit_code})", elapsed_s
+            )
+            logging.warning(
+                "Rendering fieldcompare diff visualizations failed for %s "
+                "after %.1fs (exit %s). See %s",
+                self,
+                elapsed_s,
+                exit_code,
+                DIFF_VISUALIZER_LOG,
+            )
+            return
+
+        self.__append_diff_visualizer_status("ok", elapsed_s)
+        logging.info(
+            "Diff visualizations for %s took %.1fs",
+            self,
+            elapsed_s,
+        )
+
     def __copy_rerun_system_test_script(self) -> None:
         """Copy tests/rerun-system-test.sh into the run directory for artifact replay."""
         rerun_src = PRECICE_TESTS_DIR / "rerun-system-test.sh"
@@ -1139,6 +1290,7 @@ def run(self, run_directory: Path):
             std_err.extend(fieldcompare_result.stderr_data)
             if fieldcompare_result.exit_code != 0:
                 self.__archive_fieldcompare_diffs()
+                self.__visualize_fieldcompare_diffs()
                 logging.critical(f"Fieldcompare returned non zero exit code, therefore {self} failed")
                 return SystemtestResult(
                     False,