Skip to content

Disable persistent workers in sphinx_docs#453

Merged
MaximilianSoerenPollak merged 1 commit intomainfrom
disable_persistent_workers
Mar 30, 2026
Merged

Disable persistent workers in sphinx_docs#453
MaximilianSoerenPollak merged 1 commit intomainfrom
disable_persistent_workers

Conversation

@4og
Copy link
Copy Markdown
Member

@4og 4og commented Mar 26, 2026

Persistent workers cause Bazel cache corruption when external dependency versions change (stale symlinks remain visible to Sphinx). There is a performance cost to disabling this feature.

Fixes #452

📌 Description

🚨 Impact Analysis

  • This change does not violate any tool requirements and is covered by existing tool requirements
  • This change does not violate any design decisions
  • Otherwise I have created a ticket for new tool qualification

✅ Checklist

  • Added/updated documentation for new or changed features
  • Added/updated tests to cover the changes
  • Followed project coding standards and guidelines

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run --lockfile_mode=error //src:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.3.0) and connecting to it...
INFO: Invocation ID: 135e867a-78cb-42b5-b600-69128dcb8876
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: src
Loading: 0 packages loaded
    currently loading: src
Analyzing: target //src:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //src:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //src:license-check (68 packages loaded, 9 targets configured)

Analyzing: target //src:license-check (70 packages loaded, 9 targets configured)

Analyzing: target //src:license-check (76 packages loaded, 9 targets configured)

Analyzing: target //src:license-check (127 packages loaded, 1095 targets configured)

Analyzing: target //src:license-check (137 packages loaded, 1726 targets configured)

Analyzing: target //src:license-check (137 packages loaded, 1738 targets configured)

Analyzing: target //src:license-check (137 packages loaded, 1738 targets configured)

Analyzing: target //src:license-check (140 packages loaded, 3626 targets configured)

Analyzing: target //src:license-check (143 packages loaded, 4581 targets configured)

Analyzing: target //src:license-check (144 packages loaded, 4589 targets configured)

Analyzing: target //src:license-check (144 packages loaded, 4589 targets configured)

Analyzing: target //src:license-check (144 packages loaded, 4589 targets configured)

INFO: Analyzed target //src:license-check (145 packages loaded, 4715 targets configured).
[1 / 1] no actions running
[14 / 16] [Prepa] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar
[15 / 16] Building src/license.check.license_check.jar (); 0s disk-cache, multiplex-worker
INFO: Found 1 target...
Target //src:license.check.license_check up-to-date:
  bazel-bin/src/license.check.license_check
  bazel-bin/src/license.check.license_check.jar
INFO: Elapsed time: 26.692s, Critical Path: 2.49s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/src/license.check.license_check src/formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

@github-actions
Copy link
Copy Markdown

The created documentation from the pull request is available at: docu-html

Persistent workers cause Bazel cache corruption when external
dependency versions change (stale symlinks remain visible to Sphinx).
There is a performance cost to disabling this feature.
@4og 4og force-pushed the disable_persistent_workers branch from adcb024 to 521a147 Compare March 27, 2026 08:31
@MaximilianSoerenPollak
Copy link
Copy Markdown
Contributor

This fixes the bug for you that you have described in #452 ?

I just very crudely tested the performance implications, but on my machines I can't see any significant difference between this change and main.
That could either be an issue by me, or maybe there isn't as much of performance issue as thought.
What type of performance drop did you observe?

@4og
Copy link
Copy Markdown
Member Author

4og commented Mar 30, 2026

@MaximilianSoerenPollak, yes, with disabled persistent workers, I don't see cache issues like in #452 anymore.

The persistent worker feature was implemented in bazel-contrib/rules_python#2938, and as I unterstand, it's meant to improve performance of incremental builds. That's why a I thought if we disable it, we'll see longer documentation generation times. Interestingly, this feature is marked as experimental, but enabled by default 🤷‍♂️

I propose to merge this PR, stable builds are more important than the potential performance impact.

@MaximilianSoerenPollak MaximilianSoerenPollak merged commit 1d195b0 into main Mar 30, 2026
16 of 18 checks passed
@MaximilianSoerenPollak MaximilianSoerenPollak deleted the disable_persistent_workers branch March 30, 2026 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

Bazel cache corruption when generating documentation

2 participants