Catch deprecated API warnings from tutorial builds

[svekars]

When tutorials use deprecated PyTorch APIs, Python fires DeprecationWarning and FutureWarning at runtime — but these warnings are currently silently ignored during the Sphinx Gallery build. We do not notice them until a user tries to run a tutorial and hits a broken API.

This PR attempts two capture those during the CI build, parses them, and automatically creates/updates a GitHub Issue listing the deprecated API usage by tutorial file.

Changes

• tools/deprecation_checker/ (new) — parses build log for DeprecationWarning/FutureWarning, generates Markdown report, optionally creates/updates a GitHub Issue
• .jenkins/build.sh — add a function to capture make docs output to _build/build.log via tee, generates report per worker, merges logs and files issue in manager. Guarded with || true.
• .github/workflows/_build-tutorials-base.yml — added an entry to upload report as build artifact

How it works

• Runs during the CI build when Sphinx Gallery executes tutorials on GPU workers
• Single GitHub Issue with docs-agent-deprecations label, updated each build (no duplicates)
• Auto-closes when all warnings are resolved; reopens if new ones appear
• Tags cc: @svekars

Test Plan

• Needs CI run to verify end-to-end

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3810

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 4a1896e with merge base a793306 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[svekars]

@claude

[claude]

Claude finished @svekars's task in 2m 49s —— View job

---

PR Review: #3810 — Catch deprecated API warnings from tutorial builds

Summary

This PR adds a deprecation warning detection system that parses Sphinx Gallery build logs for DeprecationWarning/FutureWarning, generates a Markdown report, and can auto-create/update a GitHub issue. The approach is sound and well-guarded with || true / continue-on-error so it won't break builds. However, there are several issues that should be addressed before merging.

---

Code Correctness

Bug: prototype_source/ instead of unstable_source/ — build_warning_parser.py:36 lists "prototype_source/" in _SOURCE_DIRS, but the repo uses unstable_source/ (see conf.py:170, CLAUDE.md). The prototype_source/ directory doesn't exist. Warnings from unstable_source/ tutorials would be misclassified as third-party dependency warnings instead of tutorial warnings.

Fix this →

Build log may not escape the Docker container (workers) — In .jenkins/build.sh:74, make docs 2>&1 | tee _build/build.log runs inside the Docker container (the build step runs via docker exec). The workspace is mounted as a volume (-v "${GITHUB_WORKSPACE}:/var/lib/workspace"), so _build/build.log should be written at ${GITHUB_WORKSPACE}/_build/build.log on the host. The Upload build log step at line 113 of the workflow then looks for _build/build.log relative to GITHUB_WORKSPACE. This should work, but is worth verifying in the CI run since path resolution across Docker mounts can be tricky.

PIPESTATUS may not work in all shells — build.sh:75 uses test ${PIPESTATUS[0]} -eq 0. The script runs via sh -c ".jenkins/build.sh" (line 111 of the workflow). If sh is dash rather than bash, PIPESTATUS won't be available. However, the script has #!/bin/bash and set -ex, so as long as the container invokes the script directly (not sourced by sh), this should be fine. The sh -c wrapper just invokes the script which then runs under bash. Worth confirming.

---

Structure & Formatting

No tests — This is a parseable, deterministic tool with clear input/output. Even a small unit test with a synthetic log snippet would help prevent regressions and validate the regex patterns. Consider adding tools/deprecation_checker/test_parser.py with a few cases for the warning regex, ANSI stripping, path normalization, and deduplication logic.

Dual log-merge strategies — The build logs are merged in two different ways:

1. Worker uploads build log as a GitHub artifact, manager downloads and cats them together (workflow lines 162-172)
2. Worker uploads build log to S3 (build.sh:139), which the manager could also download

The S3 upload in build.sh:139 appears to be unused — no corresponding S3 download exists in the manager section of build.sh. This dead code path could confuse future maintainers.

Fix this →

Worker also generates report — build.sh:132 generates a per-worker report, but the per-worker report is never uploaded or used. The meaningful report is generated in the manager step from merged logs. The worker step just writes a local file that gets discarded. Consider removing it to avoid confusion.

Fix this →

---

Build Compatibility

PYTHONWARNINGS may be noisy — build.sh:72 sets PYTHONWARNINGS="all::DeprecationWarning,all::FutureWarning". This enables all deprecation and future warnings, including from every dependency (numpy, PIL, sphinx, etc.). This could produce extremely large build logs and slow down the build due to I/O from tee. Consider whether default::DeprecationWarning,default::FutureWarning (which shows each warning once per location) would be sufficient and less noisy.

Issue creation on PRs — The Generate API deprecation report step runs with if: always() in the manager job, and the manager runs when inputs.UPLOAD == 1. The build-tutorials.yml workflow sets UPLOAD: 1 for all pushes to main and all pull_request events. This means the tool will attempt to create/update issues on PR builds too. This could result in noisy issue creation/updates on every PR. Consider guarding the --create-issue flag to only run on main pushes:

run: |
  python3 -m tools.deprecation_checker.api_report \
    --build-log _build/build.log \
    -o _build/api_report.md \
    ${{ github.ref == 'refs/heads/main' && '--create-issue' || '' }}

Permissions broadened at workflow level — build-tutorials.yml and build-tutorials-nightly.yml now have top-level permissions: contents: write, issues: write. These permissions propagate to all jobs, including the 15 worker shards which don't need issue or write permissions. A more minimal approach would be to only set permissions in the _build-tutorials-base.yml on the manager job (which this PR already does at line 130-131). The top-level permissions in the calling workflows may be unnecessary if the workflow_call inherits them via secrets: inherit.

---

Security

GITHUB_TOKEN passed into Docker — The manager step at workflow line 199 passes GITHUB_TOKEN into the Docker container via -e GITHUB_TOKEN. This gives the build script running inside Docker access to a token with issues: write and contents: write permissions. Currently build.sh doesn't use this token in the manager section, and the --create-issue flag is only used in the workflow step outside Docker. If the token is not needed inside the container, remove -e GITHUB_TOKEN from the Docker run command to reduce the attack surface.

Fix this →

---

Recommendation

Request Changes

The concept is good and the code is clean and well-structured. The main issues to address are:

1. Bug: prototype_source/ should be unstable_source/ — this will misclassify warnings
2. Security: Remove GITHUB_TOKEN from Docker environment if unused inside container
3. Noise: Guard --create-issue to only run on main to avoid spurious issues on PRs
4. Cleanup: Remove unused S3 build log upload and unnecessary per-worker report generation

---

|

[meta-cla]

Hi @svekars!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!