Skip to content

docs: add an advisory PR-time doc-integrity check to docs-ui-tests - #2991

Open
marcel-rbro wants to merge 1 commit into
masterfrom
docs/docs-tests-pr-integrity-check
Open

marcel-rbro wants to merge 1 commit into
masterfrom
docs/docs-tests-pr-integrity-check

Conversation

@marcel-rbro

@marcel-rbro marcel-rbro commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Part of #2671. Addresses the cause behind #2988.

The harness only ran weekly, so #2919 moved a doc on 09-01 and nothing surfaced it until 09-07 — into an unassigned issue that then sat 10 days.

Splits the workflow into two jobs:

Job Runs on Does
doc-integrity (new) PRs touching sources/platform/** or docs-tests/** doc-side checks only — each assertion's source_file/source_quote still resolves
ui-drift (unchanged) weekly + manual full suite against staging, files a drift issue

Informative only: continue-on-error: true, so a failure never blocks the merge. Findings render as a table in the job summary. Promoting it to a gate later means dropping that flag and marking the check required.

No playwright install and no secrets — the doc-side project never opens a browser, so it also works on fork PRs. Verified against the pre-#2988 state with no browser present: all 16 stale references reported in 4.3s.

Note: this PR only touches .github/workflows/, so the job won't self-trigger here. It first runs on the next docs PR after merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FSj1mTUbMKgmu2JoGJLSKD

Splits the workflow into two jobs so doc-side drift is caught in review
instead of up to a week later.

doc-integrity runs on pull requests touching sources/platform/** or
docs-tests/**. It runs only the `integrity` project — static checks that
each assertion's source_file and source_quote still resolve. No browser
and no credentials, so it needs no Playwright install and works on fork
PRs, where secrets are unavailable.

The check is informative only: `continue-on-error` keeps a failure from
blocking the merge, and findings are written to the job summary as a
table of assertion IDs and their doc sources, so a reviewer sees them
without opening logs. Promoting it to a gate later means removing
`continue-on-error` and marking it required in branch protection.

ui-drift is unchanged except for an `if` guard keeping it off pull
requests. It still runs weekly and on demand, against staging, and still
files a drift issue on failure.

The motivation is #2951: #2919 moved account/settings.md on 09-01, the
baseline went stale, and nothing surfaced it until the scheduled run on
09-07 — into an unassigned issue that sat another 10 days. Run against
that state locally, this job reports all 16 stale back-references in
4.3 seconds with no browser and no credentials.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSj1mTUbMKgmu2JoGJLSKD
@marcel-rbro marcel-rbro added documentation Improvements or additions to documentation. t-docs Issues owned by technical writing team. labels Sep 17, 2026
@marcel-rbro marcel-rbro self-assigned this Sep 17, 2026
@apify-service-account apify-service-account linked an issue Sep 17, 2026 that may be closed by this pull request
6 tasks
@apify-service-account apify-service-account added the tested Temporary label used only programatically for some analytics. label Sep 17, 2026
@apify-service-account

apify-service-account commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

✅ Preview for this PR (commit b07ef8cc) is ready at https://pr-2991.preview.docs.apify.com (see action run).

@TC-MO

TC-MO commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

I spy with my little 👀 a red PR check?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation. t-docs Issues owned by technical writing team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs-as-tests: automated Console UI drift detection

3 participants