Skip to content

[backport] ci: fall back off a slow Azure apt mirror in smoke tests - #15204

Merged
cderv merged 1 commit into
v1.10from
backport/apt-mirror-fallback
Oct 27, 2026
Merged

cderv merged 1 commit into
v1.10from
backport/apt-mirror-fallback

Conversation

@cderv

@cderv cderv commented Oct 27, 2026

Copy link
Copy Markdown
Member

Important

Backport from #15203

Backport so CI on v1.10 backport PRs gets the same protection as main. When a node in the azure.archive.ubuntu.com mirror pool is degraded, apt downloads crawl and the Playwright system dependencies step hits its 15 minute timeout, failing the Linux smoke job. Smoke tests now sample the Azure mirror before setup-r and drop it for archive.ubuntu.com when it is slow, and the other apt steps retry with a per-attempt timeout, dropping Azure after a failed attempt. Healthy runs keep using the Azure mirror.

CI-only change, so no changelog entry.

Checklist

I have (if applicable):

  • referenced the GitHub issue this PR closes
  • updated the appropriate changelog in the PR
  • ensured the present test suite passes
  • added new tests
  • created a separate documentation PR in Quarto's website repo and linked it to this PR
AI-assisted PR
  • AI tool used: Claude Code
  • Codebase grounding: local clone
  • Human review: I have reviewed, tested, and verified the AI-generated content before submitting.

Note: autonomous AI agents submitting PRs without human oversight are not permitted — see the Code of Conduct.

A degraded node in the azure.archive.ubuntu.com pool serves package
indexes at full speed but archives at tens of kB/s. The runner images
set apt timeouts that only catch a full stall, so apt never moves to
archive.ubuntu.com. The Playwright dependency step then hits its 15
minute limit and setup-r takes about 20 minutes. The Azure line is
dropped from apt-mirrors.txt only when the mirror is measured slow, so
healthy runs keep using it.

- Sample one .deb from the Azure mirror before setup-r, whose apt
  calls cannot be wrapped, and drop the mirror when it stays under
  1 MB/s for 10 seconds
- Run the later apt steps through a helper that gives each attempt
  3 minutes, drops the mirror after a failure, and retries
- Run dpkg --configure -a before each retry, in case a timeout
  interrupted an install (not exercised by the probes)

The logic is inline in test-smokes.yml rather than a local composite
action, because release mode checks out older tags that lack the
action directory.

See #15186 (job that hit the 15 minute limit),
actions/runner-images#14823, and actions/runner-images#14596.

(cherry picked from commit 1d82b12)
@posit-snyk-bot

posit-snyk-bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@cderv
cderv merged commit 2ed1b35 into v1.10 Oct 27, 2026
44 checks passed
@cderv
cderv deleted the backport/apt-mirror-fallback branch October 27, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants