Skip to content

ci: build-native-matrix build job has no timeout-minutes — an apt-get stall burned 6h of runner time #2589

Description

@carlos-alm

What happened

On PR #2586, build / Build x86_64-unknown-linux-musl (job log) ran for 6h 0m 51s before GitHub killed it with The operation was canceled. It never got past the first line of the "Install cross-compilation tools (x86_64-musl)" step:

06:54:11Z  Run sudo apt-get update
06:54:42Z  Ign:2 http://azure.archive.ubuntu.com/ubuntu noble InRelease
06:54:49Z  Get:5 https://archive.ubuntu.com/ubuntu noble-security InRelease [126 kB]
12:54:47Z  ##[error]The operation was canceled.

apt-get update hung against the Azure Ubuntu mirror (note the Ign: retries against azure.archive.ubuntu.com) and never returned. The job sat idle for six hours, failed the PR, and consumed a runner slot the whole time.

Root cause

The build job in .github/workflows/build-native-matrix.yml declares no job-level timeout-minutes, so it inherits GitHub's 360-minute default. Individual steps are mostly unbounded too — only "Install napi-rs CLI" has timeout-minutes: 5. This is inconsistent with ci.yml, where nearly every step carries an explicit 5–30 minute bound.

Because build-native-matrix.yml is a reusable workflow shared by build-native.yml (PR validation) and publish.yml (release builds), the same unbounded stall can also hang a release.

Suggested fix

  • Add a job-level timeout-minutes to the build job sized to the real build time (the passing targets in that same run took 1m44s–7m10s, so ~30 min is generous headroom).
  • Bound the apt step specifically, and consider making the mirror stall self-healing rather than fatal — e.g. timeout-minutes on the step plus a retry, in the spirit of the existing 3-attempt npm install retry in ci.yml.

Impact

Infrastructure only — no product bug. It converts a transient mirror outage into a 6-hour red PR instead of a fast, retryable failure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions