Skip to content

Build/Test Tools: Retry and verify the Gutenberg download on the 7.0 branch - #13374

Open
lancewillett wants to merge 1 commit into
WordPress:7.0from
lancewillett:fix/gutenberg-download-retries-7-0
Open

Build/Test Tools: Retry and verify the Gutenberg download on the 7.0 branch#13374
lancewillett wants to merge 1 commit into
WordPress:7.0from
lancewillett:fix/gutenberg-download-retries-7-0

Conversation

@lancewillett

Copy link
Copy Markdown
Member

Trac ticket: https://core.trac.wordpress.org/ticket/66028

Summary

Merge the downloader half of r62859 to the 7.0 branch: tools/gutenberg/download.js and tools/gutenberg/utils.js only. Both files are byte-identical to trunk after this change.

Why

r62873 merged r62859 and r62862 to 7.0, but carried only the workflow half. It touched a single file, .github/workflows/phpunit-tests.yml (+30/-1). The downloader rewrite never reached the branch.

So on 7.0 the downloader still streams the 31 MiB archive from the GitHub Container Registry as fetch body, gunzip, then tar stdin in one pipeline, with no temporary file, no size or SHA-256 check, no timeout, and no retries. An early stream close raises Node's ERR_STREAM_PREMATURE_CLOSE and the build fails before a test runs:

Running "gutenberg:verify" task
   Ref: 28c0dedc4eaf001a24237a1fbba4b0887698b000
Gutenberg directory not found. Downloading...
Blob digest: sha256:cfc2d88a...
Download/extraction failed: Premature close
Warning: Task "gutenberg:verify" failed. Use --force to continue.

tools/gutenberg/utils.js has the same gap in its token and manifest requests.

Core's own PHPUnit workflow on 7.0 is insulated, because r62873 wired it to consume the shared prepare-gutenberg run artifact and it never calls the downloader. Anything else that checks out this branch and runs npm run build:dev still hits the raw downloader, once per job.

7.0 is the only affected branch. 6.9 and earlier have no tools/gutenberg directory and never download the archive; 7.1 and trunk already carry r62859.

Scope

Exactly two files. tools/gutenberg/copy.js also differs between the branches, but that drift is from unrelated work and is deliberately left alone. No workflow, package.json, or Gruntfile.js change is needed.

Compatibility

  • download.js on trunk imports only Node built-ins and ./utils.
  • The module.exports blocks in the old and new utils.js are identical.
  • The only consumers of tools/gutenberg/utils.js on this branch are Gruntfile.js, which spawns the file, and download.js, which is replaced alongside it. Neither uses a dropped or renamed export.
  • GUTENBERG_EXPECTED_SHA is additive, and closes a latent gap: reusable-phpunit-tests-v3.yml@trunk already passes it, and this branch's utils.js currently ignores it.
  • .jshintrc is identical between the branches, jshint does not cover tools/, and this branch has no eslint.config.js.

Testing

node tools/gutenberg/download.js on this branch, against its pinned Gutenberg SHA:

Ref: 28c0dedc4eaf001a24237a1fbba4b0887698b000
Blob digest: sha256:cfc2d88a8139b1d71f6479fa109c19893016bfd39ad07b40cfc2490cae6ba845

Download attempt 1/3...
   Response: 200 OK from pkg-containers.githubusercontent.com
   Content-Length: 31.02 MiB (32529291 bytes)
   Manifest size: 31.02 MiB (32529291 bytes)
Downloaded 31.02 MiB (32529291 bytes) and verified SHA-256

Extracting verified artifact...
Extraction complete

Run twice; the extracted .gutenberg-hash matches the branch's pinned SHA both times. Note the attempt counter, Content-Length, manifest size, and SHA-256 verification lines, none of which the current downloader emits.

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code, Codex
Model(s): Claude Opus, GPT-5
Used for: Diagnosing the failure from CI logs and branch history, checking export and workflow compatibility, and running the live download verification. The change itself is the two files taken unmodified from trunk. The final diff was reviewed by me.


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request.

…branch.

[62873] merged [62859] and [62862] to this branch, but carried only the workflow
half: it touched a single file, .github/workflows/phpunit-tests.yml. The
downloader rewrite from [62859] never landed here.

So tools/gutenberg/download.js still streams the archive from the GitHub
Container Registry as fetch body, gunzip, then tar stdin in one pipeline, with
no temporary file, no size or SHA-256 check, no timeout, and no retries. An
early stream close raises Node's ERR_STREAM_PREMATURE_CLOSE and the build fails
with "Download/extraction failed: Premature close" before a test runs.
tools/gutenberg/utils.js has the same gap in its token and manifest requests.

The PHPUnit workflow on this branch is insulated, because [62873] wired it to
consume the shared prepare-gutenberg run artifact and it never calls the
downloader. Anything else that checks out this branch and runs `npm run
build:dev` still hits the raw downloader, once per job.

Merge the downloader half of [62859]: stage the blob to a temporary file, verify
the manifest size and SHA-256, extract only after verification, and retry
interrupted transfers up to three times. Metadata requests get the same bounded
retries and a timeout. This also makes the branch honour GUTENBERG_EXPECTED_SHA,
which reusable-phpunit-tests-v3.yml already passes it.

tools/gutenberg/copy.js also differs between the branches, but that drift is
from unrelated work and is left alone.

Fixes #66028.
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props lancewillett.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

3 participants