Repository navigation
Conversation
The metadata route caches a package's packument in the abbreviated install format, unless cooldown or npm_full_metadata needs the full one. Tarball downloads read the packument from the same cache row but asked for application/json. Once the row was due for revalidation, a tarball the proxy had not cached yet fetched the full packument, which did not match the abbreviated document's ETag, and stored it in its place. Until the next refresh every client was then served the full document, typically several times the size. Both now use packumentAccept. Downloads read only dist.tarball, dist.integrity and dist.shasum, which the abbreviated format carries; the publish times the cooldown needs come with the full format it selects.
1 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The metadata route requests packuments with
npmAcceptDefault(abbreviated install format, falling back toapplication/json), or plainapplication/jsonwhen cooldown ornpm_full_metadataneeds the full document. Tarball downloads read the packument from the same cache row (cacheKey) but always asked forapplication/json.So once the row was past
metadata_ttl, a download of a tarball the proxy hadn't cached yet revalidated with the abbreviated document's ETag while asking for the full one. On npmjs that's a 200 with the full packument, whichcacheMetadataBlobstored over the abbreviated one. Until the next refresh the metadata route then rewrote and served the full document to every client, typically several times the size for large packages, and the rewrite-cache entry changed with it. The next metadata refresh flipped it back, so the row alternated between formats during installs.Both paths now use
packumentAccept(), which holds the existing logic and comment. Downloads only readdist.tarball,dist.integrityanddist.shasum, which the abbreviated format carries, and the publish timesversionInCooldownneeds come with the full format that cooldown selects anyway.Test
TestNPMTarballKeepsCachedPackumentFormatserves the abbreviated and full formats with different ETags, fetches the metadata, and then downloads a tarball after the TTL. It checks that every packument request used the same Accept and that the cache row still holds the abbreviated document. On main it fails on both counts.go test -race ./...andgolangci-lintpass.