modelartifacts: cache key over-discriminates on allow_patterns, forcing a full re-fetch when patterns are narrowed

[localai-bot]

Summary

CacheKey hashes allow_patterns / ignore_patterns into the artifact identity, so any change to a gallery entry's pattern set produces a new cache key and a complete re-download — even when the new pattern set is a strict subset of what is already materialized on disk.

Narrowing patterns to fetch less data therefore costs a full re-fetch of everything, and orphans the previous tree.

Where

pkg/modelartifacts/path.go:62

identity := struct {
    Type           string   `json:"type"`
    Endpoint       string   `json:"endpoint"`
    Repo           string   `json:"repo"`
    Revision       string   `json:"revision"`
    AllowPatterns  []string `json:"allow_patterns,omitempty"`
    IgnorePatterns []string `json:"ignore_patterns,omitempty"`
}{...}
sum := sha256.Sum256(encoded)

Observed

On a distributed cluster, longcat-video-avatar-1.5 was installed under an older config with no allow_patterns, materializing the full repo:

70G  /models/.artifacts/huggingface/cc9cef3a6d15443f7c56a6c3e0e57b8fd58b5121154f871c7c69e2addb78424f

Reinstalling from the current gallery entry — which adds allow_patterns that exclude base_model_int8/**, i.e. asks for less — produced a new key and began re-downloading from HuggingFace:

resuming an abandoned artifact download partial=".../23c6c8baa875bcd06ea4446450e0aacc7b917e0d3aef3b187c447f8181fe9d51..."
Downloading url="https://huggingface.co/meituan-longcat/LongCat-Video-Avatar-1.5/resolve/.../base_model/config.json"
Downloading url=".../base_model/diffusion_pytorch_model-00001-of-00006.safetensors"

Every file requested by the new pattern set was already present under the old key. The old 70 GB tree is now orphaned — nothing references it.

Why the current behaviour is sound but too coarse

Hashing the patterns is correct in the sense that matters: it can never serve an incomplete tree. If the key ignored patterns, a config requesting base_model_int8/** would find a tree materialized without it, believe it complete, and fail at load with missing files. That safety property must be preserved.

The problem is that it treats pattern sets as equal-or-different, when the real relationship is a partial order:

• requested ⊆ materialized → the existing tree already satisfies the request; no fetch needed
• requested ⊄ materialized → fetch is needed, but only for the delta

Only the second case justifies downloading, and even then not from scratch.

The machinery already exists

Completed artifact directories carry a manifest next to the snapshot:

/models/.artifacts/huggingface/<key>/
  manifest.json     (18757 bytes for this repo)
  snapshot/

So the file inventory needed for a subset check is already persisted. A lookup could key on type + endpoint + repo + revision and consult the manifest to decide reuse vs delta-fetch, keeping the safety property without the redundant download.

Wrinkle worth designing around

A superset tree contains files the consumer should not see (base_model_int8/ in this case — the entry's comment notes only one of the two base variants is ever read, and fetching both roughly doubles the download). So reuse needs either a filtered view at stage time or a consumer that tolerates extra files. Staging already walks an explicit file list, so filtering there is likely the cheaper half.

Impact

• Refining a gallery entry's patterns — the main way to reduce download size — is punished with a full re-fetch, so entries are expensive to improve.
• Orphaned trees accumulate silently; nothing prunes the superseded key.
• On a distributed cluster this lands on the controller's shared model volume, where the re-fetch is tens of GB over the internet and blocks the model becoming usable.

Not proposing a specific implementation

Filing this as the observation plus the constraint (never serve an incomplete tree). The subset-check-with-manifest direction above is the obvious shape, but the filtered-view question deserves a design decision rather than being settled in the issue.

[localai-org-maint-bot]

@mudler The model-artifact cache key is causing avoidable full refetches when allow-patterns narrow; this needs an owner/decision on cache-key normalization.

[Dennisadira]

I looked at this with a view to implementing it, and stopped at the design decision the issue flags. Posting what I found so the choice is easier to make.

The safety property is load-bearing in a specific place. committedResult (pkg/modelartifacts/materializer.go:122) decides reuse purely from the cache key: it resolves the layout for spec.Resolved.CacheKey, reads the manifest sitting there, and re-derives CacheKey(manifest.Artifact) to confirm it matches. Ensure calls it at line 163 — before the resolver runs. So today the cache decision needs no network at all.

That matters for the obvious fix. If patterns come out of the key, a broader request finds a tree materialized under narrower patterns, sees every file its manifest lists present at the right size, and returns CacheHit: true — an incomplete model served as complete. Restoring safety means comparing the requested file set against the materialized one, and the requested set is only known after ResolveSnapshot. That turns the offline fast path into a network round trip on every load, which seems like a bigger regression than the one being fixed.

A narrower shape that avoids the question. Keep the key exactly as it is, so identity and the offline path are untouched, and instead widen file reuse. materializeLocked already skips work per file via reuseMaterializedFile (materializer.go:399, helper at :545), but only inside the writer's own staging tree. That hook could also consult committed sibling trees for the same type + endpoint + repo + revision, matching on path plus size/digest from their manifests, and hard-link or copy rather than fetch.

The reported symptom then goes away without any semantic change: narrowing patterns still produces a new key and a new tree, but it is populated from local bytes instead of a 70 GB re-fetch. The superset-contains-extra-files wrinkle disappears too, since the new tree only ever contains what its own pattern set selects — no filtered view needed, and no consumer has to tolerate strays. Cost is transient double disk usage until the old tree is pruned, which is already an open problem here.

The tradeoff against the manifest-subset direction: this doesn't make a subset request free (it still walks and links every file), it only makes it local. If avoiding that walk entirely is the goal, the subset check is the way and the filtered-view question has to be answered.

So the question, concretely: is the offline cache-hit path at materializer.go:163 something you want to keep? If yes, the sibling-reuse shape fits underneath it and I'm happy to implement that. If you'd rather have true subset reuse and are fine with resolving the snapshot before deciding, then it needs your call on filtered view vs. tolerant consumer, and I'd rather have that from you than guess.

Assisted-by: Claude:claude-opus-5