feat(gallery): let one gallery entry offer several builds of the same model

[localai-bot]

What this does

Lets a single gallery entry offer several builds of the same model, so a user installs nanbeige4.1-3b-q4 without first hunting through the gallery for every quantization and runtime of it.

An entry may now declare variants:, which is nothing but references to other gallery entries:

- name: nanbeige4.1-3b-q4      # unchanged: a normal, complete, installable entry
  url: github:mudler/LocalAI/gallery/nanbeige4.1.yaml@master
  overrides: {parameters: {model: nanbeige4.1-3b-q4_k_m.gguf}}
  files: [...]
  variants:
    - model: nanbeige4.1-3b-q8

At install time LocalAI either installs a variant the caller named, or picks one automatically:

1. Drop variants whose backend cannot run on this host. SystemState.IsBackendCompatible already derives that from the backend name, so MLX disappears on Linux and CUDA disappears on a Mac. Authors never write hardware conditions.
2. Drop variants that do not fit available memory: VRAM on GPU hosts, cgroup-aware system RAM on CPU hosts (so a container sees its own limit, not the node's).
3. Pick the largest option that fits, since a bigger footprint is a higher quality build of the same model.
4. The entry's own build competes in that ranking like any other option, and is never filtered out, so selection always ends with something installable.

Whichever build wins, the installed model keeps the entry's name. Presentation metadata (description, icon, license, urls, tags) comes from the entry; the payload (url, config_file, files, overrides) comes from the selected build.

Sizes ride the existing pkg/vram estimator, whose cascade is remote GGUF header, HTTP HEAD content length, the entry's declared size:, then the HF repo listing. Nothing is downloaded to decide. Probes are cached and time-bounded, and a probe failure never fails an install.

Selecting a variant explicitly

Auto-selection is the default; every surface can override it.

• REST: variant on POST /models/apply and POST /api/models/install/:id
• CLI: local-ai models install <name> --variant <variant>
• MCP: the variant parameter on the install_model tool
• UI: the models table keeps a plain Install that auto-selects, plus a split-button menu for overriding, and the expanded detail row lists every variant with its backend, size and whether it fits
• Listing: GET /api/models emits a cheap has_variants flag; GET /api/models/variants/:id returns the full description on demand

An explicit selection is honored even when it does not fit the host, with a warning, since that is a deliberate operator override. Naming a variant an entry does not declare is an error rather than a silent fallback.

Narrowing the gallery to models that offer variants

GET /api/models accepts has_variants=true, and the models page has a toggle for it,
matching the existing fitsFilter control on the same page.

The polarity is deliberately the inverse of where this is heading. Eventually the gallery
should show parent entries and hide the individual builds they reference, so a model
appears once instead of once per quantization. Defaulting to that today would empty the
gallery, since one entry declares variants. So the default is unchanged and the toggle
narrows to declaring entries only, as a preview.

That means has_variants=true is a placeholder shape, not a step toward the final API.
The end state needs the opposite param (reveal the referenced builds rather than hide
them), and it needs a reverse index over the gallery to know which entries are referenced
by someone else's variants: list, which is a different query from the per-entry flag.
Worth deciding deliberately rather than discovering at flip time.

Why existing installations are unaffected

Every released LocalAI reads gallery/index.yaml live from master and silently ignores keys it does not understand. An older client therefore drops variants: and installs the entry exactly as it does today, because the entry is still a complete entry with its own url, overrides and files.

This is the property the design was reshaped around, and it is tested: a spec re-parses the real gallery/index.yaml through a legacy-shaped struct and asserts every variant-declaring entry still carries its own payload, guarded against passing vacuously when no such entry exists.

Entries without variants: are untouched, and the listing issues no size probe for them.

Tested behavior

Each of these has a spec that was verified to fail when the behavior is broken:

• a variant whose backend cannot run here is never selected
• the largest fitting option wins, regardless of the order variants are written in
• the entry's own build wins when it is the largest fitting option
• the entry installs even when nothing fits, including when the entry itself does not fit
• a variant of unmeasurable size never displaces a measurable one
• a caller-supplied unknown variant name fails loudly, including on an entry declaring no variants
• a pin recorded on disk that a later gallery edit removed degrades to auto-selection with a warning instead of failing every future upgrade
• a probe failure completes the install
• the listing probes nothing for entries without variants
• the resolved entry never aliases the gallery's own maps, including nested ones

One deliberate behavior change

An entry whose config_file declares urls: previously persisted each URL twice in ._gallery_<name>.yaml; it now persists once. The field is display-only and no consumer indexes by position.

Known gaps

• Distributed mode resolves against the frontend. InstallModel delegates to the local manager, so memory is read from the frontend node rather than the worker that will serve the model. A cluster with a small frontend and large workers will select conservatively. Not introduced here, but variant selection is the first feature where it changes what gets installed.
• A cached probe error persists until the gallery generation changes. pkg/vram caches errors alongside values. A cancelled in-flight probe (a user navigating away from the gallery page) can leave a model sized as unknown until the next generation bump.
• Probing is still serial and uncapped within a single entry. The companion endpoint caps the blast radius at one entry per request, but an entry with many variants probes them one at a time with a per-probe rather than per-request timeout, no singleflight, and the remote GGUF path omits SkipLargeMetadata() where the file:// path passes it. Worth a follow-up before broad adoption.

Why variant description is a separate endpoint

Describing a variant means probing its size over the network. Doing that inline in the
gallery listing looked fine with one declaring entry and was a landmine at scale: probes
are serial, each is an HTTP HEAD plus a ranged GET, and useGalleryEnrichment re-fetches
the whole gallery with items: 9999 for the Manage page. At 200 entries declaring 4
variants each that is roughly 1000 serial probes, several minutes and gigabytes of traffic
for one page load.

So the listing emits only a has_variants flag (a length check on already-loaded
metadata) and the description moved to GET /api/models/variants/:id, fetched lazily when
a user opens the picker or expands the row. This mirrors the existing estimate/:id
endpoint, whose comment says it exists for exactly this reason. A spec backed by a
request-counting server asserts the listing issues zero probes.

Not in this PR

• Renaming gallery entries to drop quantization suffixes, so the stable name is the model rather than a specific build.
• Broader gallery adoption. One entry declares variants today.

Verification

auto_variant on the listing became auto_selected on the new endpoint, since EntryVariants already serialized that name. The field never shipped in a release, and the docs are updated.

make lint clean (0 issues). The full pre-commit coverage gate was run against the final tree and passed, with coverage rising to 53.2% against a 48.5% baseline. Suites green across core/gallery, core/services/galleryop, core/cli, pkg/mcp/localaitools and subpackages, core/http/routes, core/http/endpoints/localai, pkg/system, pkg/vram.

The coverage baseline in the repo is left at 48.5% rather than ratcheted to 53.2%. The rise is real but larger than one feature plausibly accounts for, so it is worth confirming which suites the baseline was originally measured over before moving the ratchet.

Still pending, and not claimed as done: end-to-end install verified on NVIDIA and on Metal. Selection was exercised against those hardware shapes only through synthetic host values, not real devices.

Note on process: per-commit hooks were skipped during development at the maintainer's explicit direction, since the coverage gate takes roughly 25 minutes per commit. The full gate was run once against the final tree instead.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]