Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/content/features/model-gallery.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,21 @@ Select a specific build with `--variant minicpm-v-4_7-1b-q4`,
`--variant minicpm-v-4_7-1b-q5`, `--variant minicpm-v-4_7-1b-q6`,
or `--variant minicpm-v-4_7-1b-q8`.

### Underdog Saluki abliterated builds

`underdog-saluki-27b-1.0-abliterated` provides Kujira's mixed IQ2 adaptation
of Underdog Saluki for text chat, reasoning, and tool use. Its MTP variant
enables multi-token prediction with up to two draft tokens. Both builds use
the same weights, the embedded chat template, and a 32,768-token context.

To select the MTP build explicitly:

```bash
local-ai models install --variant underdog-saluki-27b-1.0-abliterated-mtp underdog-saluki-27b-1.0-abliterated
```

These entries are text-only and do not include a vision projector.

### Repairing an existing model configuration

Gallery template updates do not rewrite YAML files for already installed models.
Expand Down
87 changes: 87 additions & 0 deletions gallery/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,93 @@
# written by scripts/models/cmd/modelcatalog and stays only so releases before
# the cutover keep working. CI fails when it differs from the generated copy:
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: underdog-saluki-27b-1.0-abliterated
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
uses ordinary decoding; the MTP variant enables speculative decoding.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: underdog-saluki-27b-1.0-abliterated-mtp
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: underdog-saluki-27b-1.0-abliterated-mtp
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
enables multi-token prediction with up to two draft tokens.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
- mtp
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
- spec_type:draft-mtp
- spec_n_max:2
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: adavar-3b-q4
description: |
Expand Down
103 changes: 103 additions & 0 deletions gallery/models/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,109 @@
# changes and never edit it by hand. CI fails when it is stale. A build from
# master reads it as the default model source; a tagged build reads the gallery
# until the release cutover.
- kind: model
id: underdog-saluki-27b-1.0-abliterated
name: underdog-saluki-27b-1.0-abliterated
version: 0.0.0-gallery.cdab61f388f4
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
uses ordinary decoding; the MTP variant enables speculative decoding.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: underdog-saluki-27b-1.0-abliterated-mtp
- kind: model
id: underdog-saluki-27b-1.0-abliterated-mtp
name: underdog-saluki-27b-1.0-abliterated-mtp
version: 0.0.0-gallery.774b2b47f0dc
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
enables multi-token prediction with up to two draft tokens.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
- mtp
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
- spec_type:draft-mtp
- spec_n_max:2
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- kind: model
id: adavar-3b-q4
name: adavar-3b-q4
Expand Down
Loading