Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.26.10"
go-version: "1.26.11"

- name: Download dependencies
run: go mod download
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ and pull requests are welcome.

## Development

The project requires Go 1.26.10 or newer.
The project requires Go 1.26.11 or newer.

```sh
go test ./...
Expand Down
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ print(response.choices[0].message.content)
| --- | --- |
| `GET /v1/models` | Lists the available models, prices and capabilities. |
| `POST /v1/chat/completions` | Creates chat completions; supports streaming. |
| `POST /v1/images/generations` | Generates images with image models (OpenAI images API, inline `b64_json` only; H3 per token; `Idempotency-Key` supported). |

Supported capabilities depend on the selected model and can include streaming,
tool calls, parallel tool calls, structured outputs and reasoning. The current
Expand All @@ -91,6 +92,16 @@ keep a valid `Retry-After` header. Errors include `origin` (`gateway` or
details are not exposed. A configured fallback provider is tried only after a
rate limit or a server-side failure, never after a rejected request.

Image generation follows the OpenAI images API: `model`, `prompt`, and `n`
(1 to 4). Images come back inline as `b64_json` and are not stored; a prompt
the provider refuses returns 422 `content_policy_violation` and costs
nothing. Send an `Idempotency-Key` header to retry safely: a retry of a
finished request returns the same images without a second charge.
`IMAGE_GENERATION_TIMEOUT_SECONDS` (default 660) bounds one generation and
`IMAGE_GENERATION_CONCURRENCY_PER_ACCOUNT` (default 2) how many one account
runs at once. Operator notes are in the Nexus repository's
`docs/image-generation.md`.

Responses have no limit on total duration. Two stall guards apply instead:
`PROVIDER_SILENCE_TIMEOUT_SECONDS` (default 900) ends a request when the
provider sends nothing, neither headers nor bytes, for that long, and
Expand Down Expand Up @@ -150,3 +161,18 @@ provider enforces it. Metering holds credit for that limit, so a guest's hold
fits its credit; other requests without `max_tokens` hold the model's maximum
output. An invalid value stops the gateway at startup. The Nexus repository's
`docs/guest-chat.md` describes guest chat.

### Image billing

H3 uses separate retail
rates for text input, reference-image input and image output tokens. Model discovery
publishes `image_input_price_per_1m_tokens_cents` alongside text-input/output rates,
and `image_max_cost_per_image_cents` for the maximum base charge and credit hold.
Exact token counts are validated from MiniMax usage and sent privately to metering.
Provider token costs remain independent of retail prices.

Successful responses include `nexus_billing: {"charged_cents": 4.3, "currency": "EUR"}`
when metering returns a charge receipt. It is the amount actually deducted, including
any prepaid surcharge. Idempotent replays include the same receipt without charging
again. Failed/refused requests have no retail charge. Billing uses the rates and
ceiling saved when the hold was admitted; completion releases the unused hold.
2 changes: 1 addition & 1 deletion apps/gateway/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# ---- Build stage ----
FROM golang:1.26.10-alpine AS builder
FROM golang:1.26.11-alpine AS builder

WORKDIR /app

Expand Down
2 changes: 1 addition & 1 deletion apps/gateway/Dockerfile.enclave
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# deploy/nitro/verify-build.sh. Build this image with `docker buildx build`
# only -- a plain `docker build` skips the timestamp normalisation and silently
# produces different measurements.
FROM golang:1.26.10-alpine3.24@sha256:27f2d3b83cfc5803c8078f8e730af0f7c431bf067afaec8f60a36fb763187cba AS builder
FROM golang:1.26.11-alpine3.24@sha256:ae52b5bc6ed045a9ed26ab03dc19327a802a9cc9aa8b27ed57f5c6b60579b0bb AS builder

ARG SOURCE_REVISION
ARG AWS_REGION
Expand Down
5 changes: 5 additions & 0 deletions apps/gateway/internal/adapters/http/dto/models_response.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ type ModelData struct {
OutputPricePer1MTokensUSD *float64 `json:"output_price_per_1m_tokens_usd,omitempty"`
CacheReadPricePer1MTokensUSD *float64 `json:"cache_read_price_per_1m_tokens_usd,omitempty"`
CacheWritePricePer1MTokensUSD *float64 `json:"cache_write_price_per_1m_tokens_usd,omitempty"`
// Image models use three token rates plus a
// maximum per image. The maximum excludes any account surcharge.
ImageInputPricePer1MTokensCents *int64 `json:"image_input_price_per_1m_tokens_cents,omitempty"`
ImageInputPricePer1MTokensUSD *float64 `json:"image_input_price_per_1m_tokens_usd,omitempty"`
ImageMaxCostPerImageCents *float64 `json:"image_max_cost_per_image_cents,omitempty"`
// Capabilities is what the operator states the model accepts, using
// models.dev field names. A missing field means not stated.
Capabilities json.RawMessage `json:"capabilities,omitempty"`
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package handlers

import (
"io"
"net/http"
"strings"

"github.com/dappnode/dappnode-nexus-gateway/apps/gateway/internal/adapters/http/mapper"
"github.com/dappnode/dappnode-nexus-gateway/apps/gateway/internal/application/ports"
"github.com/dappnode/dappnode-nexus-gateway/apps/gateway/internal/application/services"
"github.com/dappnode/dappnode-nexus-gateway/pkg/domain"
)

// ImageGenerationsHandler handles POST /v1/images/generations, the OpenAI
// images API. Images come back inline (b64_json) and are not stored.
type ImageGenerationsHandler struct {
service *services.GenerateService
logger ports.Logger
}

func NewImageGenerationsHandler(service *services.GenerateService, logger ports.Logger) *ImageGenerationsHandler {
return &ImageGenerationsHandler{service: service, logger: logger}
}

func (h *ImageGenerationsHandler) Handle(w http.ResponseWriter, r *http.Request) {
token, err := ExtractBearerToken(r)
if err != nil {
rejectRequest(w, r, h.logger, domain.EndpointImageGenerations, err)
return
}
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 32<<20))
if err != nil {
rejectRequest(w, r, h.logger, domain.EndpointImageGenerations, domain.ErrInvalidField("failed to read request body"))
return
}
req, err := mapper.ImageGenerationRequestToDomain(body)
if err != nil {
rejectRequest(w, r, h.logger, domain.EndpointImageGenerations, err)
return
}
req.ImageEdit = r.URL.Path == "/v1/images/edits"
if req.ImageEdit != mapper.HasImageInputs(body) {
rejectRequest(w, r, h.logger, domain.EndpointImageGenerations, domain.ErrInvalidField("send reference images to /v1/images/edits; generations takes text only"))
return
}
call, err := h.service.GenerateImages(r.Context(), body, req, token, strings.TrimSpace(r.Header.Get("Idempotency-Key")))
if err != nil {
WriteErrorWithLog(w, r, h.logger, err)
return
}
w.Header().Set("Content-Type", "application/json")
w.Header().Set("Cache-Control", "no-store")
if call.Replayed {
w.Header().Set("Idempotent-Replayed", "true")
}
w.WriteHeader(http.StatusOK)
_, _ = w.Write(call.Body)
}
30 changes: 28 additions & 2 deletions apps/gateway/internal/adapters/http/handlers/models_handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ func toPublicModelData(m domain.PublicModel, eurToUSDRate float64) dto.ModelData
inputPrice := eurToCents(m.InputPricePerMillion)
outputPrice := eurToCents(m.OutputPricePerMillion)
proofMode := m.EffectiveProofMode()
return dto.ModelData{
data := dto.ModelData{
ID: m.PublicModelID,
Object: "model",
Kind: string(domain.CatalogKindPublicModel),
Expand All @@ -73,11 +73,21 @@ func toPublicModelData(m domain.PublicModel, eurToUSDRate float64) dto.ModelData
OutputPricePer1MTokensUSD: priceToUSDPtr(m.Currency, m.OutputPricePerMillion, eurToUSDRate),
CacheReadPricePer1MTokensUSD: pricePtrToUSDPtr(m.Currency, m.CacheReadPricePerMillion, eurToUSDRate),
CacheWritePricePer1MTokensUSD: pricePtrToUSDPtr(m.Currency, m.CacheWritePricePerMillion, eurToUSDRate),
ImageInputPricePer1MTokensCents: eurPtrToCentsPtr(m.ImageInputPricePerMillion),
ImageInputPricePer1MTokensUSD: pricePtrToUSDPtr(m.Currency, m.ImageInputPricePerMillion, eurToUSDRate),
ImageMaxCostPerImageCents: fractionalCents(m.ImageMaxCostPerImage),
Capabilities: publishedCapabilities(m),
Endpoints: modelEndpoints(m),
ProofMode: proofMode,
ProofsEnabled: domain.ProofModeEnabled(proofMode),
}
if !m.SupportsChatCompletions && m.SupportsEndpoint(domain.EndpointImageGenerations) {
// Image requests have no chat token limits; token-priced image models
// still publish their retail text/reference/output rates.
data.ContextSize, data.MaxOutputTokens = nil, nil

}
return data
}

func toRouterData(r domain.RouterEntry) dto.ModelData {
Expand Down Expand Up @@ -112,13 +122,29 @@ func modelEndpoints(m domain.PublicModel) []string {
if m.SupportsChatCompletions {
endpoints = append(endpoints, "chat/completions")
}
if m.SupportsEndpoint(domain.EndpointImageGenerations) {
endpoints = append(endpoints, "images/generations")
if m.SupportsImageEdits() {
endpoints = append(endpoints, "images/edits")
}
}
return endpoints
}

func eurToCents(price decimal.Decimal) int64 {
return price.Mul(decimal.NewFromInt(100)).Round(0).IntPart()
}

// fractionalCents is a price in cents to microcent precision: per-image prices are
// often fractions of a cent.
func fractionalCents(price *decimal.Decimal) *float64 {
if price == nil {
return nil
}
cents, _ := price.Mul(decimal.NewFromInt(100)).Round(6).Float64()
return &cents
}

func eurPtrToCentsPtr(price *decimal.Decimal) *int64 {
if price == nil {
return nil
Expand All @@ -145,7 +171,7 @@ func priceToUSDPtr(currency string, price decimal.Decimal, eurToUSDRate float64)
if factor <= 0 {
return nil
}
usd, _ := price.Mul(decimal.NewFromFloat(factor)).Round(4).Float64()
usd, _ := price.Mul(decimal.NewFromFloat(factor)).Round(8).Float64()
return &usd
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"testing"

"github.com/dappnode/dappnode-nexus-gateway/pkg/domain"
"github.com/shopspring/decimal"
)

// The operator's record is published unchanged, including fields this Gateway
Expand All @@ -26,3 +27,26 @@ func TestModelsHandler_PublishesCapabilitiesUnchanged(t *testing.T) {
t.Fatalf("published capabilities nobody stated: %s", body)
}
}

func TestToPublicModelData_ImageModel(t *testing.T) {
rate := decimal.RequireFromString("1.5")
cap := decimal.RequireFromString("0.50360083")
data := toPublicModelData(domain.PublicModel{
PublicModelID: "minimax/h3-image", Currency: "EUR", UpstreamModelName: "MiniMax-H3-Image",
ProviderConfig: domain.ProviderConfig{ProviderName: "minimax"},
SupportsImageGenerations: true, ImageInputPricePerMillion: &rate, ImageMaxCostPerImage: &cap,
InputPricePerMillion: decimal.NewFromInt(2), OutputPricePerMillion: decimal.NewFromInt(4),
}, 1.1)
if len(data.Endpoints) != 2 || data.Endpoints[0] != "images/generations" || data.Endpoints[1] != "images/edits" {
t.Fatalf("endpoints = %v", data.Endpoints)
}
if data.ImageInputPricePer1MTokensCents == nil || *data.ImageInputPricePer1MTokensCents != 150 {
t.Fatalf("reference token rate = %v", data.ImageInputPricePer1MTokensCents)
}
if data.ImageMaxCostPerImageCents == nil || *data.ImageMaxCostPerImageCents != 50.360083 {
t.Fatalf("ceiling precision = %v", data.ImageMaxCostPerImageCents)
}
if data.ContextSize != nil || data.InputPricePer1MTokensCents == nil || *data.InputPricePer1MTokensCents != 200 {
t.Fatal("image model needs token prices and no context size")
}
}
116 changes: 116 additions & 0 deletions apps/gateway/internal/adapters/http/mapper/image_to_domain.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
package mapper

import (
"bytes"
"encoding/base64"
"encoding/json"
"fmt"
_ "golang.org/x/image/webp"
"image"
_ "image/jpeg"
_ "image/png"
"strings"

"github.com/dappnode/dappnode-nexus-gateway/pkg/domain"
)

// MaxImagesPerRequest bounds n on /v1/images/generations, so one request's
// hold and response stay small.
const MaxImagesPerRequest = 4

// imageRequest is what the gateway reads from an images/generations body.
// The body itself is forwarded as sent.
type imageRequest struct {
Model string `json:"model"`
Prompt *string `json:"prompt"`
N json.RawMessage `json:"n"`
Stream *bool `json:"stream"`
ResponseFormat *string `json:"response_format"`
}

// ImageGenerationRequestToDomain validates a /v1/images/generations body:
// a model, a prompt, and how many images (n, 1 by default). Images always
// come back inline as b64_json; streaming partial images is not offered.
func ImageGenerationRequestToDomain(raw json.RawMessage) (domain.GenerateRequest, error) {
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
return domain.GenerateRequest{}, domain.ErrInvalidField("invalid JSON body")
}
var req imageRequest
if err := json.Unmarshal(raw, &req); err != nil {
return domain.GenerateRequest{}, domain.ErrInvalidField("invalid request body: " + err.Error())
}
if strings.TrimSpace(req.Model) == "" {
return domain.GenerateRequest{}, domain.ErrInvalidField("model is required")
}
if req.Prompt == nil || strings.TrimSpace(*req.Prompt) == "" {
return domain.GenerateRequest{}, domain.ErrInvalidField("prompt is required")
}
if req.Stream != nil && *req.Stream {
return domain.GenerateRequest{}, domain.ErrUnsupportedFeature("streaming image generation")
}
if _, ok := fields["partial_images"]; ok {
return domain.GenerateRequest{}, domain.ErrUnsupportedFeature("partial_images")
}
if req.ResponseFormat != nil && *req.ResponseFormat != "b64_json" {
return domain.GenerateRequest{}, domain.ErrInvalidField("response_format must be b64_json: Nexus returns images inline and stores none")
}
if err := validateImageInputs(fields); err != nil {
return domain.GenerateRequest{}, err
}
n := 1
if len(req.N) > 0 && string(req.N) != "null" {
if err := json.Unmarshal(req.N, &n); err != nil || n < 1 || n > MaxImagesPerRequest {
return domain.GenerateRequest{}, domain.ErrInvalidField(fmt.Sprintf("n must be a whole number from 1 to %d", MaxImagesPerRequest))
}
}
return domain.GenerateRequest{
PublicModelID: req.Model,
RequestedModelID: req.Model,
Images: &n,
}, nil
}

// HasImageInputs distinguishes editing from text-to-image requests.
func HasImageInputs(raw []byte) bool {
var body map[string]json.RawMessage
_ = json.Unmarshal(raw, &body)
_, ok := body["images"]
return ok
}

func validateImageInputs(fields map[string]json.RawMessage) error {
raw, exists := fields["images"]
if !exists {
return nil
}
var images []struct {
ImageURL string `json:"image_url"`
}
if json.Unmarshal(raw, &images) != nil || len(images) < 1 || len(images) > 4 {
return domain.ErrInvalidField("images must contain 1 to 4 inline reference images")
}
total := 0
for _, item := range images {
header, encoded, ok := strings.Cut(item.ImageURL, ",")
if !ok || (header != "data:image/png;base64" && header != "data:image/jpeg;base64" && header != "data:image/webp;base64") {
return domain.ErrInvalidField("reference images must be PNG, JPEG or WebP base64 data URLs; remote URLs are not accepted")
}
if len(encoded) > 14<<20 {
return domain.ErrInvalidField("each reference image must be at most 10 MiB")
}
decoded, err := base64.StdEncoding.DecodeString(encoded)
if err != nil || len(decoded) > 10<<20 {
return domain.ErrInvalidField("invalid reference image or image exceeds 10 MiB")
}
total += len(decoded)
cfg, format, err := image.DecodeConfig(bytes.NewReader(decoded))
if err != nil || "data:image/"+format+";base64" != header || cfg.Width < 128 || cfg.Height < 128 || cfg.Width > 8192 || cfg.Height > 8192 || cfg.Width > 4*cfg.Height || cfg.Height > 4*cfg.Width {
return domain.ErrInvalidField("reference images must be 128–8192 px on each side, with an aspect ratio at most 4:1")
}
}
if total > 20<<20 {
return domain.ErrInvalidField("reference images must total at most 20 MiB")
}
return nil
}
Loading
Loading