External models (Gemini Nano Banana & OpenAI GPT Image) (#8633)

docs-old/contributing/index.md

@@ -8,6 +8,10 @@ We welcome contributions, whether features, bug fixes, code cleanup, testing, co
 
 If you’d like to help with development, please see our [development guide](contribution_guides/development.md).
 
+## External Providers
+
+If you are adding external image generation providers or configs, see our [external provider integration guide](EXTERNAL_PROVIDERS.md).
+
 **New Contributors:** If you’re unfamiliar with contributing to open source projects, take a look at our [new contributor guide](contribution_guides/newContributorChecklist.md).
 
 ## Nodes

docs/contributing/EXTERNAL_PROVIDERS.md

@@ -0,0 +1,129 @@
+# External Provider Integration
+
+This guide covers:
+
+1. Adding a new **external model** (most common; existing provider).
+2. Adding a brand-new **external provider** (adapter + config + UI wiring).
+
+## 1) Add a New External Model (Existing Provider)
+
+For provider-backed models (for example, OpenAI or Gemini), the source of truth is
+`invokeai/backend/model_manager/starter_models.py`.
+
+### Required model fields
+
+Define a `StarterModel` with:
+
+- `base=BaseModelType.External`
+- `type=ModelType.ExternalImageGenerator`
+- `format=ModelFormat.ExternalApi`
+- `source="external://<provider_id>/<provider_model_id>"`
+- `name`, `description`
+- `capabilities=ExternalModelCapabilities(...)`
+- optional `default_settings=ExternalApiModelDefaultSettings(...)`
+
+Example:
+
+```python
+new_external_model = StarterModel(
+    name="Provider Model Name",
+    base=BaseModelType.External,
+    source="external://openai/my-model-id",
+    description=(
+        "Provider model (external API). "
+        "Requires a configured OpenAI API key and may incur provider usage costs."
+    ),
+    type=ModelType.ExternalImageGenerator,
+    format=ModelFormat.ExternalApi,
+    capabilities=ExternalModelCapabilities(
+        modes=["txt2img", "img2img", "inpaint"],
+        supports_negative_prompt=False,
+        supports_seed=False,
+        supports_guidance=False,
+        supports_steps=False,
+        supports_reference_images=True,
+        max_images_per_request=4,
+    ),
+    default_settings=ExternalApiModelDefaultSettings(
+        width=1024,
+        height=1024,
+        num_images=1,
+    ),
+)
+```
+
+Then append it to `STARTER_MODELS`.
+
+### Required description text
+
+External starter model descriptions must clearly state:
+
+- an API key is required
+- usage may incur provider-side costs
+
+### Capabilities must be accurate
+
+These flags directly control UI visibility and request payload fields:
+
+- `supports_negative_prompt`
+- `supports_seed`
+- `supports_guidance`
+- `supports_steps`
+- `supports_reference_images`
+
+`supports_steps` is especially important: if `False`, steps are hidden for that model and `steps` is sent as `null`.
+
+### Source string stability
+
+Starter overrides are matched by `source` (`external://provider/model-id`). Keep this stable:
+
+- runtime capability/default overrides depend on it
+- installation detection in starter-model APIs depends on it
+
+`STARTER_MODELS` enforces unique `source` values with an assertion.
+
+### Install behavior notes
+
+- External starter models are managed in **External Providers** setup (not the regular Starter Models tab).
+- External starter models auto-install when a provider is configured.
+- Removing a provider API key removes installed external models for that provider.
+
+## 2) Credentials and Config
+
+External provider API keys are stored separately from `invokeai.yaml`:
+
+- default file: `~/invokeai/api_keys.yaml`
+- resolved path: `<INVOKEAI_ROOT>/api_keys.yaml`
+
+Non-secret provider settings (for example base URL overrides) stay in `invokeai.yaml`.
+
+Environment variables are still supported, e.g.:
+
+- `INVOKEAI_EXTERNAL_GEMINI_API_KEY`
+- `INVOKEAI_EXTERNAL_OPENAI_API_KEY`
+
+## 3) Add a New Provider (Only If Needed)
+
+If your model uses a provider that is not already integrated:
+
+1. Add config fields in `invokeai/app/services/config/config_default.py`
+   `external_<provider>_api_key` and optional `external_<provider>_base_url`.
+2. Add provider field mapping in `invokeai/app/api/routers/app_info.py`
+   (`EXTERNAL_PROVIDER_FIELDS`).
+3. Implement provider adapter in `invokeai/app/services/external_generation/providers/`
+   by subclassing `ExternalProvider`.
+4. Register the provider in `invokeai/app/api/dependencies.py` when building
+   `ExternalGenerationService`.
+5. Add starter model entries using `source="external://<provider>/<model-id>"`.
+6. Optional UI ordering tweak:
+   `invokeai/frontend/web/src/features/modelManagerV2/subpanels/AddModelPanel/ExternalProviders/ExternalProvidersForm.tsx`
+   (`PROVIDER_SORT_ORDER`).
+
+## 4) Optional Manual Installation
+
+You can also install external models directly via:
+
+`POST /api/v2/models/install?source=external://<provider_id>/<provider_model_id>`
+
+If omitted, `path`, `source`, and `hash` are auto-populated for external model configs.
+Set capabilities conservatively; the external generation service enforces capability checks at runtime.

docs/features/external-models/gemini.md

@@ -0,0 +1,46 @@
+---
+title: Google Gemini
+---
+
+# :material-google: Google Gemini
+
+Invoke supports Google's Gemini image generation models through the Gemini API. This provider is a good fit if you want high-quality text-to-image and reference-based image edits without running a local model.
+
+## Getting an API Key
+
+1. Open [Google AI Studio](https://aistudio.google.com/) and sign in with your Google account.
+2. Generate a new API key.
+3. Note the key — it will only be shown once.
+
+## Configuration
+
+Add your key to `api_keys.yaml` in your Invoke root directory:
+
+```yaml
+external_gemini_api_key: "your-gemini-api-key"
+
+# Optional — only set this if you need to route requests through a different endpoint
+external_gemini_base_url: "https://generativelanguage.googleapis.com"
+```
+
+Restart Invoke for the change to take effect.
+
+## Available Models
+
+| Model | Modes | Reference Images | Notes |
+| --- | --- | --- | --- |
+| **Gemini 2.5 Flash Image** | txt2img, img2img, inpaint | Yes | 10 aspect ratios, fixed per-ratio resolutions. |
+| **Gemini 3 Pro Image Preview** | txt2img, img2img, inpaint | Up to 14 (6 object + 5 character) | 1K / 2K / 4K resolution presets. |
+| **Gemini 3.1 Flash Image Preview** | txt2img, img2img, inpaint | Up to 14 (10 object + 4 character) | 512 / 1K / 2K / 4K resolution presets. |
+
+All Gemini models are single-image-per-request — batch size is fixed at 1. To generate multiple variations, queue multiple invocations.
+
+## Provider-Specific Options
+
+Gemini exposes a **temperature** control in the parameters panel. Lower values make outputs more deterministic, higher values increase variability.
+
+## Tips
+
+- **Reference images** are sent directly to the API as inlined PNG data. Large references increase request latency and cost — crop tightly where possible.
+- **Aspect ratios** are mapped to the closest Gemini-supported ratio. For Gemini 3 models, use the resolution presets to stay at the provider's native output sizes and avoid unnecessary rescaling.
+- **Pricing** varies by model and region. Check Google's documentation before running large batches.

docs/features/external-models/index.md

@@ -0,0 +1,58 @@
+---
+title: External Models
+---
+
+# :material-cloud-outline: External Models
+
+External models let you generate images in Invoke by calling third-party image generation APIs instead of running a model locally. This is useful when:
+
+- You don't have the GPU or VRAM to run a model locally.
+- You want access to closed-source models (e.g. GPT Image, Gemini).
+- You need a specific provider capability (very high resolutions, fast batches, bilingual text rendering, etc.).
+
+External models appear in the model picker alongside locally installed models. Generations are routed to the provider's API, billed against your provider account, and the resulting images are imported back into Invoke like any other generation.
+
+## Supported Providers
+
+- [Google Gemini](gemini.md) — Gemini 2.5 Flash Image, Gemini 3 Pro Image Preview, Gemini 3.1 Flash Image Preview
+- [OpenAI](openai.md) — GPT Image 1 / 1.5 / 1-mini, DALL·E 3, DALL·E 2
+
+## Configuring API Keys
+
+External provider credentials are stored in a dedicated `api_keys.yaml` file alongside `invokeai.yaml` in your Invoke root directory.
+
+```yaml
+# api_keys.yaml
+external_gemini_api_key: "your-gemini-api-key"
+external_openai_api_key: "your-openai-api-key"
+
+# Optional: override the provider base URL (e.g. for a compatible proxy or regional endpoint)
+external_gemini_base_url: "https://generativelanguage.googleapis.com"
+external_openai_base_url: "https://api.openai.com"
+```
+
+Restart Invoke after editing `api_keys.yaml` so the new values are picked up.
+
+!!! warning "Keep your keys private"
+    `api_keys.yaml` contains secrets. Do not commit it to version control and do not share it with other users of your machine.
+
+## Installing External Models
+
+External models are listed in the starter models dialog under their provider. Install them like any other starter model — Invoke records a model reference but does not download weights (there are no weights to download).
+
+Once installed, external models show up everywhere a model can be selected. Choose one, set the usual parameters (prompt, dimensions, num images, etc.), and invoke as normal.
+
+## Capabilities and Settings Visibility
+
+Each external model declares its own **capabilities** — for example:
+
+- Which generation modes it supports (`txt2img`, `img2img`, `inpaint`).
+- Whether it accepts reference images, and how many.
+- Which aspect ratios and resolutions it allows.
+- Whether it supports a negative prompt, seed, or batch size > 1.
+
+Invoke uses these capabilities to drive the UI: only the settings a given model actually supports will be shown in the parameters panel. If a field you expect is missing, it's because the selected model does not support it.
+
+## Costs and Rate Limits
+
+External providers charge for each request. Check the provider's pricing page before running large batches. Rate-limit errors from the provider are surfaced in Invoke as generation failures — wait a moment and try again, or lower your concurrent batch size.

docs/features/external-models/openai.md

@@ -0,0 +1,56 @@
+---
+title: OpenAI
+---
+
+# :material-alpha-o-circle-outline: OpenAI
+
+Invoke supports OpenAI's image generation models — both the GPT Image family and the older DALL·E models — through the OpenAI API.
+
+## Getting an API Key
+
+1. Open the [OpenAI API Platform](https://platform.openai.com/api-keys) and sign in.
+2. Create a new secret API key.
+3. Make sure your account has billing set up — image endpoints are paid per request.
+
+## Configuration
+
+Add your key to `api_keys.yaml` in your Invoke root directory:
+
+```yaml
+external_openai_api_key: "sk-..."
+
+# Optional — use this to point at a compatible proxy or Azure OpenAI deployment
+external_openai_base_url: "https://api.openai.com"
+```
+
+Restart Invoke for the change to take effect.
+
+## Available Models
+
+| Model | Modes | Aspect Ratios | Batch | Notes |
+| --- | --- | --- | --- | --- |
+| **GPT Image 1.5** | txt2img, img2img, inpaint | 1:1, 3:2, 2:3 | up to 10 | Fastest and cheapest GPT Image model. |
+| **GPT Image 1** | txt2img, img2img, inpaint | 1:1, 3:2, 2:3 | up to 10 | Highest quality of the GPT Image family. |
+| **GPT Image 1 Mini** | txt2img, img2img, inpaint | 1:1, 3:2, 2:3 | up to 10 | ~80% cheaper than GPT Image 1. |
+| **DALL·E 3** | txt2img only | 1:1, 7:4, 4:7 | 1 | No reference-image / edit support. |
+| **DALL·E 2** | txt2img, img2img, inpaint | 1:1 | up to 10 | Square only. |
+
+## Provider-Specific Options
+
+For **GPT Image** models, Invoke surfaces two provider-specific options in the parameters panel:
+
+- **Quality** — `low`, `medium`, `high`, or `auto`. Higher quality costs more and takes longer.
+- **Background** — `auto`, `transparent`, or `opaque`. Use `transparent` for PNG output with an alpha channel.
+
+DALL·E 2 and DALL·E 3 do not expose these options.
+
+## How Requests Are Routed
+
+- Pure text-to-image requests hit `/v1/images/generations`.
+- Any request with an init image or reference images is sent to `/v1/images/edits` instead. This is done transparently — you don't need to pick an endpoint.
+
+## Tips
+
+- **Batching** on GPT Image and DALL·E 2 tops out at 10 per request. Larger batches are split into multiple API calls.
+- **Costs** can climb quickly with high-quality GPT Image generations. Start with GPT Image 1 Mini when iterating on prompts.
+- **Rate limits** from OpenAI surface as failed invocations — retry after a short wait.

docs/src/generated/settings.json

@@ -723,6 +723,50 @@
       "required": false,
       "type": "<class 'bool'>",
       "validation": {}
+    },
+    {
+      "category": "EXTERNAL PROVIDERS",
+      "default": null,
+      "description": "API key for Gemini image generation.",
+      "env_var": "INVOKEAI_EXTERNAL_GEMINI_API_KEY",
+      "literal_values": [],
+      "name": "external_gemini_api_key",
+      "required": false,
+      "type": "typing.Optional[str]",
+      "validation": {}
+    },
+    {
+      "category": "EXTERNAL PROVIDERS",
+      "default": null,
+      "description": "API key for OpenAI image generation.",
+      "env_var": "INVOKEAI_EXTERNAL_OPENAI_API_KEY",
+      "literal_values": [],
+      "name": "external_openai_api_key",
+      "required": false,
+      "type": "typing.Optional[str]",
+      "validation": {}
+    },
+    {
+      "category": "EXTERNAL PROVIDERS",
+      "default": null,
+      "description": "Base URL override for Gemini image generation.",
+      "env_var": "INVOKEAI_EXTERNAL_GEMINI_BASE_URL",
+      "literal_values": [],
+      "name": "external_gemini_base_url",
+      "required": false,
+      "type": "typing.Optional[str]",
+      "validation": {}
+    },
+    {
+      "category": "EXTERNAL PROVIDERS",
+      "default": null,
+      "description": "Base URL override for OpenAI image generation.",
+      "env_var": "INVOKEAI_EXTERNAL_OPENAI_BASE_URL",
+      "literal_values": [],
+      "name": "external_openai_base_url",
+      "required": false,
+      "type": "typing.Optional[str]",
+      "validation": {}
     }
   ]
 }