Adds Multimodal search page by kosabogi · Pull Request #7706 · elastic/docs-content

kosabogi · 2026-08-03T11:42:51Z

Summary

This PR:

Adds a Multimodal search section under vector search, and renames the existing semantic search page to Semantic search for text.
Documents multimodal concepts, model options, and the semantic field as the managed mapping path.
Moved the tutorial from the semantic field under the new Multimodal search section.
Moves the dense/sparse and OpenAI-compatible tutorials under semantic search for text.

Related issue: #7603

Generative AI disclosure

Did you use a generative AI (GenAI) tool to assist in creating this contribution?

Yes
No

Curson agents

github-actions · 2026-08-03T11:43:33Z

Elastic Docs AI PR menu

Check the box to run an AI review for this pull request.

Review docs changes (docs-review). Status: not started.

Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team.

github-actions · 2026-08-03T11:49:05Z

🔍 Preview links for changed docs

github-actions · 2026-08-03T11:49:15Z

Elastic Docs Style Checker (Vale)

Summary: 2 warnings, 2 suggestions found

⚠️ Warnings (2): Fix when the suggestion improves clarity or correctness.

File	Line	Rule	Message
solutions/search/multimodal-search.md	90	Elastic.Spelling	'Omni' is a possible misspelling.
solutions/search/search-approaches.md	24	Elastic.DontUse	Don't use 'just'.

💡 Suggestions (2): Optional style improvements. Apply when helpful.

File	Line	Rule	Message
solutions/search/multimodal-search.md	106	Elastic.Semicolons	Use semicolons judiciously.
solutions/search/multimodal-search/multimodal-search-tutorial.md	324	Elastic.WordChoice	Consider using 'open' instead of 'launch', unless the term is in the UI.

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

leemthompo

Looking good! Few thoughts as I read over this quickly :)

leemthompo · 2026-08-03T12:38:12Z

+
+## Multimodal embedding models [multimodal-embedding-models]
+
+Compare the multimodal embedding models available with {{es}}:


We're creating additional maintenance burden here by hardcoding another list of models. Let's put the embedding models into shared snippets and we can reuse them across multiple pages :)

leemthompo · 2026-08-03T12:39:08Z

+
+For the full Jina catalog, deployment matrix, and input examples, refer to [Jina models](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md). For EIS availability by stack version, refer to [Supported models on EIS](/explore-analyze/elastic-inference/eis-supported-models.md).
+
+You can deploy or access Jina multimodal models in these ways:


nit: even this is a candidate for a shared snippet considering the core info is already in Jina pages

leemthompo · 2026-08-03T12:42:44Z

+
+## Use the `semantic` field type [semantic-field-for-multimodal-search]
+
+The `semantic` field type is the simplest way to run multimodal search in {{es}}. For field parameters, defaults, supported input types, and limitations, refer to the [`semantic` field documentation](elasticsearch://reference/elasticsearch/mapping-reference/semantic-field.md).


i'd delete "The semantic field type is the simplest way to run multimodal search in {{es}}."

because we say the same thing better in "The semantic field type simplifies semantic and multimodal search across text, images, audio, video, and PDF files."

For field parameters, defaults, supported input types, and limitations, refer to the semantic field documentation.

I'd move this to just before the example

leemthompo · 2026-08-03T12:43:17Z

+
+Here's an example using the Jina Embeddings v5 Omni Small endpoint:
+
+```json


Suggested change

```json

```console

leemthompo · 2026-08-03T12:43:49Z

+    "properties": {
+      "content": {
+        "type": "semantic",
+        "inference_id": ".jina-embeddings-v5-omni-small"


nit: an annotation making it clear that inference_id is required for semantic, might be useful here

leemthompo · 2026-08-03T12:51:05Z

+- **[Jina API](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md#jina-external)**: The model runs on the hosted Jina platform. Use this when you want Jina-hosted {{infer}} outside EIS.
+- **[On-prem](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md#jina-on-prem)**: You run the model in Docker on your own infrastructure. Use this for air-gapped, offline, or compliance scenarios.
+
+## Use the `semantic` field type [semantic-field-for-multimodal-search]


imperative is out of the sync with the rest of the subheadings on this page

leemthompo · 2026-08-03T12:53:46Z

+Elasticsearch provides multiple ways to implement vector, semantic, and multimodal search, depending on how much control you need over embedding generation and retrieval.

-### Semantic search (managed workflows)
+### Semantic search for text (managed workflows)


We could use this opportunity to trim down the links on this page, to focus on the core value stuff and rely on the left-nav for the longer tail stuff like ELSER, Cohere etc.

This overview section is kinda overwhelming now which is the opposite of what we want in a curated overview :)

leemthompo · 2026-08-03T12:56:01Z

+
+You now have a `semantic` field that embeds images at index time and accepts text, image, or PDF input at search time. To adapt this tutorial to an application:
+
+- Replace the sample images and metadata with your own visual archive.


Suggested change

- Replace the sample images and metadata with your own visual archive.

- Ingest your own multimedia data

the original tutorial was a little rushed, can be more general here :)

leemthompo · 2026-08-03T12:56:44Z

+
+- [Multimodal search](../multimodal-search.md)
+- [`semantic` field type](elasticsearch://reference/elasticsearch/mapping-reference/semantic-field.md)
+- [Jina multimodal embeddings](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md#jina-multimodal-embeddings)


Suggested change

- [Jina multimodal embeddings](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md#jina-multimodal-embeddings)

- [Jina multimodal embeddings models](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md#jina-multimodal-embeddings)

leemthompo · 2026-08-03T12:57:11Z

+  - id: cloud-serverless
+type: tutorial
+---
+# Build multimodal search with a semantic field [multimodal-search-tutorial]


Suggested change

# Build multimodal search with a semantic field [multimodal-search-tutorial]

# Quickstart: Build a multimodal search in {{es}} [multimodal-search-tutorial]

adding "in {{es}}" for SEO + George's SEO proposal has a suggestion to prefix "quickstarts" e.g.

Quickstart: Build a multimodal search in {{es}}

Also I think "with a semantic field" is potentially superfluous info in the heading. Not super strongly held opinion however.

(using "quickstart" with one on the future gives us more naming options if we add new tutorials going forward)

kosabogi and others added 3 commits July 29, 2026 13:50

First draft of multimodal search content

43c0b99

Adds Multimodal search section

8bbd2bd

Merge branch 'main' into multimodal-search

226d44a

Fixes anchor errors

af32294

github-actions Bot deployed to docs-preview August 3, 2026 11:51 View deployment

leemthompo reviewed Aug 3, 2026

View reviewed changes


		## Multimodal embedding models [multimodal-embedding-models]

		Compare the multimodal embedding models available with {{es}}:


		For the full Jina catalog, deployment matrix, and input examples, refer to [Jina models](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md). For EIS availability by stack version, refer to [Supported models on EIS](/explore-analyze/elastic-inference/eis-supported-models.md).

		You can deploy or access Jina multimodal models in these ways:


		## Use the `semantic` field type [semantic-field-for-multimodal-search]

		The `semantic` field type is the simplest way to run multimodal search in {{es}}. For field parameters, defaults, supported input types, and limitations, refer to the [`semantic` field documentation](elasticsearch://reference/elasticsearch/mapping-reference/semantic-field.md).


		Here's an example using the Jina Embeddings v5 Omni Small endpoint:

		```json


		You now have a `semantic` field that embeds images at index time and accepts text, image, or PDF input at search time. To adapt this tutorial to an application:

		- Replace the sample images and metadata with your own visual archive.

	- Replace the sample images and metadata with your own visual archive.
	- Ingest your own multimedia data

	- [Jina multimodal embeddings](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md#jina-multimodal-embeddings)
	- [Jina multimodal embeddings models](/explore-analyze/machine-learning/nlp/ml-nlp-jina.md#jina-multimodal-embeddings)

	# Build multimodal search with a semantic field [multimodal-search-tutorial]
	# Quickstart: Build a multimodal search in {{es}} [multimodal-search-tutorial]

Uh oh!

Conversation

kosabogi commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Generative AI disclosure

Uh oh!

github-actions Bot commented Aug 3, 2026

Elastic Docs AI PR menu

Uh oh!

github-actions Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

github-actions Bot commented Aug 3, 2026

Elastic Docs Style Checker (Vale)

Uh oh!

leemthompo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leemthompo Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leemthompo Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kosabogi commented Aug 3, 2026 •

edited

Loading

github-actions Bot commented Aug 3, 2026 •

edited

Loading

leemthompo left a comment •

edited

Loading

leemthompo Aug 3, 2026 •

edited

Loading

leemthompo Aug 3, 2026 •

edited

Loading