Skip to content

docs: add LocalAI integration tutorial - #13770

Open
moonming wants to merge 4 commits into
apache:masterfrom
moonming:localai-integration-docs
Open

docs: add LocalAI integration tutorial#13770
moonming wants to merge 4 commits into
apache:masterfrom
moonming:localai-integration-docs

Conversation

@moonming

@moonming moonming commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

  • add an official tutorial for proxying LocalAI's OpenAI-compatible APIs through APISIX
  • allowlist only /v1/models and /v1/chat/completions; bind LocalAI to loopback so clients cannot bypass APISIX
  • cover non-streaming and streaming chat, LocalAI bearer authentication, optional APISIX key-auth, production safeguards, and cleanup
  • add the tutorial to the English documentation sidebar

Validation

Current head 513ad3fa8faa4027000029a8459f04efa765f9a7:

  • npx -p markdownlint-cli@0.25.0 markdownlint docs/en/latest/tutorials/proxy-localai.md
  • uvx --from codespell==2.1.0 codespell --ignore-words=.ignore_words docs/en/latest/tutorials/proxy-localai.md
  • python3 utils/check-category.py
  • python3 -m json.tool docs/en/latest/config.json
  • git diff --check origin/master...HEAD
  • extracted all shell blocks and passed them to bash -n
  • verified in Bash and Zsh that a failed streaming request, either failed SSE assertion, or the first failed 401 assertion leaves a nonzero final status

Real, no-mock E2E was run at parent head 63234f354030c426d0f4736752af8769424a2204. The subsequent commits only strengthen shell failure propagation, cleanup, credential wording, and the Docker version requirement; the Route, plugins, upstream, and authentication configuration exercised below are unchanged:

  • APISIX 3.17.0-0; Docker Engine 29.2.1; LocalAI v4.7.1 image quay.io/go-skynet/local-ai@sha256:66c8824a4bd85f24c55c14b916ecac14550c684d64c7b8fcb5000952d665c30a
  • real Llama 3.2 1B GGUF pinned to Hugging Face revision 7d1f70022fcab2038000074bd0342e03e1d8b755, SHA-256 1d0e9419ec4e12aef73ccf4ffd122703e94c48344a96bc7c5f0f2772c2152ce3
  • Docker port binding was exactly 127.0.0.1:8080; no public LocalAI listener
  • authenticated /v1/models returned 200; missing LocalAI bearer returned 401
  • non-streaming response was chat.completion with the exact text APISIX reaches LocalAI
  • streaming response produced 99 data: events and one [DONE]; the first event arrived 1,404 ms before completion
  • /v1/backend/monitor returned 404 through APISIX, while the direct LocalAI path was registered (non-404)
  • after enabling key-auth, missing gateway key and missing LocalAI bearer independently returned 401; both credentials returned 200
  • the Route PATCH retained the exact URI allowlist, proxy-buffering, proxy-rewrite, timeouts, and added key-auth
  • Route and Consumer deletion returned 200; both Admin API lists were empty afterward; the isolated test profile and its volumes were deleted

This repository does not have a tutorial E2E harness, so the PR keeps its scope documentation-only and records the reproducible real-service validation above.

LocalAI publishes versioned CPU images to both Docker Hub and Quay: https://localai.io/installation/containers/

Related: mudler/LocalAI#11215 and mudler/LocalAI#11294

Signed-off-by: Ming Wen <moonbingbing@gmail.com>
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. doc Documentation things labels Aug 3, 2026
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Documentation things size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant