docs: add LocalAI integration tutorial - #13770
Open
moonming wants to merge 4 commits into
Open
Conversation
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/v1/modelsand/v1/chat/completions; bind LocalAI to loopback so clients cannot bypass APISIXkey-auth, production safeguards, and cleanupValidation
Current head
513ad3fa8faa4027000029a8459f04efa765f9a7:npx -p markdownlint-cli@0.25.0 markdownlint docs/en/latest/tutorials/proxy-localai.mduvx --from codespell==2.1.0 codespell --ignore-words=.ignore_words docs/en/latest/tutorials/proxy-localai.mdpython3 utils/check-category.pypython3 -m json.tool docs/en/latest/config.jsongit diff --check origin/master...HEADshellblocks and passed them tobash -n401assertion leaves a nonzero final statusReal, no-mock E2E was run at parent head
63234f354030c426d0f4736752af8769424a2204. The subsequent commits only strengthen shell failure propagation, cleanup, credential wording, and the Docker version requirement; the Route, plugins, upstream, and authentication configuration exercised below are unchanged:3.17.0-0; Docker Engine29.2.1; LocalAIv4.7.1imagequay.io/go-skynet/local-ai@sha256:66c8824a4bd85f24c55c14b916ecac14550c684d64c7b8fcb5000952d665c30a7d1f70022fcab2038000074bd0342e03e1d8b755, SHA-2561d0e9419ec4e12aef73ccf4ffd122703e94c48344a96bc7c5f0f2772c2152ce3127.0.0.1:8080; no public LocalAI listener/v1/modelsreturned200; missing LocalAI bearer returned401chat.completionwith the exact textAPISIX reaches LocalAIdata:events and one[DONE]; the first event arrived 1,404 ms before completion/v1/backend/monitorreturned404through APISIX, while the direct LocalAI path was registered (non-404)key-auth, missing gateway key and missing LocalAI bearer independently returned401; both credentials returned200proxy-buffering,proxy-rewrite, timeouts, and addedkey-auth200; both Admin API lists were empty afterward; the isolated test profile and its volumes were deletedThis repository does not have a tutorial E2E harness, so the PR keeps its scope documentation-only and records the reproducible real-service validation above.
LocalAI publishes versioned CPU images to both Docker Hub and Quay: https://localai.io/installation/containers/
Related: mudler/LocalAI#11215 and mudler/LocalAI#11294