rag-api

A local, from-scratch Retrieval-Augmented Generation (RAG) API. It stores personal profile text in a vector database, retrieves the most relevant chunks for a question, and uses a local LLM (via Ollama) to generate a grounded answer — all running on your own machine, no external API calls.

This started as a learning project to understand how RAG actually works under the hood: chunking, embeddings, retrieval, prompt augmentation, and generation.

What is RAG?

Retrieval-Augmented Generation combines a retrieval step with a generative model so the LLM answers using your own data instead of guessing from what it was trained on:

1. Retrieval — search a knowledge base (here, a ChromaDB vector store) for the chunks most relevant to the question.
2. Augmentation — merge those retrieved chunks with the original question into a single prompt, so the model has the exact context it needs.
3. Generation — send that augmented prompt to the LLM, which answers using only the provided context.

How it works

• profile.txt — a plain-text personal profile, split into chunks by blank line (paragraph). (Not included in this repo — it's .gitignored since it contains real personal info. Add your own to run this locally.)
• build_knowledge.py — reads profile.txt, chunks it, generates embeddings via Ollama's nomic-embed-text model, and stores everything in a persistent ChromaDB collection (./chroma_db).
• main.py — a single-user FastAPI app with one GET /ask endpoint. It embeds the question, retrieves the top 2 matching chunks, builds an augmented prompt, and sends it to a local qwen3:4b model via Ollama for a grounded answer.
• MultiUser.py — an extended version that supports multiple people's profiles in the same collection. Adds a POST /documents endpoint to submit a profile under a user_name, and lets GET /ask filter retrieval by user so each person only gets answers grounded in their own data.

Stack

• FastAPI — web framework for the API, with auto-generated Swagger UI for testing.
• Uvicorn — ASGI server that runs the FastAPI app.
• ChromaDB — vector database for storing and searching text via embeddings.
• Ollama — runs local models for both embeddings (nomic-embed-text) and generation (qwen3:4b), so nothing leaves your machine.

Setup

1. Install and start Ollama, then pull the models this project uses:

   ollama pull nomic-embed-text
   ollama pull qwen3:4b

2. Create a virtual environment and install dependencies:

   python3 -m venv venv
   source venv/bin/activate
   pip install fastapi uvicorn chromadb ollama pydantic

3. Add your profile data: Create a profile.txt in the project root with your own bio, written as paragraphs (separated by blank lines) — each paragraph becomes one retrievable chunk.

4. Build the knowledge base:

   python build_knowledge.py

   This embeds and stores your profile chunks in ./chroma_db.

5. Run the API:

   uvicorn main:app --reload

   For the multi-user version instead:

   uvicorn MultiUser:app --reload

Testing with Swagger UI

Once the server is running, open http://127.0.0.1:8000/docs in your browser. Swagger UI lists all available endpoints and lets you send requests and see responses directly, without needing curl or a separate client.

Endpoints

main.py

• GET /ask?question=... — retrieves the top 2 relevant chunks, augments the prompt, and returns a generated answer along with the context used.

MultiUser.py

• POST /documents — submit a profile: {"user_name": "...", "content": "..."}. The content is chunked and stored with user_name as metadata.
• GET /ask?question=...&user=... — same as above, but if user is provided, retrieval is filtered to that user's chunks only.

Notes

• Everything runs locally — Ollama serves both the embedding model and the chat model on http://localhost:11434.
• chroma_db/ is generated output (rebuild it any time with build_knowledge.py) and is .gitignored, along with venv/, __pycache__/, and profile.txt.