DocBot is a modern, production-grade AI application that enables users to have interactive conversations with their PDF documents. By leveraging Retrieval-Augmented Generation (RAG), DocBot provides context-aware answers grounded specifically in the content of your uploaded files.
- Asynchronous PDF Ingestion: High-performance background processing using BullMQ and Redis.
- Intelligent RAG Pipeline: Powered by LangChain and Google Gemini for precise context retrieval and response generation.
- Semantic Vector Search: Utilizes Qdrant for lightning-fast similarity searches across document embeddings.
- Real-time Streaming: ChatGPT-like streaming experience for instant AI feedback.
- Modern UI/UX: Professional dashboard built with Next.js, Tailwind CSS, and shadcn/ui.
- Dark/Light Mode: Full theme support with persistence via
next-themes. - Robust Error Handling: Integrated status tracking for processing, success, and failure states.
DocBot follows a decoupled, event-driven architecture designed for scalability:
- Frontend (Next.js): A responsive React interface for file management and chatting.
- API Layer: Serverless routes that handle uploads, document metadata, and the chat stream.
- Background Worker: A dedicated Node.js process that handles the heavy lifting of PDF parsing and vectorization.
- Vector Store (Qdrant): Stores high-dimensional embeddings for semantic retrieval.
- NoSQL Database (MongoDB): Manages file metadata, processing statuses, and project state.
- Framework: Next.js 15+ (App Router)
- Styling: Tailwind CSS v4, shadcn/ui
- Icons: Lucide React
- Themes: next-themes
- Runtime: Node.js
- Database: MongoDB (Mongoose)
- Vector DB: Qdrant
- Task Queue: BullMQ
- Cache/Queue Store: Redis
- LLM: Google Gemini 1.5 Pro / 2.5 Flash
- Embeddings: Google AI (text-embedding-004 / 2-preview)
- Orchestration: LangChain
- PDF Parsing: pdf-parse
- Upload: User uploads a PDF. The file is saved and metadata is stored in MongoDB with a
processingstatus. - Queue: An ingestion job is dispatched to a BullMQ queue backed by Redis.
- Worker: A standalone TS Worker picks up the job, extracts text using
pdf-parse, and splits it into semantic chunks using LangChain. - Embeddings: Each text chunk is converted into a vector embedding via the Gemini API.
- Store: Vectors and metadata are upserted into Qdrant. MongoDB status is updated to
completed. - Chat: When a user asks a question, the query is vectorized, the most relevant chunks are retrieved from Qdrant, and Gemini generates a grounded response.
Ensure you have the following installed:
- Node.js (v20+)
- Redis (Local or Cloud)
- Qdrant (Running via Docker or Managed)
- MongoDB (Compass/Local or Atlas)
git clone https://github.com Sohail52/Docbot.git
cd Docbotnpm installCreate a .env.local file in the root directory:
MONGODB_URI=mongodb://127.0.0.1:27017/docbot
REDIS_URL=redis://127.0.0.1:6379
QDRANT_URL=http://localhost:6333
GEMINI_API_KEY=your_google_ai_api_key_hereYou need to run two separate processes:
The Web Server:
npm run devThe Background Worker:
npm run worker- Select a Document: Upload a PDF via the drag-and-drop area in the sidebar.
- Wait for Processing: The sidebar will show a loading spinner until the vectorization is finished.
- Chat: Click on the "Ready" document. Type any question in the input bar.
- Context: Use the quick suggestions like "Summarize this file" for immediate insights.
POST /api/upload: Handlers PDF file uploads.GET /api/documents: Returns a list of all documents and their statuses.DELETE /api/documents/[id]: Cleans up MongoDB, physical files, and Qdrant vectors.POST /api/chat: The RAG endpoint for streaming query responses.
Distributed under the MIT License. See LICENSE for more information.