-# Code-Graph-RAG: A Graph-Based RAG System for Any Codebases
-
-An accurate Retrieval-Augmented Generation (RAG) system that analyzes multi-language codebases using Tree-sitter, builds comprehensive knowledge graphs, and enables natural language querying of codebase structure and relationships as well as editing capabilities.
+# Code-Graph-RAG
+Code-Graph-RAG parses a multi-language codebase with Tree-sitter, builds a knowledge graph of its structure in Memgraph, and lets you query, edit, and optimise that code in plain English. It works across a monorepo of mixed languages under one unified graph schema.
@@ -35,855 +67,128 @@ An accurate Retrieval-Augmented Generation (RAG) system that analyzes multi-lang
## Latest News 🔥
-- **[NEW]** **MCP Server Integration**: Code-Graph-RAG now works as an MCP server with Claude Code! Query and edit your codebase using natural language directly from Claude Code. [Setup Guide](docs/claude-code-setup.md)
-- [2025/10/21] **Semantic Code Search**: Added intent-based code search using UniXcoder embeddings. Find functions by describing what they do (e.g., "error handling functions", "authentication code") rather than by exact names.
-
-## 🚀 Features
-
-- **Multi-Language Support**:
-
-
-| Language | Status | Extensions | Functions | Classes/Structs | Modules | Package Detection | Additional Features |
-|--------|------|----------|---------|---------------|-------|-----------------|-------------------|
-| C++ | Fully Supported | .cpp, .h, .hpp, .cc, .cxx, .hxx, .hh, .ixx, .cppm, .ccm | ✓ | ✓ | ✓ | ✓ | Constructors, destructors, operator overloading, templates, lambdas, C++20 modules, namespaces |
-| Java | Fully Supported | .java | ✓ | ✓ | ✓ | - | Generics, annotations, modern features (records/sealed classes), concurrency, reflection |
-| JavaScript | Fully Supported | .js, .jsx | ✓ | ✓ | ✓ | - | ES6 modules, CommonJS, prototype methods, object methods, arrow functions |
-| Lua | Fully Supported | .lua | ✓ | - | ✓ | - | Local/global functions, metatables, closures, coroutines |
-| Python | Fully Supported | .py | ✓ | ✓ | ✓ | ✓ | Type inference, decorators, nested functions |
-| Rust | Fully Supported | .rs | ✓ | ✓ | ✓ | ✓ | impl blocks, associated functions |
-| TypeScript | Fully Supported | .ts, .tsx | ✓ | ✓ | ✓ | - | Interfaces, type aliases, enums, namespaces, ES6/CommonJS modules |
-| C# | In Development | .cs | ✓ | ✓ | ✓ | - | Classes, interfaces, generics (planned) |
-| Go | In Development | .go | ✓ | ✓ | ✓ | - | Methods, type declarations |
-| PHP | In Development | .php | ✓ | ✓ | ✓ | - | Classes, functions, namespaces |
-| Scala | In Development | .scala, .sc | ✓ | ✓ | ✓ | - | Case classes, objects |
-
-- **🌳 Tree-sitter Parsing**: Uses Tree-sitter for robust, language-agnostic AST parsing
-- **📊 Knowledge Graph Storage**: Uses Memgraph to store codebase structure as an interconnected graph
-- **🗣️ Natural Language Querying**: Ask questions about your codebase in plain English
-- **🤖 AI-Powered Cypher Generation**: Supports both cloud models (Google Gemini), local models (Ollama), and OpenAI models for natural language to Cypher translation
-- **🤖 OpenAI Integration**: Leverage OpenAI models to enhance AI functionalities.
-- **📝 Code Snippet Retrieval**: Retrieves actual source code snippets for found functions/methods
-- **✍️ Advanced File Editing**: Surgical code replacement with AST-based function targeting, visual diff previews, and exact code block modifications
-- **⚡️ Shell Command Execution**: Can execute terminal commands for tasks like running tests or using CLI tools.
-- **🚀 Interactive Code Optimization**: AI-powered codebase optimization with language-specific best practices and interactive approval workflow
-- **📚 Reference-Guided Optimization**: Use your own coding standards and architectural documents to guide optimization suggestions
-- **🔗 Dependency Analysis**: Parses `pyproject.toml` to understand external dependencies
-- **🎯 Nested Function Support**: Handles complex nested functions and class hierarchies
-- **🔄 Language-Agnostic Design**: Unified graph schema across all supported languages
-
-## 🏗️ Architecture
-
-The system consists of two main components:
-
-1. **Multi-language Parser**: Tree-sitter based parsing system that analyzes codebases and ingests data into Memgraph
-2. **RAG System** (`codebase_rag/`): Interactive CLI for querying the stored knowledge graph
-
-
-## 📋 Prerequisites
-
-- Python 3.12+
-- Docker & Docker Compose (for Memgraph)
-- **cmake** (required for building pymgclient dependency)
-- **ripgrep** (`rg`) (required for shell command text searching)
-- **For cloud models**: Google Gemini API key
-- **For local models**: Ollama installed and running
-- `uv` package manager
-
-### Installing cmake and ripgrep
-
-On macOS:
-```bash
-brew install cmake ripgrep
-```
+
+- **Ruby Support**: Ruby joins the graph through a new pluggable ast-grep tier that adds a language from a single YAML pattern file, emitting `Module`, `Function`, and `Class` nodes plus import edges without a hand-written parser.
+- **Structural Search & Replace**: Find and rewrite code by AST pattern with ast-grep, exposed as agent tools so you can match and transform structure across the whole codebase instead of relying on text or regex.
+- **Data-Flow Tracing**: New `FLOWS_TO` taint edges follow values through assignments, function calls, and I/O sinks, with coverage across C#, Java, C, and Go.
+
-On Linux (Ubuntu/Debian):
-```bash
-sudo apt-get update
-sudo apt-get install cmake ripgrep
-```
+See [NEWS.md](NEWS.md) for the full history.
-On Linux (CentOS/RHEL):
-```bash
-sudo yum install cmake
-sudo dnf install ripgrep
-# Note: ripgrep may need to be installed from EPEL or via cargo
-```
+## What It Does
-## 🛠️ Installation
+Point Code-Graph-RAG at a repository and it reads every source file, extracts functions, classes, methods, modules, and the relationships between them, and stores the result as an interconnected graph. Once the graph exists you can:
-```bash
-git clone https://github.com/vitali87/code-graph-rag.git
+- Ask questions about the codebase in natural language and get answers grounded in the real structure.
+- Retrieve the actual source of any function, class, or method by name or by intent.
+- Edit code through the agent with AST-based surgical patching and a diff preview before anything changes.
+- Optimise code against language best practices or your own coding standards.
+- Find dead code by walking call and reference edges from entry points.
+- Search and rewrite structurally by AST pattern with ast-grep.
-cd code-graph-rag
-```
+## How It Works
-2. **Install dependencies**:
+The system has two components:
-For basic Python support:
-```bash
-uv sync
-```
+1. **Multi-language parser.** A Tree-sitter based parser reads the codebase and ingests functions, classes, methods, modules, and their relationships into Memgraph under a single language-agnostic schema.
+2. **RAG system** (`codebase_rag/`). An interactive CLI that turns natural language into Cypher queries, retrieves matching code, and drives AI-powered editing and optimisation.
-For full multi-language support:
-```bash
-uv sync --extra treesitter-full
```
-
-For development (including tests and pre-commit hooks):
-```bash
-make dev
+Source Code -> Tree-sitter Parser -> AST Analysis -> Memgraph Knowledge Graph
+ |
+User Query -> AI Model (Cypher Gen) -> Cypher Query -> Graph Results -> Response
```
-This installs all dependencies and sets up pre-commit hooks automatically.
+See the [Architecture Overview](docs/architecture/overview.md) and [Graph Schema](docs/architecture/graph-schema.md) for the full picture.
-This installs Tree-sitter grammars for all supported languages (see Multi-Language Support section).
-
-3. **Set up environment variables**:
-```bash
-cp .env.example .env
-# Edit .env with your configuration (see options below)
-```
+## Supported Languages
-### Configuration Options
+Python, TypeScript, TSX, JavaScript, Rust, Go, Java, C, C++, C#, PHP, Lua, and Dart are fully supported. Scala is in development, and Ruby has structural support (modules, functions, classes, and imports) through the pluggable ast-grep tier. See the [Language Support](docs/architecture/language-support.md) matrix for per-language capabilities.
-The new provider-explicit configuration supports mixing different providers for orchestrator and cypher models.
+## Installation
-#### Option 1: All Ollama (Local Models)
+`cgr` is published to PyPI. Install it system-wide with the `treesitter-full` (all languages) and `semantic` (vector search) extras:
```bash
-# .env file
-ORCHESTRATOR_PROVIDER=ollama
-ORCHESTRATOR_MODEL=llama3.2
-ORCHESTRATOR_ENDPOINT=http://localhost:11434/v1
-
-CYPHER_PROVIDER=ollama
-CYPHER_MODEL=codellama
-CYPHER_ENDPOINT=http://localhost:11434/v1
-```
-
-#### Option 2: All OpenAI Models
-```bash
-# .env file
-ORCHESTRATOR_PROVIDER=openai
-ORCHESTRATOR_MODEL=gpt-4o
-ORCHESTRATOR_API_KEY=sk-your-openai-key
-
-CYPHER_PROVIDER=openai
-CYPHER_MODEL=gpt-4o-mini
-CYPHER_API_KEY=sk-your-openai-key
-```
+# with uv (recommended)
+uv tool install "code-graph-rag[treesitter-full,semantic]"
-#### Option 3: All Google Models
-```bash
-# .env file
-ORCHESTRATOR_PROVIDER=google
-ORCHESTRATOR_MODEL=gemini-2.5-pro
-ORCHESTRATOR_API_KEY=your-google-api-key
-
-CYPHER_PROVIDER=google
-CYPHER_MODEL=gemini-2.5-flash
-CYPHER_API_KEY=your-google-api-key
+# or with pipx
+pipx install "code-graph-rag[treesitter-full,semantic]"
```
-#### Option 4: Mixed Providers
-```bash
-# .env file - Google orchestrator + Ollama cypher
-ORCHESTRATOR_PROVIDER=google
-ORCHESTRATOR_MODEL=gemini-2.5-pro
-ORCHESTRATOR_API_KEY=your-google-api-key
-
-CYPHER_PROVIDER=ollama
-CYPHER_MODEL=codellama
-CYPHER_ENDPOINT=http://localhost:11434/v1
-```
+You also need Docker (for Memgraph), `cmake`, and `ripgrep`. Full prerequisites, source installs, and environment setup are in the [Installation](docs/getting-started/installation.md) guide.
-Get your Google API key from [Google AI Studio](https://aistudio.google.com/app/apikey).
+## Quick Start
-**Install and run Ollama**:
```bash
-# Install Ollama (macOS/Linux)
-curl -fsSL https://ollama.ai/install.sh | sh
-
-# Pull required models
-ollama pull llama3.2
-# Or try other models like:
-# ollama pull llama3
-# ollama pull mistral
-# ollama pull codellama
+# Start the packaged Memgraph + Qdrant stack (no compose file needed)
+cgr daemon up
-# Ollama will automatically start serving on localhost:11434
+# Parse a repository into the graph, then query it
+cgr start --repo-path /path/to/repo --update-graph --clean
+cgr start --repo-path /path/to/repo
```
-> **Note**: Local models provide privacy and no API costs, but may have lower accuracy compared to cloud models like Gemini.
+The [Quick Start](docs/getting-started/quickstart.md) guide walks through parsing, querying, and exporting in five minutes.
-4. **Start Memgraph database**:
-```bash
-docker-compose up -d
-```
+## MCP Server
-## 🛠️ Makefile Commands
-
-Use the Makefile for common development tasks:
-
-
-| Command | Description |
-|-------|-----------|
-| `make help` | Show this help message |
-| `make all` | Install everything for full development environment (deps, grammars, hooks, tests) |
-| `make install` | Install project dependencies with full language support |
-| `make python` | Install project dependencies for Python only |
-| `make dev` | Setup development environment (install deps + pre-commit hooks) |
-| `make test` | Run unit tests only (fast, no Docker) |
-| `make test-parallel` | Run unit tests in parallel (fast, no Docker) |
-| `make test-integration` | Run integration tests (requires Docker) |
-| `make test-all` | Run all tests including integration and e2e (requires Docker) |
-| `make test-parallel-all` | Run all tests in parallel including integration and e2e (requires Docker) |
-| `make clean` | Clean up build artifacts and cache |
-| `make build-grammars` | Build grammar submodules |
-| `make watch` | Watch repository for changes and update graph in real-time |
-| `make readme` | Regenerate README.md from codebase |
-| `make lint` | Run ruff check |
-| `make format` | Run ruff format |
-| `make typecheck` | Run type checking with ty |
-| `make check` | Run all checks: lint, typecheck, test |
-| `make pre-commit` | Run all pre-commit checks locally (comprehensive test before commit) |
-
-
-## 🎯 Usage
-
-The Code-Graph-RAG system offers four main modes of operation:
-1. **Parse & Ingest**: Build knowledge graph from your codebase
-2. **Interactive Query**: Ask questions about your code in natural language
-3. **Export & Analyze**: Export graph data for programmatic analysis
-4. **AI Optimization**: Get AI-powered optimization suggestions for your code.
-5. **Editing**: Perform surgical code replacements and modifications with precise targeting.
-
-### Step 1: Parse a Repository
-
-Parse and ingest a multi-language repository into the knowledge graph:
-
-**For the first repository (clean start):**
-```bash
-cgr start --repo-path /path/to/repo1 --update-graph --clean
-```
+Code-Graph-RAG runs as an [MCP](https://modelcontextprotocol.io) server so Claude Code and other MCP clients can query and edit your codebase directly. See the [MCP Server](docs/guide/mcp-server.md) guide for setup.
-**For additional repositories (preserve existing data):**
-```bash
-cgr start --repo-path /path/to/repo2 --update-graph
-cgr start --repo-path /path/to/repo3 --update-graph
-```
+## Documentation
-**Control Memgraph batch flushing:**
-```bash
-# Flush every 5,000 records instead of the default from settings
-cgr start --repo-path /path/to/repo --update-graph \
- --batch-size 5000
-```
+**Getting Started**
+- [Installation](docs/getting-started/installation.md)
+- [Quick Start](docs/getting-started/quickstart.md)
+- [Configuration](docs/getting-started/configuration.md)
-The system automatically detects and processes files for all supported languages (see Multi-Language Support section).
+**User Guide**
+- [CLI Reference](docs/guide/cli-reference.md)
+- [Interactive Querying](docs/guide/interactive-querying.md)
+- [Code Optimisation](docs/guide/code-optimization.md)
+- [Dead Code Detection](docs/guide/dead-code.md)
+- [Graph Export](docs/guide/graph-export.md)
+- [Real-Time Updates](docs/guide/realtime-updates.md)
+- [MCP Server](docs/guide/mcp-server.md)
-### Step 2: Query the Codebase
+**Architecture**
+- [Overview](docs/architecture/overview.md)
+- [Graph Schema](docs/architecture/graph-schema.md)
+- [Language Support](docs/architecture/language-support.md)
+- [Data-Flow Edges](docs/architecture/data-flow-edges.md)
-Start the interactive RAG CLI:
+**Python SDK**
+- [Overview](docs/sdk/overview.md)
+- [Graph Loader](docs/sdk/graph-loader.md)
+- [Cypher Generator](docs/sdk/cypher-generator.md)
+- [Semantic Search](docs/sdk/semantic-search.md)
-```bash
-cgr start --repo-path /path/to/your/repo
-```
+**Advanced**
+- [Adding Languages](docs/advanced/adding-languages.md)
+- [Ignore Patterns](docs/advanced/ignore-patterns.md)
+- [Building Binaries](docs/advanced/building-binaries.md)
+- [Troubleshooting](docs/advanced/troubleshooting.md)
-### Step 2.5: Real-Time Graph Updates (Optional)
+## Enterprise Services
-For active development, you can keep your knowledge graph automatically synchronized with code changes using the realtime updater. This is particularly useful when you're actively modifying code and want the AI assistant to always work with the latest codebase structure.
-
-**What it does:**
-- Watches your repository for file changes (create, modify, delete)
-- Automatically updates the knowledge graph in real-time
-- Maintains consistency by recalculating all function call relationships
-- Filters out irrelevant files (`.git`, `node_modules`, etc.)
-
-**How to use:**
-
-Run the realtime updater in a separate terminal:
-
-```bash
-# Using Python directly
-python realtime_updater.py /path/to/your/repo
-
-# Or using the Makefile
-make watch REPO_PATH=/path/to/your/repo
-```
-
-**With custom Memgraph settings:**
-```bash
-# Python
-python realtime_updater.py /path/to/your/repo --host localhost --port 7687 --batch-size 1000
-
-# Makefile
-make watch REPO_PATH=/path/to/your/repo HOST=localhost PORT=7687 BATCH_SIZE=1000
-```
-
-**Multi-terminal workflow:**
-```bash
-# Terminal 1: Start the realtime updater
-python realtime_updater.py ~/my-project
-
-# Terminal 2: Run the AI assistant
-cgr start --repo-path ~/my-project
-```
-
-**Performance note:** The updater currently recalculates all CALLS relationships on every file change to ensure consistency. This prevents "island" problems where changes in one file aren't reflected in relationships from other files, but may impact performance on very large codebases with frequent changes. **Note:** Optimization of this behavior is a work in progress.
-
-**CLI Arguments:**
-- `repo_path` (required): Path to repository to watch
-- `--host`: Memgraph host (default: `localhost`)
-- `--port`: Memgraph port (default: `7687`)
-- `--batch-size`: Number of buffered nodes/relationships before flushing to Memgraph
-
-**Specify Custom Models:**
-```bash
-# Use specific local models
-cgr start --repo-path /path/to/your/repo \
- --orchestrator ollama:llama3.2 \
- --cypher ollama:codellama
-
-# Use specific Gemini models
-cgr start --repo-path /path/to/your/repo \
- --orchestrator google:gemini-2.0-flash-thinking-exp-01-21 \
- --cypher google:gemini-2.5-flash-lite-preview-06-17
-
-# Use mixed providers
-cgr start --repo-path /path/to/your/repo \
- --orchestrator google:gemini-2.0-flash-thinking-exp-01-21 \
- --cypher ollama:codellama
-```
-
-Example queries (works across all supported languages):
-- "Show me all classes that contain 'user' in their name"
-- "Find functions related to database operations"
-- "What methods does the User class have?"
-- "Show me functions that handle authentication"
-- "List all TypeScript components"
-- "Find Rust structs and their methods"
-- "Show me Go interfaces and implementations"
-- "Find all C++ operator overloads in the Matrix class"
-- "Show me C++ template functions with their specializations"
-- "List all C++ namespaces and their contained classes"
-- "Find C++ lambda expressions used in algorithms"
-- "Add logging to all database connection functions"
-- "Refactor the User class to use dependency injection"
-- "Convert these Python functions to async/await pattern"
-- "Add error handling to authentication methods"
-- "Optimize this function for better performance"
-
-### Step 3: Export Graph Data
-
-For programmatic access and integration with other tools, you can export the entire knowledge graph to JSON:
-
-**Export during graph update:**
-```bash
-cgr start --repo-path /path/to/repo --update-graph --clean -o my_graph.json
-```
-
-**Export existing graph without updating:**
-```bash
-cgr export -o my_graph.json
-```
-
-**Optional: adjust Memgraph batching during export:**
-```bash
-cgr export -o my_graph.json --batch-size 5000
-```
-
-**Working with exported data:**
-```python
-from codebase_rag.graph_loader import load_graph
-
-# Load the exported graph
-graph = load_graph("my_graph.json")
-
-# Get summary statistics
-summary = graph.summary()
-print(f"Total nodes: {summary['total_nodes']}")
-print(f"Total relationships: {summary['total_relationships']}")
-
-# Find specific node types
-functions = graph.find_nodes_by_label("Function")
-classes = graph.find_nodes_by_label("Class")
-
-# Analyze relationships
-for func in functions[:5]:
- relationships = graph.get_relationships_for_node(func.node_id)
- print(f"Function {func.properties['name']} has {len(relationships)} relationships")
-```
-
-**Example analysis script:**
-```bash
-python examples/graph_export_example.py my_graph.json
-```
-
-This provides a reliable, programmatic way to access your codebase structure without LLM restrictions, perfect for:
-- Integration with other tools
-- Custom analysis scripts
-- Building documentation generators
-- Creating code metrics dashboards
-
-### Step 4: Code Optimization
-
-For AI-powered codebase optimization with best practices guidance:
-
-**Basic optimization for a specific language:**
-```bash
-cgr optimize python --repo-path /path/to/your/repo
-```
-
-**Optimization with reference documentation:**
-```bash
-cgr optimize python \
- --repo-path /path/to/your/repo \
- --reference-document /path/to/best_practices.md
-```
-
-**Using specific models for optimization:**
-```bash
-cgr optimize javascript \
- --repo-path /path/to/frontend \
- --orchestrator google:gemini-2.0-flash-thinking-exp-01-21
-
-# Optional: override Memgraph batch flushing during optimization
-cgr optimize javascript --repo-path /path/to/frontend \
- --batch-size 5000
-```
+Code-Graph-RAG is open source and free to use. For organisations that need more, we offer **fully managed cloud-hosted solutions** and **on-premise deployments**:
-**Supported Languages for Optimization:**
-All supported languages: `python`, `javascript`, `typescript`, `rust`, `go`, `java`, `scala`, `cpp`
+- **Cloud-Hosted Deployment**: Managed cloud infrastructure for both the graph database and the AI agent connection. Zero infrastructure overhead, so we handle scaling, updates, and availability while your team focuses on building.
+- **On-Premise & Air-Gapped Deployment**: Deploy Code-Graph-RAG entirely within your own environment, including air-gapped networks. Full data sovereignty for regulated industries and security-sensitive organisations.
-**How It Works:**
-1. **Analysis Phase**: The agent analyzes your codebase structure using the knowledge graph
-2. **Pattern Recognition**: Identifies common anti-patterns, performance issues, and improvement opportunities
-3. **Best Practices Application**: Applies language-specific best practices and patterns
-4. **Interactive Approval**: Presents each optimization suggestion for your approval before implementation
-5. **Guided Implementation**: Implements approved changes with detailed explanations
-
-**Example Optimization Session:**
-```
-Starting python optimization session...
-┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
-┃ The agent will analyze your python codebase and propose specific ┃
-┃ optimizations. You'll be asked to approve each suggestion before ┃
-┃ implementation. Type 'exit' or 'quit' to end the session. ┃
-┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
-
-🔍 Analyzing codebase structure...
-📊 Found 23 Python modules with potential optimizations
-
-💡 Optimization Suggestion #1:
- File: src/data_processor.py
- Issue: Using list comprehension in a loop can be optimized
- Suggestion: Replace with generator expression for memory efficiency
-
- [y/n] Do you approve this optimization?
-```
-
-**Reference Document Support:**
-You can provide reference documentation (like coding standards, architectural guidelines, or best practices documents) to guide the optimization process:
-
-```bash
-# Use company coding standards
-cgr optimize python \
- --reference-document ./docs/coding_standards.md
-
-# Use architectural guidelines
-cgr optimize java \
- --reference-document ./ARCHITECTURE.md
-
-# Use performance best practices
-cgr optimize rust \
- --reference-document ./docs/performance_guide.md
-```
-
-The agent will incorporate the guidance from your reference documents when suggesting optimizations, ensuring they align with your project's standards and architectural decisions.
-
-**Common CLI Arguments:**
-- `--orchestrator`: Specify provider:model for main operations (e.g., `google:gemini-2.0-flash-thinking-exp-01-21`, `ollama:llama3.2`)
-- `--cypher`: Specify provider:model for graph queries (e.g., `google:gemini-2.5-flash-lite-preview-06-17`, `ollama:codellama`)
-- `--repo-path`: Path to repository (defaults to current directory)
-- `--batch-size`: Override Memgraph flush batch size (defaults to `MEMGRAPH_BATCH_SIZE` in settings)
-- `--reference-document`: Path to reference documentation (optimization only)
-
-## 🔌 MCP Server (Claude Code Integration)
-
-Code-Graph-RAG can run as an MCP (Model Context Protocol) server, enabling seamless integration with Claude Code and other MCP clients.
-
-### Quick Setup
-
-```bash
-claude mcp add --transport stdio code-graph-rag \
- --env TARGET_REPO_PATH=/absolute/path/to/your/project \
- --env CYPHER_PROVIDER=openai \
- --env CYPHER_MODEL=gpt-4 \
- --env CYPHER_API_KEY=your-api-key \
- -- uv run --directory /path/to/code-graph-rag code-graph-rag mcp-server
-```
-
-### Available Tools
-
-
-| Tool | Description |
-|----|-----------|
-| `list_projects` | List all indexed projects in the knowledge graph database. Returns a list of project names that have been indexed. |
-| `delete_project` | Delete a specific project from the knowledge graph database. This removes all nodes associated with the project while preserving other projects. Use list_projects first to see available projects. |
-| `wipe_database` | WARNING: Completely wipe the entire database, removing ALL indexed projects. This cannot be undone. Use delete_project for removing individual projects. |
-| `index_repository` | Parse and ingest the repository into the Memgraph knowledge graph. This builds a comprehensive graph of functions, classes, dependencies, and relationships. Note: This preserves other projects - only the current project is re-indexed. |
-| `query_code_graph` | Query the codebase knowledge graph using natural language. Ask questions like 'What functions call UserService.create_user?' or 'Show me all classes that implement the Repository interface'. |
-| `get_code_snippet` | Retrieve source code for a function, class, or method by its qualified name. Returns the source code, file path, line numbers, and docstring. |
-| `surgical_replace_code` | Surgically replace an exact code block in a file using diff-match-patch. Only modifies the exact target block, leaving the rest unchanged. |
-| `read_file` | Read the contents of a file from the project. Supports pagination for large files. |
-| `write_file` | Write content to a file, creating it if it doesn't exist. |
-| `list_directory` | List contents of a directory in the project. |
-
-
-### Example Usage
-
-```
-> Index this repository
-> What functions call UserService.create_user?
-> Update the login function to add rate limiting
-```
-
-For detailed setup, see [Claude Code Setup Guide](docs/claude-code-setup.md).
-
-## 📊 Graph Schema
-
-The knowledge graph uses the following node types and relationships:
-
-### Node Types
-
-
-| Label | Properties |
-|-----|----------|
-| Project | `{name: string}` |
-| Package | `{qualified_name: string, name: string, path: string}` |
-| Folder | `{path: string, name: string}` |
-| File | `{path: string, name: string, extension: string}` |
-| Module | `{qualified_name: string, name: string, path: string}` |
-| Class | `{qualified_name: string, name: string, decorators: list[string]}` |
-| Function | `{qualified_name: string, name: string, decorators: list[string]}` |
-| Method | `{qualified_name: string, name: string, decorators: list[string]}` |
-| Interface | `{qualified_name: string, name: string}` |
-| Enum | `{qualified_name: string, name: string}` |
-| Type | `{qualified_name: string, name: string}` |
-| Union | `{qualified_name: string, name: string}` |
-| ModuleInterface | `{qualified_name: string, name: string, path: string}` |
-| ModuleImplementation | `{qualified_name: string, name: string, path: string, implements_module: string}` |
-| ExternalPackage | `{name: string, version_spec: string}` |
-
-
-### Language-Specific Mappings
-
-
-- **C++**: `class_specifier`, `declaration`, `enum_specifier`, `field_declaration`, `function_definition`, `lambda_expression`, `struct_specifier`, `template_declaration`, `union_specifier`
-- **Java**: `annotation_type_declaration`, `class_declaration`, `constructor_declaration`, `enum_declaration`, `interface_declaration`, `method_declaration`, `record_declaration`
-- **JavaScript**: `arrow_function`, `class`, `class_declaration`, `function_declaration`, `function_expression`, `generator_function_declaration`, `method_definition`
-- **Lua**: `function_declaration`, `function_definition`
-- **Python**: `class_definition`, `function_definition`
-- **Rust**: `closure_expression`, `enum_item`, `function_item`, `function_signature_item`, `impl_item`, `struct_item`, `trait_item`, `type_item`, `union_item`
-- **TypeScript**: `abstract_class_declaration`, `arrow_function`, `class`, `class_declaration`, `enum_declaration`, `function_declaration`, `function_expression`, `function_signature`, `generator_function_declaration`, `interface_declaration`, `internal_module`, `method_definition`, `type_alias_declaration`
-- **C#**: `anonymous_method_expression`, `class_declaration`, `constructor_declaration`, `destructor_declaration`, `enum_declaration`, `function_pointer_type`, `interface_declaration`, `lambda_expression`, `local_function_statement`, `method_declaration`, `struct_declaration`
-- **Go**: `function_declaration`, `method_declaration`, `type_declaration`
-- **PHP**: `anonymous_function`, `arrow_function`, `class_declaration`, `enum_declaration`, `function_definition`, `function_static_declaration`, `interface_declaration`, `trait_declaration`
-- **Scala**: `class_definition`, `function_declaration`, `function_definition`, `object_definition`, `trait_definition`
-
-
-### Relationships
-
-
-| Source | Relationship | Target |
-|------|------------|------|
-| Project, Package, Folder | CONTAINS_PACKAGE | Package |
-| Project, Package, Folder | CONTAINS_FOLDER | Folder |
-| Project, Package, Folder | CONTAINS_FILE | File |
-| Project, Package, Folder | CONTAINS_MODULE | Module |
-| Module | DEFINES | Class, Function |
-| Class | DEFINES_METHOD | Method |
-| Module | IMPORTS | Module |
-| Module | EXPORTS | Class, Function |
-| Module | EXPORTS_MODULE | ModuleInterface |
-| Module | IMPLEMENTS_MODULE | ModuleImplementation |
-| Class | INHERITS | Class |
-| Class | IMPLEMENTS | Interface |
-| Method | OVERRIDES | Method |
-| ModuleImplementation | IMPLEMENTS | ModuleInterface |
-| Project | DEPENDS_ON_EXTERNAL | ExternalPackage |
-| Function, Method | CALLS | Function, Method |
-
-
-## 🔧 Configuration
-
-Configuration is managed through environment variables in `.env` file:
-
-### Provider-Specific Settings
-
-#### Orchestrator Model Configuration
-- `ORCHESTRATOR_PROVIDER`: Provider name (`google`, `openai`, `ollama`)
-- `ORCHESTRATOR_MODEL`: Model ID (e.g., `gemini-2.5-pro`, `gpt-4o`, `llama3.2`)
-- `ORCHESTRATOR_API_KEY`: API key for the provider (if required)
-- `ORCHESTRATOR_ENDPOINT`: Custom endpoint URL (if required)
-- `ORCHESTRATOR_PROJECT_ID`: Google Cloud project ID (for Vertex AI)
-- `ORCHESTRATOR_REGION`: Google Cloud region (default: `us-central1`)
-- `ORCHESTRATOR_PROVIDER_TYPE`: Google provider type (`gla` or `vertex`)
-- `ORCHESTRATOR_THINKING_BUDGET`: Thinking budget for reasoning models
-- `ORCHESTRATOR_SERVICE_ACCOUNT_FILE`: Path to service account file (for Vertex AI)
-
-#### Cypher Model Configuration
-- `CYPHER_PROVIDER`: Provider name (`google`, `openai`, `ollama`)
-- `CYPHER_MODEL`: Model ID (e.g., `gemini-2.5-flash`, `gpt-4o-mini`, `codellama`)
-- `CYPHER_API_KEY`: API key for the provider (if required)
-- `CYPHER_ENDPOINT`: Custom endpoint URL (if required)
-- `CYPHER_PROJECT_ID`: Google Cloud project ID (for Vertex AI)
-- `CYPHER_REGION`: Google Cloud region (default: `us-central1`)
-- `CYPHER_PROVIDER_TYPE`: Google provider type (`gla` or `vertex`)
-- `CYPHER_THINKING_BUDGET`: Thinking budget for reasoning models
-- `CYPHER_SERVICE_ACCOUNT_FILE`: Path to service account file (for Vertex AI)
-
-### System Settings
-- `MEMGRAPH_HOST`: Memgraph hostname (default: `localhost`)
-- `MEMGRAPH_PORT`: Memgraph port (default: `7687`)
-- `MEMGRAPH_HTTP_PORT`: Memgraph HTTP port (default: `7444`)
-- `LAB_PORT`: Memgraph Lab port (default: `3000`)
-- `MEMGRAPH_BATCH_SIZE`: Batch size for Memgraph operations (default: `1000`)
-- `TARGET_REPO_PATH`: Default repository path (default: `.`)
-- `LOCAL_MODEL_ENDPOINT`: Fallback endpoint for Ollama (default: `http://localhost:11434/v1`)
-
-### Custom Ignore Patterns
-
-You can specify additional directories to exclude by creating a `.cgrignore` file in your repository root:
-
-```
-# Comments start with #
-vendor
-.custom_cache
-my_build_output
-```
-
-- One directory name per line
-- Lines starting with `#` are comments
-- Blank lines are ignored
-- Patterns are exact directory name matches (not globs)
-- Patterns from `.cgrignore` are merged with `--exclude` flags and auto-detected directories
-
-### Key Dependencies
-
-
-- **loguru**: Python logging made (stupidly) simple
-- **mcp**: Model Context Protocol SDK
-- **pydantic-ai**: Agent Framework / shim to use Pydantic with LLMs
-- **pydantic-settings**: Settings management using Pydantic
-- **pymgclient**: Memgraph database adapter for Python language
-- **python-dotenv**: Read key-value pairs from a .env file and set them as environment variables
-- **toml**: Python Library for Tom's Obvious, Minimal Language
-- **tree-sitter-python**: Python grammar for tree-sitter
-- **tree-sitter**: Python bindings to the Tree-sitter parsing library
-- **watchdog**: Filesystem events monitoring
-- **typer**: Typer, build great CLIs. Easy to code. Based on Python type hints.
-- **rich**: Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal
-- **prompt-toolkit**: Library for building powerful interactive command lines in Python
-- **diff-match-patch**: Repackaging of Google's Diff Match and Patch libraries.
-- **click**: Composable command line interface toolkit
-- **protobuf**
-- **defusedxml**: XML bomb protection for Python stdlib modules
-- **huggingface-hub**: Client library to download and publish models, datasets and other repos on the huggingface.co hub
-
-
-## 🤖 Agentic Workflow & Tools
-
-The agent is designed with a deliberate workflow to ensure it acts with context and precision, especially when modifying the file system.
-
-### Core Tools
-
-The agent has access to a suite of tools to understand and interact with the codebase:
-
-
-| Tool | Description |
-|----|-----------|
-| `query_graph` | Query the codebase knowledge graph using natural language questions. Ask in plain English about classes, functions, methods, dependencies, or code structure. Examples: 'Find all functions that call each other', 'What classes are in the user module', 'Show me functions with the longest call chains'. |
-| `read_file` | Reads the content of text-based files. For documents like PDFs or images, use the 'analyze_document' tool instead. |
-| `create_file` | Creates a new file with content. IMPORTANT: Check file existence first! Overwrites completely WITHOUT showing diff. Use only for new files, not existing file modifications. |
-| `replace_code` | Surgically replaces specific code blocks in files. Requires exact target code and replacement. Only modifies the specified block, leaving rest of file unchanged. True surgical patching. |
-| `list_directory` | Lists the contents of a directory to explore the codebase. |
-| `analyze_document` | Analyzes documents (PDFs, images) to answer questions about their content. |
-| `execute_shell` | Executes shell commands from allowlist. Read-only commands run without approval; write operations require user confirmation. |
-| `semantic_search` | Performs a semantic search for functions based on a natural language query describing their purpose, returning a list of potential matches with similarity scores. |
-| `get_function_source` | Retrieves the source code for a specific function or method using its internal node ID, typically obtained from a semantic search result. |
-| `get_code_snippet` | Retrieves the source code for a specific function, class, or method using its full qualified name. |
-
-
-### Intelligent and Safe File Editing
-
-The agent uses AST-based function targeting with Tree-sitter for precise code modifications. Features include:
-- **Visual diff preview** before changes
-- **Surgical patching** that only modifies target code blocks
-- **Multi-language support** across all supported languages
-- **Security sandbox** preventing edits outside project directory
-- **Smart function matching** with qualified names and line numbers
-
-
-
-## 🌍 Multi-Language Support
-
-### Adding New Languages
-
-Code-Graph-RAG makes it easy to add support for any language that has a Tree-sitter grammar. The system automatically handles grammar compilation and integration.
-
-> **⚠️ Recommendation**: While you can add languages yourself, we recommend waiting for official full support to ensure optimal parsing quality, comprehensive feature coverage, and robust integration. The languages marked as "In Development" above will receive dedicated optimization and testing.
-
-> **💡 Request Support**: If you want a specific language to be officially supported, please [submit an issue](https://github.com/vitali87/code-graph-rag/issues) with your language request.
-
-#### Quick Start: Add a Language
-
-Use the built-in language management tool to add any Tree-sitter supported language:
-
-```bash
-# Add a language using the standard tree-sitter repository
-cgr language add-grammar
-
-# Examples:
-cgr language add-grammar c-sharp
-cgr language add-grammar php
-cgr language add-grammar ruby
-cgr language add-grammar kotlin
-```
-
-#### Custom Grammar Repositories
-
-For languages hosted outside the standard tree-sitter organization:
-
-```bash
-# Add a language from a custom repository
-cgr language add-grammar --grammar-url https://github.com/custom/tree-sitter-mylang
-```
-
-#### What Happens Automatically
-
-When you add a language, the tool automatically:
-
-1. **Downloads the Grammar**: Clones the tree-sitter grammar repository as a git submodule
-2. **Detects Configuration**: Auto-extracts language metadata from `tree-sitter.json`
-3. **Analyzes Node Types**: Automatically identifies AST node types for:
- - Functions/methods (`method_declaration`, `function_definition`, etc.)
- - Classes/structs (`class_declaration`, `struct_declaration`, etc.)
- - Modules/files (`compilation_unit`, `source_file`, etc.)
- - Function calls (`call_expression`, `method_invocation`, etc.)
-4. **Compiles Bindings**: Builds Python bindings from the grammar source
-5. **Updates Configuration**: Adds the language to `codebase_rag/language_config.py`
-6. **Enables Parsing**: Makes the language immediately available for codebase analysis
-
-#### Example: Adding C# Support
-
-```bash
-$ cgr language add-grammar c-sharp
-🔍 Using default tree-sitter URL: https://github.com/tree-sitter/tree-sitter-c-sharp
-🔄 Adding submodule from https://github.com/tree-sitter/tree-sitter-c-sharp...
-✅ Successfully added submodule at grammars/tree-sitter-c-sharp
-Auto-detected language: c-sharp
-Auto-detected file extensions: ['cs']
-Auto-detected node types:
-Functions: ['destructor_declaration', 'method_declaration', 'constructor_declaration']
-Classes: ['struct_declaration', 'enum_declaration', 'interface_declaration', 'class_declaration']
-Modules: ['compilation_unit', 'file_scoped_namespace_declaration', 'namespace_declaration']
-Calls: ['invocation_expression']
-
-✅ Language 'c-sharp' has been added to the configuration!
-📝 Updated codebase_rag/language_config.py
-```
-
-#### Managing Languages
-
-```bash
-# List all configured languages
-cgr language list-languages
-
-# Remove a language (this also removes the git submodule unless --keep-submodule is specified)
-cgr language remove-language
-```
-
-#### Language Configuration
-
-The system uses a configuration-driven approach for language support. Each language is defined in `codebase_rag/language_config.py` with the following structure:
-
-```python
-"language-name": LanguageConfig(
- name="language-name",
- file_extensions=[".ext1", ".ext2"],
- function_node_types=["function_declaration", "method_declaration"],
- class_node_types=["class_declaration", "struct_declaration"],
- module_node_types=["compilation_unit", "source_file"],
- call_node_types=["call_expression", "method_invocation"],
-),
-```
-
-#### Troubleshooting
-
-**Grammar not found**: If the automatic URL doesn't work, use a custom URL:
-```bash
-cgr language add-grammar --grammar-url https://github.com/custom/tree-sitter-mylang
-```
-
-**Version incompatibility**: If you get "Incompatible Language version" errors, update your tree-sitter package:
-```bash
-uv add tree-sitter@latest
-```
-
-**Missing node types**: The tool automatically detects common node patterns, but you can manually adjust the configuration in `language_config.py` if needed.
-
-## 📦 Building a binary
-
-You can build a binary of the application using the `build_binary.py` script. This script uses PyInstaller to package the application and its dependencies into a single executable.
-
-```bash
-python build_binary.py
-```
-The resulting binary will be located in the `dist` directory.
-
-## 🐛 Debugging
-
-1. **Check Memgraph connection**:
- - Ensure Docker containers are running: `docker-compose ps`
- - Verify Memgraph is accessible on port 7687
-
-2. **View database in Memgraph Lab**:
- - Open http://localhost:3000
- - Connect to memgraph:7687
-
-3. **For local models**:
- - Verify Ollama is running: `ollama list`
- - Check if models are downloaded: `ollama pull llama3`
- - Test Ollama API: `curl http://localhost:11434/v1/models`
- - Check Ollama logs: `ollama logs`
-
-## 🤝 Contributing
-
-Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed contribution guidelines.
-
-Good first PRs are from TODO issues.
-
-## 🙋♂️ Support
-
-For issues or questions:
-1. Check the logs for error details
-2. Verify Memgraph connection
-3. Ensure all environment variables are set
-4. Review the graph schema matches your expectations
+We also offer custom development, integration consulting, technical support contracts, and team training.
-## 💼 Enterprise Services
+**[View plans & pricing at code-graph-rag.com](https://code-graph-rag.com/enterprise)**
-Code-Graph-RAG is open source and free to use. For organizations that need more, we offer **fully managed cloud-hosted solutions** and **on-premise deployments**:
+## Contributing
-- **Cloud-Hosted Deployment** — Managed cloud infrastructure for both the graph database and AI agent connection. Zero infrastructure overhead — we handle scaling, updates, and availability so your team can focus on building.
-- **On-Premise & Air-Gapped Deployment** — Deploy Code-Graph-RAG entirely within your own environment, including air-gapped networks. Full data sovereignty for regulated industries and security-sensitive organizations.
+Please see [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines. Good first PRs come from the TODO issues.
-We also offer custom development, integration consulting, technical support contracts, and team training.
+## Support
-**[View plans & pricing at code-graph-rag.com →](https://code-graph-rag.com/enterprise)**
+For issues or questions, check the [Troubleshooting](docs/advanced/troubleshooting.md) guide first, then open an issue.
-## Star History
+## License
-[](https://www.star-history.com/#vitali87/code-graph-rag&Date)
+MIT. See [LICENSE](LICENSE).
diff --git a/TODO.md b/TODO.md
deleted file mode 100644
index 7007debed..000000000
--- a/TODO.md
+++ /dev/null
@@ -1,145 +0,0 @@
-# Type Inference Gaps
-
-## Python
-
-### 1. Parse Return Type Annotations
-
-```python
-def get_all_users() -> list[User]: # Not parsed
- ...
-```
-
-Tree-sitter: `function_definition` → `return_type` field.
-
-### 2. Extract Element Type from Generics
-
-```python
-users = get_all_users() # type: list[User]
-for user in users: # user should be: User
- user.save() # resolves to: User.save()
-```
-
-Regex: `list[X]` → `X`, `dict[K, V]` → `V`, `Optional[X]` → `X`
-
----
-
-## TypeScript
-
-### 1. Parse Type Annotations
-
-```typescript
-const users: User[] = getUsers(); // Not parsed
-function getUsers(): User[] { ... } // Not parsed
-```
-
-Tree-sitter: `variable_declarator` → `type` field, `function_declaration` → `return_type` field.
-
-### 2. Handle For-Of Loops
-
-```typescript
-for (const user of users) { // Not handled
- user.save();
-}
-```
-
-Tree-sitter: `for_in_statement` with `of` variant.
-
-### 3. Extract Element Type from Array/Generic Types
-
-```typescript
-User[] → User
-Array → User
-```
-
----
-
-## JavaScript
-
-### 1. Handle For-Of Loops
-
-```javascript
-for (const user of users) { // Not handled
- user.save();
-}
-```
-
-Same as TypeScript - no loop variable inference.
-
----
-
-## Java
-
-### 1. Extract Element Type from Generics
-
-```java
-List users = getUsers();
-users.get(0).save(); // Doesn't know get(0) returns User
-```
-
-Parse `List` → extract `User` for collection method return types.
-
-### 2. Handle `var` Type Inference
-
-```java
-var users = getUsers(); // type is List, needs inference
-var user = users.get(0); // type is User, needs inference
-```
-
-When `var` is used, infer from right-hand side expression.
-
----
-
-## C++
-
-### No Type Extraction Engine
-
-Currently returns empty `local_var_types`. Need to extract explicit types from declarations.
-
-```cpp
-User user = getUser(); // Extract "User" from declaration
-user.save(); // Resolve to User::save()
-
-auto user = getUser(); // Needs return type inference (like Java var)
-```
-
-Tree-sitter: `declaration` → `type` field, `init_declarator` → `declarator` field.
-
----
-
-## Rust
-
-### No Type Extraction Engine
-
-Currently returns empty `local_var_types`. Need to extract explicit types from declarations.
-
-```rust
-let user: User = get_user(); // Extract "User" from type annotation
-user.save(); // Resolve to User::save()
-
-let user = get_user(); // Needs return type inference
-```
-
-Tree-sitter: `let_declaration` → `type` field.
-
----
-
-## Go
-
-### No Type Extraction Engine
-
-Currently returns empty `local_var_types`. Need to extract explicit types from declarations.
-
-```go
-var user User = getUser() // Extract "User" from declaration
-user.Save() // Resolve to User.Save()
-
-user := getUser() // Short declaration - needs return type inference
-```
-
-Tree-sitter: `var_declaration` → type identifier, `short_var_declaration` needs inference.
-
----
-
-## Lua
-
-No type annotations exist. Current coverage is sufficient for dynamic language.
diff --git a/benchmarks/bench_ast_cache.py b/benchmarks/bench_ast_cache.py
new file mode 100644
index 000000000..b1e3e65d9
--- /dev/null
+++ b/benchmarks/bench_ast_cache.py
@@ -0,0 +1,134 @@
+import statistics
+import sys
+import time
+from collections import OrderedDict
+from pathlib import Path
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 50
+
+
+class MockNode:
+ __slots__ = ("data",)
+
+ def __init__(self, size: int) -> None:
+ self.data = b"\x00" * size
+
+
+def bench_ordered_dict_insert(count: int, item_size: int) -> float:
+ start = time.perf_counter()
+ cache: OrderedDict[Path, tuple[MockNode, str]] = OrderedDict()
+ for i in range(count):
+ key = Path(f"/fake/path/module_{i}.py")
+ cache[key] = (MockNode(item_size), "python")
+ return time.perf_counter() - start
+
+
+def bench_ordered_dict_lookup(cache: OrderedDict, keys: list[Path]) -> float:
+ start = time.perf_counter()
+ for key in keys:
+ _ = key in cache
+ return time.perf_counter() - start
+
+
+def bench_ordered_dict_access_lru(cache: OrderedDict, keys: list[Path]) -> float:
+ start = time.perf_counter()
+ for key in keys:
+ if key in cache:
+ cache.move_to_end(key)
+ _ = cache[key]
+ return time.perf_counter() - start
+
+
+def bench_ordered_dict_eviction(count: int, max_size: int, item_size: int) -> float:
+ start = time.perf_counter()
+ cache: OrderedDict[Path, tuple[MockNode, str]] = OrderedDict()
+ for i in range(count):
+ key = Path(f"/fake/path/module_{i}.py")
+ cache[key] = (MockNode(item_size), "python")
+ while len(cache) > max_size:
+ cache.popitem(last=False)
+ return time.perf_counter() - start
+
+
+def bench_getsizeof_overhead(cache: OrderedDict) -> float:
+ start = time.perf_counter()
+ _ = sum(sys.getsizeof(v) for v in cache.values())
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<45} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 115)
+ for r in results:
+ print(
+ f"{r['name']:<45} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ configs = [
+ (500, 1024),
+ (2000, 4096),
+ (5000, 8192),
+ ]
+
+ for count, item_size in configs:
+ print(f"\n{'='*115}")
+ print(f"BoundedASTCache Benchmark (entries={count}, item_size={item_size}B)")
+ print(f"{'='*115}")
+
+ results = []
+
+ r = run_benchmark(f"insert ({count})", bench_ordered_dict_insert, count, item_size)
+ results.append(r)
+
+ cache: OrderedDict[Path, tuple[MockNode, str]] = OrderedDict()
+ keys: list[Path] = []
+ for i in range(count):
+ key = Path(f"/fake/path/module_{i}.py")
+ keys.append(key)
+ cache[key] = (MockNode(item_size), "python")
+
+ r = run_benchmark(f"lookup ({count})", bench_ordered_dict_lookup, cache, keys)
+ results.append(r)
+
+ r = run_benchmark(f"access+LRU ({count})", bench_ordered_dict_access_lru, cache, keys)
+ results.append(r)
+
+ max_size = count // 2
+ r = run_benchmark(
+ f"insert+evict (max={max_size})",
+ bench_ordered_dict_eviction, count, max_size, item_size,
+ )
+ results.append(r)
+
+ r = run_benchmark(f"getsizeof scan ({count})", bench_getsizeof_overhead, cache)
+ results.append(r)
+
+ print_results(results)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_dropin_replacements.py b/benchmarks/bench_dropin_replacements.py
new file mode 100644
index 000000000..ee4eb0b0a
--- /dev/null
+++ b/benchmarks/bench_dropin_replacements.py
@@ -0,0 +1,267 @@
+import hashlib
+import json
+import os
+import statistics
+import tempfile
+import time
+from pathlib import Path
+
+try:
+ import blake3
+ import orjson
+except ImportError as e:
+ print(f"SKIP bench_dropin_replacements: {e}")
+ print("Install with: uv pip install blake3 orjson")
+ raise SystemExit(0)
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 30
+
+
+def generate_graph_data(num_nodes: int, num_rels: int) -> dict:
+ nodes = []
+ for i in range(num_nodes):
+ nodes.append({
+ "node_id": i,
+ "labels": ["Function" if i % 3 == 0 else "Class" if i % 3 == 1 else "Module"],
+ "properties": {
+ "qualified_name": f"project.module{i // 100}.Class{i // 10}.method{i}",
+ "name": f"method{i}",
+ "start_line": i * 10,
+ "end_line": i * 10 + 9,
+ "docstring": f"Method {i} documentation string with some content" if i % 5 == 0 else None,
+ "decorators": ["staticmethod"] if i % 7 == 0 else [],
+ "is_exported": i % 4 == 0,
+ },
+ })
+
+ rels = []
+ for i in range(num_rels):
+ rels.append({
+ "from_id": i % num_nodes,
+ "to_id": (i * 7 + 3) % num_nodes,
+ "type": "CALLS" if i % 3 == 0 else "DEFINES" if i % 3 == 1 else "IMPORTS",
+ "properties": {"weight": i % 10} if i % 5 == 0 else {},
+ })
+
+ return {
+ "nodes": nodes,
+ "relationships": rels,
+ "metadata": {
+ "total_nodes": num_nodes,
+ "total_relationships": num_rels,
+ "exported_at": "2026-03-14T10:00:00+00:00",
+ },
+ }
+
+
+def generate_snippets(count: int, avg_length: int = 200) -> list[str]:
+ import random
+ import string
+ random.seed(42)
+ snippets = []
+ for _ in range(count):
+ length = avg_length + random.randint(-50, 50)
+ snippet = "".join(random.choices(string.ascii_letters + string.digits + " \n\t", k=length))
+ snippets.append(snippet)
+ return snippets
+
+
+def create_test_files(directory: str, count: int, avg_size_kb: int) -> list[Path]:
+ paths = []
+ for i in range(count):
+ path = Path(directory) / f"file_{i}.py"
+ content = os.urandom(avg_size_kb * 1024)
+ path.write_bytes(content)
+ paths.append(path)
+ return paths
+
+
+def bench_json_dumps(data: dict) -> float:
+ start = time.perf_counter()
+ _ = json.dumps(data)
+ return time.perf_counter() - start
+
+
+def bench_orjson_dumps(data: dict) -> float:
+ start = time.perf_counter()
+ _ = orjson.dumps(data)
+ return time.perf_counter() - start
+
+
+def bench_json_dumps_indent(data: dict) -> float:
+ start = time.perf_counter()
+ _ = json.dumps(data, indent=2, ensure_ascii=False)
+ return time.perf_counter() - start
+
+
+def bench_orjson_dumps_indent(data: dict) -> float:
+ start = time.perf_counter()
+ _ = orjson.dumps(data, option=orjson.OPT_INDENT_2)
+ return time.perf_counter() - start
+
+
+def bench_json_loads(json_bytes: bytes) -> float:
+ start = time.perf_counter()
+ _ = json.loads(json_bytes)
+ return time.perf_counter() - start
+
+
+def bench_orjson_loads(json_bytes: bytes) -> float:
+ start = time.perf_counter()
+ _ = orjson.loads(json_bytes)
+ return time.perf_counter() - start
+
+
+def bench_sha256_hashing(snippets: list[str]) -> float:
+ start = time.perf_counter()
+ for s in snippets:
+ _ = hashlib.sha256(s.encode()).hexdigest()
+ return time.perf_counter() - start
+
+
+def bench_blake3_hashing(snippets: list[str]) -> float:
+ start = time.perf_counter()
+ for s in snippets:
+ _ = blake3.blake3(s.encode()).hexdigest()
+ return time.perf_counter() - start
+
+
+def bench_sha256_file(files: list[Path]) -> float:
+ start = time.perf_counter()
+ for f in files:
+ hasher = hashlib.sha256()
+ with f.open("rb") as fh:
+ while chunk := fh.read(8192):
+ hasher.update(chunk)
+ _ = hasher.hexdigest()
+ return time.perf_counter() - start
+
+
+def bench_blake3_file(files: list[Path]) -> float:
+ start = time.perf_counter()
+ for f in files:
+ hasher = blake3.blake3()
+ with f.open("rb") as fh:
+ while chunk := fh.read(8192):
+ hasher.update(chunk)
+ _ = hasher.hexdigest()
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<50} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 120)
+ for r in results:
+ print(
+ f"{r['name']:<50} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def print_comparison(baseline: dict[str, float], optimized: dict[str, float]) -> None:
+ speedup = baseline["median_ms"] / optimized["median_ms"] if optimized["median_ms"] > 0 else float("inf")
+ print(f" -> Speedup: {speedup:.1f}x (median)")
+
+
+def main() -> None:
+ print("=" * 120)
+ print("DROP-IN REPLACEMENT BENCHMARKS: Python stdlib vs Rust-backed alternatives")
+ print("=" * 120)
+
+ # --- JSON Serialization ---
+ for num_nodes, num_rels in [(1000, 2000), (5000, 10000), (20000, 50000)]:
+ print(f"\n{'='*120}")
+ print(f"JSON Serialization: stdlib json vs orjson (nodes={num_nodes}, rels={num_rels})")
+ print(f"{'='*120}")
+
+ data = generate_graph_data(num_nodes, num_rels)
+ json_bytes = json.dumps(data).encode()
+ orjson_bytes = orjson.dumps(data)
+ print(f"Data size: {len(json_bytes) / 1024:.1f} KB")
+
+ results = []
+
+ r1 = run_benchmark(f"json.dumps compact ({num_nodes}n)", bench_json_dumps, data)
+ results.append(r1)
+ r2 = run_benchmark(f"orjson.dumps compact ({num_nodes}n)", bench_orjson_dumps, data)
+ results.append(r2)
+
+ r3 = run_benchmark(f"json.dumps indented ({num_nodes}n)", bench_json_dumps_indent, data)
+ results.append(r3)
+ r4 = run_benchmark(f"orjson.dumps indented ({num_nodes}n)", bench_orjson_dumps_indent, data)
+ results.append(r4)
+
+ r5 = run_benchmark(f"json.loads ({num_nodes}n)", bench_json_loads, json_bytes)
+ results.append(r5)
+ r6 = run_benchmark(f"orjson.loads ({num_nodes}n)", bench_orjson_loads, orjson_bytes)
+ results.append(r6)
+
+ print_results(results)
+
+ print("\nSpeedups:")
+ print(f" dumps compact: {r1['median_ms'] / r2['median_ms']:.1f}x")
+ print(f" dumps indented: {r3['median_ms'] / r4['median_ms']:.1f}x")
+ print(f" loads: {r5['median_ms'] / r6['median_ms']:.1f}x")
+
+ # --- Hashing: SHA256 vs BLAKE3 ---
+ print(f"\n\n{'='*120}")
+ print("Hashing: hashlib.sha256 vs blake3 (snippet hashing for EmbeddingCache)")
+ print(f"{'='*120}")
+
+ for size in [500, 2000, 10000]:
+ snippets = generate_snippets(size)
+ print(f"\n--- Snippet count: {size} ---")
+
+ results = []
+ r1 = run_benchmark(f"hashlib.sha256 ({size} snippets)", bench_sha256_hashing, snippets)
+ results.append(r1)
+ r2 = run_benchmark(f"blake3 ({size} snippets)", bench_blake3_hashing, snippets)
+ results.append(r2)
+
+ print_results(results)
+ print(f" Speedup: {r1['median_ms'] / r2['median_ms']:.1f}x")
+
+ # --- File Hashing ---
+ print(f"\n\n{'='*120}")
+ print("File Hashing: SHA256 vs BLAKE3 (incremental build file change detection)")
+ print(f"{'='*120}")
+
+ for file_count, avg_size_kb in [(50, 5), (200, 10), (500, 20)]:
+ with tempfile.TemporaryDirectory() as tmpdir:
+ files = create_test_files(tmpdir, file_count, avg_size_kb)
+ total_mb = sum(f.stat().st_size for f in files) / (1024 * 1024)
+ print(f"\n--- Files: {file_count}, Total: {total_mb:.1f} MB ---")
+
+ results = []
+ r1 = run_benchmark(f"sha256 ({file_count}f, {avg_size_kb}KB avg)", bench_sha256_file, files)
+ results.append(r1)
+ r2 = run_benchmark(f"blake3 ({file_count}f, {avg_size_kb}KB avg)", bench_blake3_file, files)
+ results.append(r2)
+
+ print_results(results)
+ print(f" Speedup: {r1['median_ms'] / r2['median_ms']:.1f}x")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_embedding_cache.py b/benchmarks/bench_embedding_cache.py
new file mode 100644
index 000000000..b63e93338
--- /dev/null
+++ b/benchmarks/bench_embedding_cache.py
@@ -0,0 +1,130 @@
+import hashlib
+import random
+import statistics
+import string
+import time
+
+from codebase_rag.embedder import EmbeddingCache
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 50
+EMBEDDING_DIM = 768
+
+
+def generate_snippets(count: int, avg_length: int = 200) -> list[str]:
+ snippets = []
+ for i in range(count):
+ length = avg_length + random.randint(-50, 50)
+ snippet = "".join(random.choices(string.ascii_letters + string.digits + " \n\t", k=length))
+ snippets.append(snippet)
+ return snippets
+
+
+def generate_embedding() -> list[float]:
+ return [random.random() for _ in range(EMBEDDING_DIM)]
+
+
+def bench_sha256_hashing(snippets: list[str]) -> float:
+ start = time.perf_counter()
+ for s in snippets:
+ _ = hashlib.sha256(s.encode()).hexdigest()
+ return time.perf_counter() - start
+
+
+def bench_cache_put(cache: EmbeddingCache, snippets: list[str], embeddings: list[list[float]]) -> float:
+ start = time.perf_counter()
+ for s, e in zip(snippets, embeddings):
+ cache.put(s, e)
+ return time.perf_counter() - start
+
+
+def bench_cache_get_hit(cache: EmbeddingCache, snippets: list[str]) -> float:
+ start = time.perf_counter()
+ for s in snippets:
+ _ = cache.get(s)
+ return time.perf_counter() - start
+
+
+def bench_cache_get_miss(cache: EmbeddingCache, miss_snippets: list[str]) -> float:
+ start = time.perf_counter()
+ for s in miss_snippets:
+ _ = cache.get(s)
+ return time.perf_counter() - start
+
+
+def bench_cache_get_many(cache: EmbeddingCache, snippets: list[str]) -> float:
+ start = time.perf_counter()
+ _ = cache.get_many(snippets)
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<40} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 110)
+ for r in results:
+ print(
+ f"{r['name']:<40} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ random.seed(42)
+
+ sizes = [500, 2000, 10000]
+
+ for size in sizes:
+ print(f"\n{'='*110}")
+ print(f"EmbeddingCache Benchmark (n={size})")
+ print(f"{'='*110}")
+
+ snippets = generate_snippets(size)
+ embeddings = [generate_embedding() for _ in range(size)]
+ miss_snippets = generate_snippets(size, avg_length=300)
+
+ results = []
+
+ r = run_benchmark(f"sha256 hashing ({size})", bench_sha256_hashing, snippets)
+ results.append(r)
+
+ cache = EmbeddingCache()
+ r = run_benchmark(f"cache.put ({size})", bench_cache_put, cache, snippets, embeddings)
+ results.append(r)
+
+ cache = EmbeddingCache()
+ cache.put_many(snippets, embeddings)
+
+ r = run_benchmark(f"cache.get hit ({size})", bench_cache_get_hit, cache, snippets)
+ results.append(r)
+
+ r = run_benchmark(f"cache.get miss ({size})", bench_cache_get_miss, cache, miss_snippets)
+ results.append(r)
+
+ r = run_benchmark(f"cache.get_many ({size})", bench_cache_get_many, cache, snippets)
+ results.append(r)
+
+ print_results(results)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_file_hashing.py b/benchmarks/bench_file_hashing.py
new file mode 100644
index 000000000..3be76059b
--- /dev/null
+++ b/benchmarks/bench_file_hashing.py
@@ -0,0 +1,138 @@
+import hashlib
+import os
+import statistics
+import tempfile
+import time
+from pathlib import Path
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 30
+
+
+def create_test_files(directory: str, count: int, avg_size_kb: int) -> list[Path]:
+ paths = []
+ for i in range(count):
+ path = Path(directory) / f"file_{i}.py"
+ content = os.urandom(avg_size_kb * 1024)
+ path.write_bytes(content)
+ paths.append(path)
+ return paths
+
+
+def hash_file_sha256(filepath: Path) -> str:
+ hasher = hashlib.sha256()
+ with filepath.open("rb") as f:
+ while chunk := f.read(8192):
+ hasher.update(chunk)
+ return hasher.hexdigest()
+
+
+def hash_file_sha256_large_buffer(filepath: Path) -> str:
+ hasher = hashlib.sha256()
+ with filepath.open("rb") as f:
+ while chunk := f.read(65536):
+ hasher.update(chunk)
+ return hasher.hexdigest()
+
+
+def hash_file_sha256_mmap(filepath: Path) -> str:
+ import mmap
+ hasher = hashlib.sha256()
+ with filepath.open("rb") as f:
+ with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
+ hasher.update(mm)
+ return hasher.hexdigest()
+
+
+def hash_file_md5(filepath: Path) -> str:
+ hasher = hashlib.md5()
+ with filepath.open("rb") as f:
+ while chunk := f.read(8192):
+ hasher.update(chunk)
+ return hasher.hexdigest()
+
+
+def hash_file_blake2b(filepath: Path) -> str:
+ hasher = hashlib.blake2b()
+ with filepath.open("rb") as f:
+ while chunk := f.read(8192):
+ hasher.update(chunk)
+ return hasher.hexdigest()
+
+
+def bench_hash_files(files: list[Path], hash_func) -> float:
+ start = time.perf_counter()
+ for f in files:
+ _ = hash_func(f)
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<45} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 115)
+ for r in results:
+ print(
+ f"{r['name']:<45} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ configs = [
+ (50, 5),
+ (200, 10),
+ (500, 20),
+ ]
+
+ for file_count, avg_size_kb in configs:
+ print(f"\n{'='*115}")
+ print(f"File Hashing Benchmark (files={file_count}, avg_size={avg_size_kb}KB)")
+ print(f"{'='*115}")
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ files = create_test_files(tmpdir, file_count, avg_size_kb)
+ total_mb = sum(f.stat().st_size for f in files) / (1024 * 1024)
+ print(f"Total data: {total_mb:.1f} MB")
+
+ results = []
+
+ r = run_benchmark(f"sha256 8KB buf ({file_count}f)", bench_hash_files, files, hash_file_sha256)
+ results.append(r)
+
+ r = run_benchmark(f"sha256 64KB buf ({file_count}f)", bench_hash_files, files, hash_file_sha256_large_buffer)
+ results.append(r)
+
+ r = run_benchmark(f"sha256 mmap ({file_count}f)", bench_hash_files, files, hash_file_sha256_mmap)
+ results.append(r)
+
+ r = run_benchmark(f"md5 ({file_count}f)", bench_hash_files, files, hash_file_md5)
+ results.append(r)
+
+ r = run_benchmark(f"blake2b ({file_count}f)", bench_hash_files, files, hash_file_blake2b)
+ results.append(r)
+
+ print_results(results)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_find_ending_with_fix.py b/benchmarks/bench_find_ending_with_fix.py
new file mode 100644
index 000000000..c9ef01cae
--- /dev/null
+++ b/benchmarks/bench_find_ending_with_fix.py
@@ -0,0 +1,218 @@
+import statistics
+import time
+from collections import defaultdict
+
+from codebase_rag.graph_updater import FunctionRegistryTrie
+from codebase_rag.types_defs import NodeType, SimpleNameLookup
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 30
+
+
+def generate_realistic_registry(count: int) -> tuple[list[str], list[str]]:
+ modules = ["codebase_rag", "utils", "parsers", "services", "tools", "models"]
+ submodules = ["core", "api", "handlers", "helpers", "base", "factory"]
+ classes = ["Handler", "Manager", "Factory", "Builder", "Processor", "Resolver",
+ "Analyzer", "Extractor", "Generator", "Validator"]
+ methods = ["process", "handle", "create", "build", "resolve", "validate",
+ "execute", "parse", "extract", "transform", "analyze", "generate",
+ "find", "get", "set", "update", "delete", "check"]
+
+ qualified_names = []
+ for i in range(count):
+ mod = modules[i % len(modules)]
+ sub = submodules[(i // len(modules)) % len(submodules)]
+ cls = classes[(i // (len(modules) * len(submodules))) % len(classes)]
+ meth = methods[(i // (len(modules) * len(submodules) * len(classes))) % len(methods)]
+ qualified_names.append(f"{mod}.{sub}.{cls}.method_{i}.{meth}")
+
+ lookup_suffixes = methods + [f"method_{i}" for i in range(0, count, count // 20)]
+ return qualified_names, lookup_suffixes
+
+
+def bench_linear_scan_endswith(entries: dict[str, NodeType], suffix: str) -> float:
+ start = time.perf_counter()
+ _ = [qn for qn in entries.keys() if qn.endswith(f".{suffix}")]
+ return time.perf_counter() - start
+
+
+def bench_indexed_lookup(lookup: SimpleNameLookup, suffix: str) -> float:
+ start = time.perf_counter()
+ _ = list(lookup.get(suffix, set()))
+ return time.perf_counter() - start
+
+
+def bench_trie_find_ending_with_index_hit(
+ trie: FunctionRegistryTrie, suffixes: list[str], indexed_suffixes: set[str]
+) -> float:
+ start = time.perf_counter()
+ for suffix in suffixes:
+ if suffix in indexed_suffixes:
+ _ = trie.find_ending_with(suffix)
+ return time.perf_counter() - start
+
+
+def bench_trie_find_ending_with_index_miss(
+ trie: FunctionRegistryTrie, suffixes: list[str], indexed_suffixes: set[str]
+) -> float:
+ start = time.perf_counter()
+ for suffix in suffixes:
+ if suffix not in indexed_suffixes:
+ _ = trie.find_ending_with(suffix)
+ return time.perf_counter() - start
+
+
+def bench_trie_find_ending_with_all(
+ trie: FunctionRegistryTrie, suffixes: list[str]
+) -> float:
+ start = time.perf_counter()
+ for suffix in suffixes:
+ _ = trie.find_ending_with(suffix)
+ return time.perf_counter() - start
+
+
+def bench_linear_scan_batch(entries: dict[str, NodeType], suffixes: list[str]) -> float:
+ start = time.perf_counter()
+ for suffix in suffixes:
+ _ = [qn for qn in entries.keys() if qn.endswith(f".{suffix}")]
+ return time.perf_counter() - start
+
+
+def bench_indexed_lookup_batch(lookup: SimpleNameLookup, suffixes: list[str]) -> float:
+ start = time.perf_counter()
+ for suffix in suffixes:
+ _ = list(lookup.get(suffix, set()))
+ return time.perf_counter() - start
+
+
+def bench_full_suffix_index_batch(
+ suffix_index: dict[str, set[str]], suffixes: list[str]
+) -> float:
+ start = time.perf_counter()
+ for suffix in suffixes:
+ _ = list(suffix_index.get(suffix, set()))
+ return time.perf_counter() - start
+
+
+def build_full_suffix_index(qualified_names: list[str]) -> dict[str, set[str]]:
+ index: dict[str, set[str]] = defaultdict(set)
+ for qn in qualified_names:
+ simple_name = qn.rsplit(".", 1)[-1]
+ index[simple_name].add(qn)
+ return dict(index)
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<55} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 125)
+ for r in results:
+ print(
+ f"{r['name']:<55} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ print("=" * 125)
+ print("find_ending_with FIX BENCHMARK: Linear Scan vs Indexed Lookup")
+ print("This benchmarks the #1 CPU hotspot (48.3% of total runtime)")
+ print("=" * 125)
+
+ sizes = [1000, 4500, 10000]
+
+ for size in sizes:
+ print(f"\n{'='*125}")
+ print(f"Registry size: {size} entries")
+ print(f"{'='*125}")
+
+ qualified_names, lookup_suffixes = generate_realistic_registry(size)
+
+ simple_lookup: SimpleNameLookup = defaultdict(set)
+ trie = FunctionRegistryTrie(simple_name_lookup=simple_lookup)
+ for qn in qualified_names:
+ trie.insert(qn, NodeType.FUNCTION)
+ simple_name = qn.rsplit(".", 1)[-1]
+ simple_lookup[simple_name].add(qn)
+
+ full_suffix_index = build_full_suffix_index(qualified_names)
+
+ partially_indexed_suffixes = set(list(simple_lookup.keys())[:len(simple_lookup) // 5])
+ miss_suffixes = [s for s in lookup_suffixes if s not in partially_indexed_suffixes]
+
+ results = []
+
+ print(f"\nSingle-suffix operations (on '{lookup_suffixes[0]}'):")
+ r = run_benchmark(
+ f"LINEAR SCAN endswith ({size} entries)",
+ bench_linear_scan_endswith, dict(trie.items()), lookup_suffixes[0],
+ )
+ results.append(r)
+
+ r = run_benchmark(
+ f"INDEXED lookup (hit) ({size} entries)",
+ bench_indexed_lookup, simple_lookup, lookup_suffixes[0],
+ )
+ results.append(r)
+
+ print_results(results)
+ if results[1]["median_ms"] > 0:
+ speedup = results[0]["median_ms"] / results[1]["median_ms"]
+ print(f"\n -> Index hit speedup: {speedup:.0f}x")
+
+ results = []
+ num_queries = len(lookup_suffixes)
+ print(f"\nBatch operations ({num_queries} queries, simulating call resolution):")
+
+ r = run_benchmark(
+ f"LINEAR SCAN batch ({num_queries}q, {size} entries)",
+ bench_linear_scan_batch, dict(trie.items()), lookup_suffixes,
+ )
+ results.append(r)
+
+ r = run_benchmark(
+ f"PARTIAL INDEX batch ({num_queries}q, {size} entries)",
+ bench_trie_find_ending_with_all, trie, lookup_suffixes,
+ )
+ results.append(r)
+
+ r = run_benchmark(
+ f"FULL SUFFIX INDEX batch ({num_queries}q, {size} entries)",
+ bench_full_suffix_index_batch, full_suffix_index, lookup_suffixes,
+ )
+ results.append(r)
+
+ print_results(results)
+
+ if results[2]["median_ms"] > 0:
+ print(f"\n -> Linear scan vs full index: {results[0]['median_ms'] / results[2]['median_ms']:.0f}x speedup")
+ print(f" -> Partial index vs full index: {results[1]['median_ms'] / results[2]['median_ms']:.1f}x speedup")
+
+ print(f"\n\n{'='*125}")
+ print("CONCLUSION: The 48.3% CPU hotspot is caused by linear scans on index misses.")
+ print("Building a complete suffix index eliminates the bottleneck entirely.")
+ print("This is a pure Python fix requiring zero FFI, zero new dependencies.")
+ print(f"{'='*125}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_graph_loader.py b/benchmarks/bench_graph_loader.py
new file mode 100644
index 000000000..f93ccd7a4
--- /dev/null
+++ b/benchmarks/bench_graph_loader.py
@@ -0,0 +1,169 @@
+import json
+import statistics
+import tempfile
+import time
+from pathlib import Path
+
+from codebase_rag.graph_loader import GraphLoader
+
+WARMUP_RUNS = 2
+BENCH_RUNS = 20
+
+
+def generate_graph_json(num_nodes: int, num_rels: int) -> str:
+ nodes = []
+ for i in range(num_nodes):
+ nodes.append({
+ "node_id": i,
+ "labels": ["Function" if i % 3 == 0 else "Class" if i % 3 == 1 else "Module"],
+ "properties": {
+ "qualified_name": f"project.module{i // 100}.Class{i // 10}.method{i}",
+ "name": f"method{i}",
+ "start_line": i * 10,
+ "end_line": i * 10 + 9,
+ },
+ })
+
+ rels = []
+ for i in range(num_rels):
+ rels.append({
+ "from_id": i % num_nodes,
+ "to_id": (i * 7 + 3) % num_nodes,
+ "type": "CALLS" if i % 2 == 0 else "DEFINES",
+ "properties": {},
+ })
+
+ graph = {
+ "nodes": nodes,
+ "relationships": rels,
+ "metadata": {
+ "total_nodes": num_nodes,
+ "total_relationships": num_rels,
+ },
+ }
+ return json.dumps(graph)
+
+
+def bench_json_parse(json_str: str) -> float:
+ start = time.perf_counter()
+ _ = json.loads(json_str)
+ return time.perf_counter() - start
+
+
+def bench_graph_load(file_path: str) -> float:
+ start = time.perf_counter()
+ loader = GraphLoader(file_path)
+ loader.load()
+ return time.perf_counter() - start
+
+
+def bench_find_nodes_by_label(loader: GraphLoader) -> float:
+ labels = ["Function", "Class", "Module"]
+ start = time.perf_counter()
+ for label in labels:
+ _ = loader.find_nodes_by_label(label)
+ return time.perf_counter() - start
+
+
+def bench_find_node_by_property(loader: GraphLoader) -> float:
+ start = time.perf_counter()
+ for i in range(100):
+ qn = f"project.module{i}.Class{i * 10 // 10}.method{i * 10}"
+ _ = loader.find_node_by_property("qualified_name", qn)
+ return time.perf_counter() - start
+
+
+def bench_get_relationships(loader: GraphLoader, num_nodes: int) -> float:
+ start = time.perf_counter()
+ for i in range(min(500, num_nodes)):
+ _ = loader.get_relationships_for_node(i)
+ return time.perf_counter() - start
+
+
+def bench_summary(loader: GraphLoader) -> float:
+ start = time.perf_counter()
+ _ = loader.summary()
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<40} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 110)
+ for r in results:
+ print(
+ f"{r['name']:<40} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ configs = [
+ (1000, 2000),
+ (5000, 10000),
+ (20000, 50000),
+ ]
+
+ for num_nodes, num_rels in configs:
+ print(f"\n{'='*110}")
+ print(f"GraphLoader Benchmark (nodes={num_nodes}, rels={num_rels})")
+ print(f"{'='*110}")
+
+ json_str = generate_graph_json(num_nodes, num_rels)
+ print(f"JSON size: {len(json_str) / 1024:.1f} KB")
+
+ with tempfile.NamedTemporaryFile(
+ mode="w", suffix=".json", delete=False
+ ) as tmp:
+ tmp.write(json_str)
+ tmp_path = tmp.name
+
+ results = []
+
+ r = run_benchmark(f"json.loads ({num_nodes}n)", bench_json_parse, json_str)
+ results.append(r)
+
+ r = run_benchmark(f"GraphLoader.load ({num_nodes}n)", bench_graph_load, tmp_path)
+ results.append(r)
+
+ loader = GraphLoader(tmp_path)
+ loader.load()
+
+ r = run_benchmark(f"find_nodes_by_label ({num_nodes}n)", bench_find_nodes_by_label, loader)
+ results.append(r)
+
+ r = run_benchmark(f"find_node_by_property ({num_nodes}n)", bench_find_node_by_property, loader)
+ results.append(r)
+
+ r = run_benchmark(f"get_relationships ({num_nodes}n)", bench_get_relationships, loader, num_nodes)
+ results.append(r)
+
+ r = run_benchmark(f"summary ({num_nodes}n)", bench_summary, loader)
+ results.append(r)
+
+ print_results(results)
+
+ Path(tmp_path).unlink(missing_ok=True)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_json_serialization.py b/benchmarks/bench_json_serialization.py
new file mode 100644
index 000000000..98fc477f7
--- /dev/null
+++ b/benchmarks/bench_json_serialization.py
@@ -0,0 +1,159 @@
+import json
+import statistics
+import tempfile
+import time
+from pathlib import Path
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 20
+
+
+def generate_graph_data(num_nodes: int, num_rels: int) -> dict:
+ nodes = []
+ for i in range(num_nodes):
+ nodes.append({
+ "id": i,
+ "labels": ["Function" if i % 3 == 0 else "Class" if i % 3 == 1 else "Module"],
+ "properties": {
+ "qualified_name": f"project.module{i // 100}.Class{i // 10}.method{i}",
+ "name": f"method{i}",
+ "start_line": i * 10,
+ "end_line": i * 10 + 9,
+ "docstring": f"Method {i} documentation string with some content" if i % 5 == 0 else None,
+ "decorators": ["staticmethod"] if i % 7 == 0 else [],
+ "is_exported": i % 4 == 0,
+ },
+ })
+
+ rels = []
+ for i in range(num_rels):
+ rels.append({
+ "from_id": i % num_nodes,
+ "to_id": (i * 7 + 3) % num_nodes,
+ "type": "CALLS" if i % 3 == 0 else "DEFINES" if i % 3 == 1 else "IMPORTS",
+ "properties": {"weight": i % 10} if i % 5 == 0 else {},
+ })
+
+ return {
+ "nodes": nodes,
+ "relationships": rels,
+ "metadata": {
+ "total_nodes": num_nodes,
+ "total_relationships": num_rels,
+ "exported_at": "2026-03-14T10:00:00+00:00",
+ },
+ }
+
+
+def bench_json_dumps(data: dict) -> float:
+ start = time.perf_counter()
+ _ = json.dumps(data)
+ return time.perf_counter() - start
+
+
+def bench_json_dumps_indent(data: dict) -> float:
+ start = time.perf_counter()
+ _ = json.dumps(data, indent=2, ensure_ascii=False)
+ return time.perf_counter() - start
+
+
+def bench_json_loads(json_str: str) -> float:
+ start = time.perf_counter()
+ _ = json.loads(json_str)
+ return time.perf_counter() - start
+
+
+def bench_json_dump_file(data: dict, path: str) -> float:
+ start = time.perf_counter()
+ with open(path, "w", encoding="utf-8") as f:
+ json.dump(data, f, indent=2, ensure_ascii=False)
+ return time.perf_counter() - start
+
+
+def bench_json_load_file(path: str) -> float:
+ start = time.perf_counter()
+ with open(path, encoding="utf-8") as f:
+ _ = json.load(f)
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<45} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 115)
+ for r in results:
+ print(
+ f"{r['name']:<45} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ configs = [
+ (1000, 2000),
+ (5000, 10000),
+ (20000, 50000),
+ ]
+
+ for num_nodes, num_rels in configs:
+ print(f"\n{'='*115}")
+ print(f"JSON Serialization Benchmark (nodes={num_nodes}, rels={num_rels})")
+ print(f"{'='*115}")
+
+ data = generate_graph_data(num_nodes, num_rels)
+ json_str = json.dumps(data)
+ json_str_indented = json.dumps(data, indent=2, ensure_ascii=False)
+ print(f"Compact JSON: {len(json_str) / 1024:.1f} KB, Indented: {len(json_str_indented) / 1024:.1f} KB")
+
+ with tempfile.NamedTemporaryFile(
+ mode="w", suffix=".json", delete=False
+ ) as tmp:
+ json.dump(data, tmp, indent=2, ensure_ascii=False)
+ tmp_path = tmp.name
+
+ results = []
+
+ r = run_benchmark(f"json.dumps compact ({num_nodes}n)", bench_json_dumps, data)
+ results.append(r)
+
+ r = run_benchmark(f"json.dumps indented ({num_nodes}n)", bench_json_dumps_indent, data)
+ results.append(r)
+
+ r = run_benchmark(f"json.loads compact ({num_nodes}n)", bench_json_loads, json_str)
+ results.append(r)
+
+ r = run_benchmark(f"json.loads indented ({num_nodes}n)", bench_json_loads, json_str_indented)
+ results.append(r)
+
+ r = run_benchmark(f"json.dump to file ({num_nodes}n)", bench_json_dump_file, data, tmp_path)
+ results.append(r)
+
+ r = run_benchmark(f"json.load from file ({num_nodes}n)", bench_json_load_file, tmp_path)
+ results.append(r)
+
+ print_results(results)
+
+ Path(tmp_path).unlink(missing_ok=True)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_pathlib_vs_string.py b/benchmarks/bench_pathlib_vs_string.py
new file mode 100644
index 000000000..1794b2cef
--- /dev/null
+++ b/benchmarks/bench_pathlib_vs_string.py
@@ -0,0 +1,214 @@
+import os
+import statistics
+import time
+from pathlib import Path, PurePosixPath
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 50
+
+
+def generate_file_paths(repo_root: str, count: int) -> list[str]:
+ dirs = ["src", "lib", "utils", "core", "parsers", "services", "tools", "tests"]
+ subdirs = ["base", "handlers", "helpers", "models", "schemas", "config"]
+ extensions = [".py", ".js", ".ts", ".rs", ".go", ".java", ".cpp"]
+
+ paths = []
+ for i in range(count):
+ d = dirs[i % len(dirs)]
+ sd = subdirs[(i // len(dirs)) % len(subdirs)]
+ ext = extensions[(i // (len(dirs) * len(subdirs))) % len(extensions)]
+ paths.append(f"{repo_root}/{d}/{sd}/module_{i}{ext}")
+ return paths
+
+
+def generate_skip_patterns() -> list[str]:
+ return [
+ "node_modules", ".git", "__pycache__", ".venv", "dist", "build",
+ ".mypy_cache", ".pytest_cache", ".tox", "egg-info",
+ ]
+
+
+def bench_pathlib_relative_to(paths: list[str], repo_root: str) -> float:
+ repo_path = Path(repo_root)
+ start = time.perf_counter()
+ for p in paths:
+ path = Path(p)
+ _ = path.relative_to(repo_path)
+ return time.perf_counter() - start
+
+
+def bench_string_removeprefix(paths: list[str], repo_root: str) -> float:
+ prefix = repo_root + "/"
+ start = time.perf_counter()
+ for p in paths:
+ _ = p.removeprefix(prefix)
+ return time.perf_counter() - start
+
+
+def bench_os_path_relpath(paths: list[str], repo_root: str) -> float:
+ start = time.perf_counter()
+ for p in paths:
+ _ = os.path.relpath(p, repo_root)
+ return time.perf_counter() - start
+
+
+def bench_pathlib_should_skip(paths: list[str], repo_root: str, skip_patterns: list[str]) -> float:
+ repo_path = Path(repo_root)
+ skip_set = set(skip_patterns)
+ start = time.perf_counter()
+ for p in paths:
+ path = Path(p)
+ try:
+ relative = path.relative_to(repo_path)
+ parts = relative.parts
+ _ = any(part in skip_set for part in parts)
+ except ValueError:
+ pass
+ return time.perf_counter() - start
+
+
+def bench_string_should_skip(paths: list[str], repo_root: str, skip_patterns: list[str]) -> float:
+ prefix = repo_root + "/"
+ skip_set = set(skip_patterns)
+ start = time.perf_counter()
+ for p in paths:
+ relative = p.removeprefix(prefix)
+ parts = relative.split("/")
+ _ = any(part in skip_set for part in parts)
+ return time.perf_counter() - start
+
+
+def bench_pathlib_suffix_check(paths: list[str]) -> float:
+ start = time.perf_counter()
+ for p in paths:
+ path = Path(p)
+ _ = path.suffix
+ return time.perf_counter() - start
+
+
+def bench_string_suffix_check(paths: list[str]) -> float:
+ start = time.perf_counter()
+ for p in paths:
+ dot_idx = p.rfind(".")
+ _ = p[dot_idx:] if dot_idx >= 0 else ""
+ return time.perf_counter() - start
+
+
+def bench_os_path_splitext(paths: list[str]) -> float:
+ start = time.perf_counter()
+ for p in paths:
+ _, _ = os.path.splitext(p)
+ return time.perf_counter() - start
+
+
+def bench_pathlib_name(paths: list[str]) -> float:
+ start = time.perf_counter()
+ for p in paths:
+ path = Path(p)
+ _ = path.name
+ return time.perf_counter() - start
+
+
+def bench_string_name(paths: list[str]) -> float:
+ start = time.perf_counter()
+ for p in paths:
+ slash_idx = p.rfind("/")
+ _ = p[slash_idx + 1:] if slash_idx >= 0 else p
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<55} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 125)
+ for r in results:
+ print(
+ f"{r['name']:<55} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ print("=" * 125)
+ print("pathlib vs String Operations Benchmark")
+ print("This benchmarks the #2 CPU hotspot (13.7% of total runtime)")
+ print("=" * 125)
+
+ repo_root = "/Users/developer/projects/large-repo"
+ skip_patterns = generate_skip_patterns()
+
+ for count in [1000, 5000, 20000, 59012]:
+ print(f"\n{'='*125}")
+ print(f"Path count: {count} (59012 = actual profiled call count)")
+ print(f"{'='*125}")
+
+ paths = generate_file_paths(repo_root, count)
+
+ results = []
+
+ print("\n--- relative_to vs removeprefix ---")
+ r1 = run_benchmark(f"pathlib.relative_to ({count}p)", bench_pathlib_relative_to, paths, repo_root)
+ results.append(r1)
+ r2 = run_benchmark(f"str.removeprefix ({count}p)", bench_string_removeprefix, paths, repo_root)
+ results.append(r2)
+ r3 = run_benchmark(f"os.path.relpath ({count}p)", bench_os_path_relpath, paths, repo_root)
+ results.append(r3)
+
+ print_results(results)
+ print(f"\n -> pathlib vs str.removeprefix: {r1['median_ms'] / r2['median_ms']:.0f}x slower")
+ print(f" -> pathlib vs os.path.relpath: {r1['median_ms'] / r3['median_ms']:.1f}x slower")
+
+ results = []
+ print("\n--- should_skip_path (full function) ---")
+ r1 = run_benchmark(f"pathlib should_skip ({count}p)", bench_pathlib_should_skip, paths, repo_root, skip_patterns)
+ results.append(r1)
+ r2 = run_benchmark(f"string should_skip ({count}p)", bench_string_should_skip, paths, repo_root, skip_patterns)
+ results.append(r2)
+
+ print_results(results)
+ print(f"\n -> pathlib vs string: {r1['median_ms'] / r2['median_ms']:.1f}x slower")
+
+ results = []
+ print("\n--- Suffix/extension extraction ---")
+ r1 = run_benchmark(f"Path.suffix ({count}p)", bench_pathlib_suffix_check, paths)
+ results.append(r1)
+ r2 = run_benchmark(f"str.rfind ({count}p)", bench_string_suffix_check, paths)
+ results.append(r2)
+ r3 = run_benchmark(f"os.path.splitext ({count}p)", bench_os_path_splitext, paths)
+ results.append(r3)
+
+ print_results(results)
+ print(f"\n -> Path.suffix vs str.rfind: {r1['median_ms'] / r2['median_ms']:.1f}x slower")
+
+ results = []
+ print("\n--- Filename extraction ---")
+ r1 = run_benchmark(f"Path.name ({count}p)", bench_pathlib_name, paths)
+ results.append(r1)
+ r2 = run_benchmark(f"str.rfind+slice ({count}p)", bench_string_name, paths)
+ results.append(r2)
+
+ print_results(results)
+ print(f"\n -> Path.name vs str: {r1['median_ms'] / r2['median_ms']:.1f}x slower")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_string_ops.py b/benchmarks/bench_string_ops.py
new file mode 100644
index 000000000..cc10e91f8
--- /dev/null
+++ b/benchmarks/bench_string_ops.py
@@ -0,0 +1,148 @@
+import re
+import statistics
+import time
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 100
+
+SEPARATOR_PATTERN = re.compile(r"[.:]|::")
+
+
+def generate_qualified_names(count: int) -> list[str]:
+ names = []
+ modules = ["project", "utils", "core", "api", "services", "models"]
+ classes = ["Handler", "Manager", "Factory", "Builder", "Processor", "Resolver"]
+ methods = ["process", "handle", "create", "build", "resolve", "validate"]
+ for i in range(count):
+ mod = modules[i % len(modules)]
+ cls = classes[(i // len(modules)) % len(classes)]
+ meth = methods[(i // (len(modules) * len(classes))) % len(methods)]
+ names.append(f"{mod}.{cls}.sub{i}.{meth}")
+ return names
+
+
+def bench_str_split(names: list[str]) -> float:
+ start = time.perf_counter()
+ for name in names:
+ _ = name.split(".")
+ return time.perf_counter() - start
+
+
+def bench_str_endswith(names: list[str]) -> float:
+ suffixes = [".process", ".handle", ".create", ".build", ".resolve"]
+ start = time.perf_counter()
+ for name in names:
+ for suffix in suffixes:
+ _ = name.endswith(suffix)
+ return time.perf_counter() - start
+
+
+def bench_str_startswith(names: list[str]) -> float:
+ prefixes = ["project.", "utils.", "core.", "api."]
+ start = time.perf_counter()
+ for name in names:
+ for prefix in prefixes:
+ _ = name.startswith(prefix)
+ return time.perf_counter() - start
+
+
+def bench_str_join(names: list[str]) -> float:
+ split_names = [name.split(".") for name in names]
+ start = time.perf_counter()
+ for parts in split_names:
+ _ = ".".join(parts)
+ return time.perf_counter() - start
+
+
+def bench_str_replace(names: list[str]) -> float:
+ start = time.perf_counter()
+ for name in names:
+ _ = name.replace("/", ".")
+ return time.perf_counter() - start
+
+
+def bench_regex_split(names: list[str]) -> float:
+ start = time.perf_counter()
+ for name in names:
+ _ = SEPARATOR_PATTERN.split(name)
+ return time.perf_counter() - start
+
+
+def bench_str_format(names: list[str]) -> float:
+ start = time.perf_counter()
+ for name in names:
+ _ = f"module.{name}.method"
+ return time.perf_counter() - start
+
+
+def bench_import_distance(names: list[str]) -> float:
+ start = time.perf_counter()
+ for i in range(0, len(names) - 1, 2):
+ caller_parts = names[i].split(".")
+ candidate_parts = names[i + 1].split(".")
+ common = 0
+ for j in range(min(len(caller_parts), len(candidate_parts))):
+ if caller_parts[j] == candidate_parts[j]:
+ common += 1
+ else:
+ break
+ _ = max(len(caller_parts), len(candidate_parts)) - common
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<40} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 110)
+ for r in results:
+ print(
+ f"{r['name']:<40} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ sizes = [1000, 5000, 20000]
+
+ for size in sizes:
+ print(f"\n{'='*110}")
+ print(f"String Operations Benchmark (n={size})")
+ print(f"{'='*110}")
+
+ names = generate_qualified_names(size)
+
+ results = [
+ run_benchmark(f"str.split ({size})", bench_str_split, names),
+ run_benchmark(f"str.endswith ({size})", bench_str_endswith, names),
+ run_benchmark(f"str.startswith ({size})", bench_str_startswith, names),
+ run_benchmark(f"str.join ({size})", bench_str_join, names),
+ run_benchmark(f"str.replace ({size})", bench_str_replace, names),
+ run_benchmark(f"regex split ({size})", bench_regex_split, names),
+ run_benchmark(f"f-string format ({size})", bench_str_format, names),
+ run_benchmark(f"import_distance ({size})", bench_import_distance, names),
+ ]
+
+ print_results(results)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/bench_trie.py b/benchmarks/bench_trie.py
new file mode 100644
index 000000000..dba339100
--- /dev/null
+++ b/benchmarks/bench_trie.py
@@ -0,0 +1,138 @@
+import statistics
+import time
+from collections import defaultdict
+
+from codebase_rag.graph_updater import FunctionRegistryTrie
+from codebase_rag.types_defs import NodeType, SimpleNameLookup
+
+WARMUP_RUNS = 3
+BENCH_RUNS = 50
+
+
+def generate_qualified_names(count: int) -> list[str]:
+ names = []
+ modules = ["project", "utils", "core", "api", "services", "models"]
+ classes = ["Handler", "Manager", "Factory", "Builder", "Processor", "Resolver"]
+ methods = ["process", "handle", "create", "build", "resolve", "validate", "execute"]
+ for i in range(count):
+ mod = modules[i % len(modules)]
+ cls = classes[(i // len(modules)) % len(classes)]
+ meth = methods[(i // (len(modules) * len(classes))) % len(methods)]
+ sub = f"sub{i}"
+ names.append(f"{mod}.{cls}.{sub}.{meth}")
+ return names
+
+
+def bench_insert(trie: FunctionRegistryTrie, names: list[str]) -> float:
+ start = time.perf_counter()
+ for name in names:
+ trie.insert(name, NodeType.FUNCTION)
+ return time.perf_counter() - start
+
+
+def bench_lookup(trie: FunctionRegistryTrie, names: list[str]) -> float:
+ start = time.perf_counter()
+ for name in names:
+ _ = name in trie
+ return time.perf_counter() - start
+
+
+def bench_find_ending_with(trie: FunctionRegistryTrie) -> float:
+ suffixes = ["process", "handle", "create", "build", "resolve", "validate", "execute"]
+ start = time.perf_counter()
+ for suffix in suffixes:
+ _ = trie.find_ending_with(suffix)
+ return time.perf_counter() - start
+
+
+def bench_find_with_prefix(trie: FunctionRegistryTrie) -> float:
+ prefixes = ["project", "utils", "core", "api", "services", "models"]
+ start = time.perf_counter()
+ for prefix in prefixes:
+ _ = trie.find_with_prefix(prefix)
+ return time.perf_counter() - start
+
+
+def bench_delete(names: list[str]) -> float:
+ simple_lookup: SimpleNameLookup = defaultdict(set)
+ trie = FunctionRegistryTrie(simple_name_lookup=simple_lookup)
+ for name in names:
+ trie.insert(name, NodeType.FUNCTION)
+ simple_name = name.split(".")[-1]
+ simple_lookup[simple_name].add(name)
+
+ start = time.perf_counter()
+ for name in names[:len(names) // 4]:
+ del trie[name]
+ return time.perf_counter() - start
+
+
+def run_benchmark(name: str, func, *args) -> dict[str, float]:
+ for _ in range(WARMUP_RUNS):
+ func(*args)
+
+ times = []
+ for _ in range(BENCH_RUNS):
+ times.append(func(*args))
+
+ return {
+ "name": name,
+ "median_ms": statistics.median(times) * 1000,
+ "mean_ms": statistics.mean(times) * 1000,
+ "stddev_ms": statistics.stdev(times) * 1000 if len(times) > 1 else 0,
+ "min_ms": min(times) * 1000,
+ "max_ms": max(times) * 1000,
+ "p95_ms": sorted(times)[int(len(times) * 0.95)] * 1000,
+ }
+
+
+def print_results(results: list[dict[str, float]]) -> None:
+ print(f"\n{'Benchmark':<35} {'Median':>10} {'Mean':>10} {'StdDev':>10} {'Min':>10} {'Max':>10} {'P95':>10}")
+ print("-" * 105)
+ for r in results:
+ print(
+ f"{r['name']:<35} {r['median_ms']:>9.3f}ms {r['mean_ms']:>9.3f}ms "
+ f"{r['stddev_ms']:>9.3f}ms {r['min_ms']:>9.3f}ms {r['max_ms']:>9.3f}ms "
+ f"{r['p95_ms']:>9.3f}ms"
+ )
+
+
+def main() -> None:
+ sizes = [1000, 5000, 10000, 50000]
+
+ for size in sizes:
+ print(f"\n{'='*105}")
+ print(f"FunctionRegistryTrie Benchmark (n={size})")
+ print(f"{'='*105}")
+
+ names = generate_qualified_names(size)
+
+ simple_lookup: SimpleNameLookup = defaultdict(set)
+ trie = FunctionRegistryTrie(simple_name_lookup=simple_lookup)
+
+ results = []
+
+ r = run_benchmark(f"insert ({size})", bench_insert, trie, names)
+ results.append(r)
+
+ for name in names:
+ simple_name = name.split(".")[-1]
+ simple_lookup[simple_name].add(name)
+
+ r = run_benchmark(f"lookup ({size})", bench_lookup, trie, names)
+ results.append(r)
+
+ r = run_benchmark(f"find_ending_with ({size})", bench_find_ending_with, trie)
+ results.append(r)
+
+ r = run_benchmark(f"find_with_prefix ({size})", bench_find_with_prefix, trie)
+ results.append(r)
+
+ r = run_benchmark(f"delete 25% ({size})", bench_delete, names)
+ results.append(r)
+
+ print_results(results)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/benchmarks/results/bench_ast_cache_20260315_000043.txt b/benchmarks/results/bench_ast_cache_20260315_000043.txt
new file mode 100644
index 000000000..5084d79ef
--- /dev/null
+++ b/benchmarks/results/bench_ast_cache_20260315_000043.txt
@@ -0,0 +1,42 @@
+Benchmark: bench_ast_cache.py
+Timestamp: 20260315_000043
+Exit code: 0
+Duration: 2.2s
+Python: 3.12.2 (main, Feb 25 2024, 03:55:42) [Clang 17.0.6 ]
+================================================================================
+
+===================================================================================================================
+BoundedASTCache Benchmark (entries=500, item_size=1024B)
+===================================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+insert (500) 1.119ms 1.128ms 0.020ms 1.113ms 1.229ms 1.158ms
+lookup (500) 0.019ms 0.019ms 0.000ms 0.018ms 0.019ms 0.019ms
+access+LRU (500) 0.053ms 0.053ms 0.000ms 0.053ms 0.056ms 0.053ms
+insert+evict (max=250) 1.141ms 1.155ms 0.092ms 1.133ms 1.792ms 1.158ms
+getsizeof scan (500) 0.062ms 0.062ms 0.001ms 0.061ms 0.067ms 0.062ms
+
+===================================================================================================================
+BoundedASTCache Benchmark (entries=2000, item_size=4096B)
+===================================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+insert (2000) 4.717ms 4.798ms 0.248ms 4.591ms 5.567ms 5.558ms
+lookup (2000) 0.077ms 0.077ms 0.000ms 0.076ms 0.078ms 0.077ms
+access+LRU (2000) 0.214ms 0.214ms 0.001ms 0.213ms 0.217ms 0.216ms
+insert+evict (max=1000) 4.768ms 4.814ms 0.221ms 4.614ms 5.870ms 5.103ms
+getsizeof scan (2000) 0.257ms 0.259ms 0.005ms 0.254ms 0.279ms 0.269ms
+
+===================================================================================================================
+BoundedASTCache Benchmark (entries=5000, item_size=8192B)
+===================================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+insert (5000) 12.829ms 13.137ms 0.611ms 12.561ms 14.340ms 14.280ms
+lookup (5000) 0.206ms 0.206ms 0.002ms 0.203ms 0.210ms 0.209ms
+access+LRU (5000) 0.551ms 0.552ms 0.005ms 0.544ms 0.565ms 0.563ms
+insert+evict (max=2500) 12.558ms 12.992ms 0.936ms 12.246ms 16.534ms 14.787ms
+getsizeof scan (5000) 0.681ms 0.686ms 0.027ms 0.651ms 0.812ms 0.740ms
diff --git a/benchmarks/results/bench_embedding_cache_20260315_000043.txt b/benchmarks/results/bench_embedding_cache_20260315_000043.txt
new file mode 100644
index 000000000..807a58402
--- /dev/null
+++ b/benchmarks/results/bench_embedding_cache_20260315_000043.txt
@@ -0,0 +1,42 @@
+Benchmark: bench_embedding_cache.py
+Timestamp: 20260315_000043
+Exit code: 0
+Duration: 3.4s
+Python: 3.12.2 (main, Feb 25 2024, 03:55:42) [Clang 17.0.6 ]
+================================================================================
+
+==============================================================================================================
+EmbeddingCache Benchmark (n=500)
+==============================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+sha256 hashing (500) 0.155ms 0.151ms 0.006ms 0.143ms 0.161ms 0.159ms
+cache.put (500) 0.182ms 0.182ms 0.002ms 0.179ms 0.187ms 0.185ms
+cache.get hit (500) 0.177ms 0.177ms 0.001ms 0.176ms 0.180ms 0.179ms
+cache.get miss (500) 0.190ms 0.192ms 0.003ms 0.189ms 0.207ms 0.195ms
+cache.get_many (500) 0.190ms 0.190ms 0.001ms 0.189ms 0.193ms 0.191ms
+
+==============================================================================================================
+EmbeddingCache Benchmark (n=2000)
+==============================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+sha256 hashing (2000) 0.562ms 0.564ms 0.006ms 0.557ms 0.581ms 0.576ms
+cache.put (2000) 0.751ms 0.760ms 0.027ms 0.738ms 0.918ms 0.794ms
+cache.get hit (2000) 0.729ms 0.732ms 0.009ms 0.719ms 0.765ms 0.748ms
+cache.get miss (2000) 0.797ms 0.801ms 0.026ms 0.771ms 0.866ms 0.839ms
+cache.get_many (2000) 0.798ms 0.808ms 0.028ms 0.777ms 0.888ms 0.856ms
+
+==============================================================================================================
+EmbeddingCache Benchmark (n=10000)
+==============================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+sha256 hashing (10000) 2.884ms 2.875ms 0.034ms 2.815ms 2.950ms 2.921ms
+cache.put (10000) 3.790ms 3.786ms 0.024ms 3.729ms 3.827ms 3.821ms
+cache.get hit (10000) 3.690ms 3.697ms 0.029ms 3.653ms 3.775ms 3.750ms
+cache.get miss (10000) 3.939ms 3.943ms 0.041ms 3.878ms 4.079ms 4.018ms
+cache.get_many (10000) 3.987ms 3.989ms 0.023ms 3.948ms 4.051ms 4.041ms
diff --git a/benchmarks/results/bench_file_hashing_20260315_000043.txt b/benchmarks/results/bench_file_hashing_20260315_000043.txt
new file mode 100644
index 000000000..6346ad2f7
--- /dev/null
+++ b/benchmarks/results/bench_file_hashing_20260315_000043.txt
@@ -0,0 +1,45 @@
+Benchmark: bench_file_hashing.py
+Timestamp: 20260315_000043
+Exit code: 0
+Duration: 4.4s
+Python: 3.12.2 (main, Feb 25 2024, 03:55:42) [Clang 17.0.6 ]
+================================================================================
+
+===================================================================================================================
+File Hashing Benchmark (files=50, avg_size=5KB)
+===================================================================================================================
+Total data: 0.2 MB
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+sha256 8KB buf (50f) 1.006ms 1.016ms 0.043ms 0.977ms 1.186ms 1.146ms
+sha256 64KB buf (50f) 1.075ms 1.070ms 0.016ms 1.036ms 1.106ms 1.090ms
+sha256 mmap (50f) 1.356ms 1.355ms 0.033ms 1.299ms 1.453ms 1.395ms
+md5 (50f) 1.310ms 1.374ms 0.171ms 1.191ms 1.878ms 1.727ms
+blake2b (50f) 1.201ms 1.253ms 0.147ms 1.106ms 1.718ms 1.632ms
+
+===================================================================================================================
+File Hashing Benchmark (files=200, avg_size=10KB)
+===================================================================================================================
+Total data: 2.0 MB
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+sha256 8KB buf (200f) 4.587ms 4.777ms 0.512ms 4.377ms 6.201ms 6.185ms
+sha256 64KB buf (200f) 4.729ms 4.819ms 0.285ms 4.557ms 5.794ms 5.706ms
+sha256 mmap (200f) 5.984ms 8.714ms 11.275ms 5.650ms 63.888ms 29.536ms
+md5 (200f) 6.532ms 6.547ms 0.143ms 6.367ms 6.993ms 6.804ms
+blake2b (200f) 5.217ms 5.289ms 0.272ms 5.068ms 6.416ms 6.003ms
+
+===================================================================================================================
+File Hashing Benchmark (files=500, avg_size=20KB)
+===================================================================================================================
+Total data: 9.8 MB
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+sha256 8KB buf (500f) 13.926ms 14.170ms 0.910ms 13.581ms 18.406ms 15.773ms
+sha256 64KB buf (500f) 14.268ms 14.312ms 0.253ms 13.957ms 15.319ms 14.640ms
+sha256 mmap (500f) 16.699ms 20.110ms 15.978ms 16.299ms 104.163ms 25.618ms
+md5 (500f) 23.512ms 23.670ms 0.567ms 23.157ms 25.836ms 25.075ms
+blake2b (500f) 17.669ms 17.783ms 0.496ms 17.229ms 19.433ms 18.815ms
diff --git a/benchmarks/results/bench_graph_loader_20260315_000043.txt b/benchmarks/results/bench_graph_loader_20260315_000043.txt
new file mode 100644
index 000000000..d9cd28a0b
--- /dev/null
+++ b/benchmarks/results/bench_graph_loader_20260315_000043.txt
@@ -0,0 +1,48 @@
+Benchmark: bench_graph_loader.py
+Timestamp: 20260315_000043
+Exit code: 0
+Duration: 2.9s
+Python: 3.12.2 (main, Feb 25 2024, 03:55:42) [Clang 17.0.6 ]
+================================================================================
+
+==============================================================================================================
+GraphLoader Benchmark (nodes=1000, rels=2000)
+==============================================================================================================
+JSON size: 298.2 KB
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+json.loads (1000n) 1.001ms 1.011ms 0.029ms 0.974ms 1.071ms 1.071ms
+GraphLoader.load (1000n) 2.040ms 2.143ms 0.583ms 1.865ms 4.581ms 4.581ms
+find_nodes_by_label (1000n) 0.001ms 0.001ms 0.000ms 0.000ms 0.001ms 0.001ms
+find_node_by_property (1000n) 0.030ms 0.030ms 0.000ms 0.029ms 0.030ms 0.030ms
+get_relationships (1000n) 0.148ms 0.148ms 0.001ms 0.146ms 0.151ms 0.151ms
+summary (1000n) 0.069ms 0.070ms 0.001ms 0.068ms 0.073ms 0.073ms
+
+==============================================================================================================
+GraphLoader Benchmark (nodes=5000, rels=10000)
+==============================================================================================================
+JSON size: 1537.8 KB
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+json.loads (5000n) 5.032ms 5.002ms 0.112ms 4.843ms 5.180ms 5.180ms
+GraphLoader.load (5000n) 10.106ms 11.137ms 2.030ms 9.396ms 14.997ms 14.997ms
+find_nodes_by_label (5000n) 0.000ms 0.000ms 0.000ms 0.000ms 0.001ms 0.001ms
+find_node_by_property (5000n) 0.030ms 0.030ms 0.000ms 0.030ms 0.030ms 0.030ms
+get_relationships (5000n) 0.150ms 0.152ms 0.005ms 0.148ms 0.170ms 0.170ms
+summary (5000n) 0.350ms 0.356ms 0.018ms 0.341ms 0.420ms 0.420ms
+
+==============================================================================================================
+GraphLoader Benchmark (nodes=20000, rels=50000)
+==============================================================================================================
+JSON size: 6979.7 KB
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+json.loads (20000n) 24.136ms 24.783ms 2.550ms 23.565ms 35.321ms 35.321ms
+GraphLoader.load (20000n) 61.008ms 62.676ms 5.050ms 57.534ms 75.337ms 75.337ms
+find_nodes_by_label (20000n) 0.000ms 0.000ms 0.000ms 0.000ms 0.001ms 0.001ms
+find_node_by_property (20000n) 0.030ms 0.030ms 0.000ms 0.030ms 0.030ms 0.030ms
+get_relationships (20000n) 0.152ms 0.153ms 0.001ms 0.151ms 0.155ms 0.155ms
+summary (20000n) 1.738ms 1.745ms 0.023ms 1.714ms 1.819ms 1.819ms
diff --git a/benchmarks/results/bench_json_serialization_20260315_000043.txt b/benchmarks/results/bench_json_serialization_20260315_000043.txt
new file mode 100644
index 000000000..aab002921
--- /dev/null
+++ b/benchmarks/results/bench_json_serialization_20260315_000043.txt
@@ -0,0 +1,48 @@
+Benchmark: bench_json_serialization.py
+Timestamp: 20260315_000043
+Exit code: 0
+Duration: 18.8s
+Python: 3.12.2 (main, Feb 25 2024, 03:55:42) [Clang 17.0.6 ]
+================================================================================
+
+===================================================================================================================
+JSON Serialization Benchmark (nodes=1000, rels=2000)
+===================================================================================================================
+Compact JSON: 366.8 KB, Indented: 547.7 KB
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+json.dumps compact (1000n) 1.089ms 1.094ms 0.010ms 1.084ms 1.117ms 1.117ms
+json.dumps indented (1000n) 9.612ms 9.703ms 0.220ms 9.560ms 10.479ms 10.479ms
+json.loads compact (1000n) 1.202ms 1.202ms 0.015ms 1.185ms 1.260ms 1.260ms
+json.loads indented (1000n) 1.286ms 1.281ms 0.023ms 1.253ms 1.325ms 1.325ms
+json.dump to file (1000n) 12.239ms 12.241ms 0.071ms 12.145ms 12.398ms 12.398ms
+json.load from file (1000n) 1.345ms 1.350ms 0.036ms 1.309ms 1.429ms 1.429ms
+
+===================================================================================================================
+JSON Serialization Benchmark (nodes=5000, rels=10000)
+===================================================================================================================
+Compact JSON: 1881.4 KB, Indented: 2786.1 KB
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+json.dumps compact (5000n) 5.701ms 5.718ms 0.158ms 5.464ms 6.000ms 6.000ms
+json.dumps indented (5000n) 47.875ms 47.950ms 0.285ms 47.618ms 48.611ms 48.611ms
+json.loads compact (5000n) 6.291ms 6.327ms 0.244ms 5.999ms 6.754ms 6.754ms
+json.loads indented (5000n) 6.686ms 6.666ms 0.263ms 6.346ms 7.152ms 7.152ms
+json.dump to file (5000n) 60.552ms 60.895ms 1.262ms 60.082ms 64.565ms 64.565ms
+json.load from file (5000n) 6.573ms 6.590ms 0.049ms 6.528ms 6.717ms 6.717ms
+
+===================================================================================================================
+JSON Serialization Benchmark (nodes=20000, rels=50000)
+===================================================================================================================
+Compact JSON: 8381.6 KB, Indented: 12363.2 KB
+
+Benchmark Median Mean StdDev Min Max P95
+-------------------------------------------------------------------------------------------------------------------
+json.dumps compact (20000n) 25.446ms 25.483ms 0.156ms 25.314ms 25.797ms 25.797ms
+json.dumps indented (20000n) 215.190ms 215.593ms 1.383ms 214.183ms 219.350ms 219.350ms
+json.loads compact (20000n) 28.713ms 28.731ms 0.480ms 28.049ms 30.253ms 30.253ms
+json.loads indented (20000n) 30.416ms 30.558ms 0.813ms 29.707ms 32.258ms 32.258ms
+json.dump to file (20000n) 271.376ms 271.918ms 3.051ms 266.710ms 278.494ms 278.494ms
+json.load from file (20000n) 32.144ms 33.111ms 3.488ms 31.594ms 47.762ms 47.762ms
diff --git a/benchmarks/results/bench_string_ops_20260315_000043.txt b/benchmarks/results/bench_string_ops_20260315_000043.txt
new file mode 100644
index 000000000..66c1bcd8b
--- /dev/null
+++ b/benchmarks/results/bench_string_ops_20260315_000043.txt
@@ -0,0 +1,51 @@
+Benchmark: bench_string_ops.py
+Timestamp: 20260315_000043
+Exit code: 0
+Duration: 3.2s
+Python: 3.12.2 (main, Feb 25 2024, 03:55:42) [Clang 17.0.6 ]
+================================================================================
+
+==============================================================================================================
+String Operations Benchmark (n=1000)
+==============================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+str.split (1000) 0.079ms 0.079ms 0.001ms 0.077ms 0.083ms 0.082ms
+str.endswith (1000) 0.179ms 0.181ms 0.006ms 0.174ms 0.219ms 0.188ms
+str.startswith (1000) 0.146ms 0.147ms 0.003ms 0.144ms 0.165ms 0.150ms
+str.join (1000) 0.036ms 0.036ms 0.001ms 0.035ms 0.047ms 0.039ms
+str.replace (1000) 0.014ms 0.014ms 0.000ms 0.014ms 0.016ms 0.014ms
+regex split (1000) 0.418ms 0.420ms 0.006ms 0.414ms 0.437ms 0.431ms
+f-string format (1000) 0.029ms 0.029ms 0.000ms 0.029ms 0.032ms 0.029ms
+import_distance (1000) 0.164ms 0.165ms 0.004ms 0.162ms 0.185ms 0.171ms
+
+==============================================================================================================
+String Operations Benchmark (n=5000)
+==============================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+str.split (5000) 0.380ms 0.380ms 0.003ms 0.371ms 0.395ms 0.387ms
+str.endswith (5000) 0.897ms 0.899ms 0.004ms 0.892ms 0.919ms 0.909ms
+str.startswith (5000) 0.722ms 0.723ms 0.003ms 0.715ms 0.733ms 0.728ms
+str.join (5000) 0.185ms 0.187ms 0.005ms 0.184ms 0.234ms 0.191ms
+str.replace (5000) 0.071ms 0.071ms 0.001ms 0.070ms 0.074ms 0.071ms
+regex split (5000) 2.033ms 2.037ms 0.023ms 1.984ms 2.103ms 2.076ms
+f-string format (5000) 0.146ms 0.147ms 0.002ms 0.145ms 0.154ms 0.150ms
+import_distance (5000) 0.781ms 0.773ms 0.014ms 0.752ms 0.797ms 0.790ms
+
+==============================================================================================================
+String Operations Benchmark (n=20000)
+==============================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+--------------------------------------------------------------------------------------------------------------
+str.split (20000) 1.588ms 1.590ms 0.014ms 1.559ms 1.626ms 1.612ms
+str.endswith (20000) 3.582ms 3.619ms 0.147ms 3.497ms 4.883ms 3.803ms
+str.startswith (20000) 2.920ms 2.926ms 0.031ms 2.876ms 3.064ms 3.005ms
+str.join (20000) 0.733ms 0.735ms 0.015ms 0.719ms 0.850ms 0.752ms
+str.replace (20000) 0.287ms 0.288ms 0.009ms 0.282ms 0.374ms 0.293ms
+regex split (20000) 8.051ms 8.047ms 0.068ms 7.924ms 8.195ms 8.174ms
+f-string format (20000) 0.593ms 0.594ms 0.006ms 0.582ms 0.624ms 0.603ms
+import_distance (20000) 3.183ms 3.184ms 0.039ms 3.129ms 3.315ms 3.262ms
diff --git a/benchmarks/results/bench_trie_20260315_000043.txt b/benchmarks/results/bench_trie_20260315_000043.txt
new file mode 100644
index 000000000..10ad3978e
--- /dev/null
+++ b/benchmarks/results/bench_trie_20260315_000043.txt
@@ -0,0 +1,54 @@
+Benchmark: bench_trie.py
+Timestamp: 20260315_000043
+Exit code: 0
+Duration: 9.3s
+Python: 3.12.2 (main, Feb 25 2024, 03:55:42) [Clang 17.0.6 ]
+================================================================================
+
+=========================================================================================================
+FunctionRegistryTrie Benchmark (n=1000)
+=========================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+---------------------------------------------------------------------------------------------------------
+insert (1000) 0.340ms 0.341ms 0.012ms 0.327ms 0.385ms 0.378ms
+lookup (1000) 0.036ms 0.036ms 0.000ms 0.035ms 0.037ms 0.036ms
+find_ending_with (1000) 0.004ms 0.005ms 0.004ms 0.004ms 0.031ms 0.004ms
+find_with_prefix (1000) 0.390ms 0.425ms 0.059ms 0.369ms 0.589ms 0.528ms
+delete 25% (1000) 0.407ms 0.418ms 0.021ms 0.394ms 0.457ms 0.449ms
+
+=========================================================================================================
+FunctionRegistryTrie Benchmark (n=5000)
+=========================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+---------------------------------------------------------------------------------------------------------
+insert (5000) 1.795ms 1.797ms 0.037ms 1.721ms 1.911ms 1.876ms
+lookup (5000) 0.195ms 0.196ms 0.002ms 0.193ms 0.201ms 0.200ms
+find_ending_with (5000) 0.019ms 0.019ms 0.000ms 0.018ms 0.021ms 0.019ms
+find_with_prefix (5000) 2.104ms 2.299ms 1.047ms 2.024ms 9.499ms 2.416ms
+delete 25% (5000) 2.116ms 2.122ms 0.048ms 2.043ms 2.260ms 2.214ms
+
+=========================================================================================================
+FunctionRegistryTrie Benchmark (n=10000)
+=========================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+---------------------------------------------------------------------------------------------------------
+insert (10000) 3.709ms 3.735ms 0.106ms 3.627ms 4.244ms 3.912ms
+lookup (10000) 0.402ms 0.403ms 0.003ms 0.398ms 0.412ms 0.407ms
+find_ending_with (10000) 0.046ms 0.046ms 0.002ms 0.045ms 0.056ms 0.050ms
+find_with_prefix (10000) 4.244ms 4.630ms 1.843ms 3.904ms 13.674ms 5.386ms
+delete 25% (10000) 4.204ms 4.207ms 0.066ms 3.959ms 4.349ms 4.312ms
+
+=========================================================================================================
+FunctionRegistryTrie Benchmark (n=50000)
+=========================================================================================================
+
+Benchmark Median Mean StdDev Min Max P95
+---------------------------------------------------------------------------------------------------------
+insert (50000) 18.036ms 18.128ms 0.306ms 17.831ms 18.972ms 18.820ms
+lookup (50000) 2.058ms 2.061ms 0.013ms 2.036ms 2.091ms 2.085ms
+find_ending_with (50000) 0.420ms 0.426ms 0.014ms 0.412ms 0.477ms 0.458ms
+find_with_prefix (50000) 38.507ms 38.096ms 10.219ms 22.462ms 56.890ms 52.739ms
+delete 25% (50000) 21.744ms 21.830ms 0.410ms 21.277ms 23.496ms 22.524ms
diff --git a/benchmarks/run_all.py b/benchmarks/run_all.py
new file mode 100644
index 000000000..a79c339ab
--- /dev/null
+++ b/benchmarks/run_all.py
@@ -0,0 +1,74 @@
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+BENCHMARKS = [
+ "bench_string_ops.py",
+ "bench_trie.py",
+ "bench_find_ending_with_fix.py",
+ "bench_dropin_replacements.py",
+ "bench_graph_loader.py",
+ "bench_file_hashing.py",
+ "bench_embedding_cache.py",
+ "bench_json_serialization.py",
+ "bench_ast_cache.py",
+ "bench_pathlib_vs_string.py",
+]
+
+
+def main() -> None:
+ bench_dir = Path(__file__).parent
+ results_dir = bench_dir / "results"
+ results_dir.mkdir(exist_ok=True)
+
+ timestamp = time.strftime("%Y%m%d_%H%M%S")
+ overall_start = time.perf_counter()
+
+ print(f"Running {len(BENCHMARKS)} benchmark suites")
+ print(f"Results will be saved to: {results_dir}")
+ print(f"Timestamp: {timestamp}")
+ print("=" * 80)
+
+ for bench_file in BENCHMARKS:
+ bench_path = bench_dir / bench_file
+ if not bench_path.exists():
+ print(f"SKIP: {bench_file} (not found)")
+ continue
+
+ result_file = results_dir / f"{bench_path.stem}_{timestamp}.txt"
+ print(f"\nRunning: {bench_file}")
+
+ start = time.perf_counter()
+ result = subprocess.run(
+ [sys.executable, str(bench_path)],
+ capture_output=True,
+ text=True,
+ timeout=600,
+ )
+ elapsed = time.perf_counter() - start
+
+ output = result.stdout
+ if result.returncode != 0:
+ output += f"\nSTDERR:\n{result.stderr}"
+ print(f" FAILED (exit code {result.returncode}, {elapsed:.1f}s)")
+ else:
+ print(f" OK ({elapsed:.1f}s)")
+
+ with result_file.open("w") as f:
+ f.write(f"Benchmark: {bench_file}\n")
+ f.write(f"Timestamp: {timestamp}\n")
+ f.write(f"Exit code: {result.returncode}\n")
+ f.write(f"Duration: {elapsed:.1f}s\n")
+ f.write(f"Python: {sys.version}\n")
+ f.write("=" * 80 + "\n")
+ f.write(output)
+
+ total = time.perf_counter() - overall_start
+ print(f"\n{'='*80}")
+ print(f"All benchmarks completed in {total:.1f}s")
+ print(f"Results saved in: {results_dir}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/build_binary.py b/build_binary.py
index b82c48c6e..60317896f 100644
--- a/build_binary.py
+++ b/build_binary.py
@@ -62,6 +62,8 @@ def build_binary() -> bool:
cs.PYINSTALLER_ARG_ONEFILE,
cs.PYINSTALLER_ARG_NOCONFIRM,
cs.PYINSTALLER_ARG_CLEAN,
+ cs.PYINSTALLER_ARG_COPY_METADATA,
+ cs.PACKAGE_NAME,
]
for ts_pkg in _get_treesitter_packages():
@@ -70,6 +72,9 @@ def build_binary() -> bool:
for pkg in cs.PYINSTALLER_PACKAGES:
cmd.extend(_build_package_args(pkg))
+ for mod in cs.PYINSTALLER_EXCLUDED_MODULES:
+ cmd.extend([cs.PYINSTALLER_ARG_EXCLUDE_MODULE, mod])
+
cmd.append(cs.PYINSTALLER_ENTRY_POINT)
logger.info(logs.BUILD_BINARY.format(name=binary_name))
diff --git a/cgr/__init__.py b/cgr/__init__.py
new file mode 100644
index 000000000..3d76ac771
--- /dev/null
+++ b/cgr/__init__.py
@@ -0,0 +1,14 @@
+from codebase_rag.config import settings
+from codebase_rag.embedder import embed_code
+from codebase_rag.graph_loader import GraphLoader, load_graph
+from codebase_rag.services.graph_service import MemgraphIngestor
+from codebase_rag.services.llm import CypherGenerator
+
+__all__ = [
+ "CypherGenerator",
+ "GraphLoader",
+ "MemgraphIngestor",
+ "embed_code",
+ "load_graph",
+ "settings",
+]
diff --git a/code-graph-rag-darwin-arm64.spec b/code-graph-rag-darwin-arm64.spec
new file mode 100644
index 000000000..0f2309fec
--- /dev/null
+++ b/code-graph-rag-darwin-arm64.spec
@@ -0,0 +1,77 @@
+# -*- mode: python ; coding: utf-8 -*-
+from PyInstaller.utils.hooks import collect_data_files
+from PyInstaller.utils.hooks import collect_all
+
+datas = []
+binaries = []
+hiddenimports = ['pydantic_ai_slim']
+datas += collect_data_files('pydantic_ai')
+tmp_ret = collect_all('tree_sitter_python')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_javascript')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_typescript')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_rust')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_go')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_scala')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_java')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_cpp')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('tree_sitter_lua')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('pydantic_ai')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('rich')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('typer')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('loguru')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('toml')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('protobuf')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+tmp_ret = collect_all('genai_prices')
+datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
+
+
+a = Analysis(
+ ['main.py'],
+ pathex=[],
+ binaries=binaries,
+ datas=datas,
+ hiddenimports=hiddenimports,
+ hookspath=[],
+ hooksconfig={},
+ runtime_hooks=[],
+ excludes=['logfire'],
+ noarchive=False,
+ optimize=0,
+)
+pyz = PYZ(a.pure)
+
+exe = EXE(
+ pyz,
+ a.scripts,
+ a.binaries,
+ a.datas,
+ [],
+ name='code-graph-rag-darwin-arm64',
+ debug=False,
+ bootloader_ignore_signals=False,
+ strip=False,
+ upx=True,
+ upx_exclude=[],
+ runtime_tmpdir=None,
+ console=True,
+ disable_windowed_traceback=False,
+ argv_emulation=False,
+ target_arch=None,
+ codesign_identity=None,
+ entitlements_file=None,
+)
diff --git a/codebase_rag/__init__.py b/codebase_rag/__init__.py
index e69de29bb..794085319 100644
--- a/codebase_rag/__init__.py
+++ b/codebase_rag/__init__.py
@@ -0,0 +1 @@
+"""Code Graph RAG: build a queryable graph of a codebase and reason over it."""
diff --git a/codebase_rag/analyzers/__init__.py b/codebase_rag/analyzers/__init__.py
new file mode 100644
index 000000000..875ded79b
--- /dev/null
+++ b/codebase_rag/analyzers/__init__.py
@@ -0,0 +1,4 @@
+# ast-grep finding analyzers (issue #413).
+from .ast_grep_analyzer import FindingAnalyzer, load_finding_rules
+
+__all__ = ["FindingAnalyzer", "load_finding_rules"]
diff --git a/codebase_rag/analyzers/ast_grep_analyzer.py b/codebase_rag/analyzers/ast_grep_analyzer.py
new file mode 100644
index 000000000..b2afa4911
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_analyzer.py
@@ -0,0 +1,200 @@
+# ast-grep finding analyzer (issue #413). A post-pass over indexed source
+# files that runs categorized ast-grep YAML rules and emits
+# Pattern/CodeSmell/SecurityIssue nodes linked to each file's Module. Rules
+# live in ast_grep_rules/{patterns,smells,security}/.yaml; a new rule is
+# a YAML entry, no code. The FINDINGS capture group is opt-in, so the analyzer
+# no-ops entirely unless a finding relationship is enabled. Findings link to
+# the Module (not the enclosing Class/Function); the finding node carries the
+# line so the site is still locatable. Symbol-level linkage is a follow-up.
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+from .. import constants as cs
+from ..utils.path_utils import cached_relative_path
+
+if TYPE_CHECKING:
+ from ..capture import CaptureSelection
+ from ..services import IngestorProtocol
+
+logger = logging.getLogger(__name__)
+
+_RULES_DIR = Path(__file__).parent / "ast_grep_rules"
+_SNIPPET_MAX = 200
+
+# rule-category directory name -> (finding node label, relationship type).
+_CATEGORY_MAP: dict[str, tuple[cs.NodeLabel, cs.RelationshipType]] = {
+ "patterns": (cs.NodeLabel.PATTERN, cs.RelationshipType.IMPLEMENTS_PATTERN),
+ "smells": (cs.NodeLabel.CODE_SMELL, cs.RelationshipType.HAS_SMELL),
+ "security": (cs.NodeLabel.SECURITY_ISSUE, cs.RelationshipType.HAS_VULNERABILITY),
+}
+
+
+@dataclass(frozen=True)
+class _Rule:
+ rule_id: str
+ message: str
+ body: dict[str, Any] # ast-grep rule body, splatted into find_all(**body)
+ node_label: cs.NodeLabel
+ rel_type: cs.RelationshipType
+
+
+@dataclass(frozen=True)
+class _LangRules:
+ ast_grep_id: str
+ rules: tuple[_Rule, ...]
+
+
+def load_finding_rules() -> dict[str, _LangRules]:
+ """Load ast_grep_rules//*.yaml, merged per file extension."""
+ lang_id_by_ext: dict[str, str] = {}
+ rules_by_ext: dict[str, list[_Rule]] = {}
+ for category_dir in sorted(_RULES_DIR.iterdir()):
+ mapping = _CATEGORY_MAP.get(category_dir.name)
+ if not category_dir.is_dir() or mapping is None:
+ continue
+ node_label, rel_type = mapping
+ for path in sorted(category_dir.glob("*.yaml")):
+ ast_grep_id, extensions, rules = _parse_rule_file(
+ path, node_label, rel_type
+ )
+ for ext in extensions:
+ lang_id_by_ext[ext] = ast_grep_id
+ rules_by_ext.setdefault(ext, []).extend(rules)
+ return {
+ ext: _LangRules(lang_id_by_ext[ext], tuple(rules))
+ for ext, rules in rules_by_ext.items()
+ }
+
+
+def _parse_rule_file(
+ path: Path, node_label: cs.NodeLabel, rel_type: cs.RelationshipType
+) -> tuple[str, list[str], list[_Rule]]:
+ import yaml
+
+ data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
+ ast_grep_id = data.get("ast_grep_id")
+ extensions = data.get("extensions")
+ if isinstance(extensions, str):
+ extensions = [ext.strip() for ext in extensions.split(",") if ext.strip()]
+ if not ast_grep_id or not extensions:
+ raise ValueError(f"{path.name}: 'ast_grep_id' and 'extensions' are required")
+ rules = [
+ _Rule(
+ rule_id=str(entry["id"]),
+ message=str(entry.get("message", entry["id"])),
+ body=entry["rule"],
+ node_label=node_label,
+ rel_type=rel_type,
+ )
+ for entry in (data.get("rules") or [])
+ ]
+ return str(ast_grep_id), list(extensions), rules
+
+
+class FindingAnalyzer:
+ """Runs ast-grep finding rules over indexed files and emits finding nodes."""
+
+ __slots__ = ("_ingestor", "_repo_path", "_enabled", "_rules")
+
+ def __init__(
+ self,
+ ingestor: IngestorProtocol,
+ repo_path: Path,
+ selection: CaptureSelection,
+ ) -> None:
+ self._ingestor = ingestor
+ self._repo_path = repo_path
+ finding_rels = cs.CAPTURE_GROUP_RELS[cs.CaptureGroup.FINDINGS]
+ self._enabled = any(selection.rel_enabled(rel) for rel in finding_rels)
+ self._rules: dict[str, _LangRules] = {}
+ if not self._enabled:
+ return
+ try:
+ import ast_grep_py # noqa: F401
+
+ self._rules = load_finding_rules()
+ except ImportError:
+ # ast-grep/pyyaml are the [ast-grep] extra; no-op if absent.
+ logger.warning("ast-grep-py unavailable; finding analyzer disabled")
+ except Exception as exc: # noqa: BLE001
+ # a malformed shipped rule file must not crash indexing.
+ logger.warning("ast-grep finding analyzer disabled: %s", exc)
+
+ def analyze(self, module_qn_to_file_path: dict[str, Path]) -> None:
+ if not self._enabled or not self._rules:
+ return
+ from ast_grep_py import SgRoot
+
+ for module_qn, file_path in module_qn_to_file_path.items():
+ lang_rules = self._rules.get(file_path.suffix)
+ if lang_rules is None:
+ continue
+ try:
+ raw = file_path.read_bytes()
+ except OSError:
+ continue
+ # decode with replacement so a few malformed bytes don't skip the
+ # whole file; ast-grep tolerates U+FFFD in the source.
+ source = raw.decode("utf-8", errors="replace")
+ try:
+ root = SgRoot(source, lang_rules.ast_grep_id).root()
+ except (RuntimeError, ValueError) as exc:
+ logger.warning("ast-grep failed to parse %s: %s", file_path, exc)
+ continue
+ relative_path = cached_relative_path(file_path, self._repo_path).as_posix()
+ for rule in lang_rules.rules:
+ self._emit_rule(root, rule, module_qn, relative_path, file_path)
+
+ def _emit_rule(
+ self,
+ root: Any,
+ rule: _Rule,
+ module_qn: str,
+ relative_path: str,
+ file_path: Path,
+ ) -> None:
+ try:
+ matches = root.find_all(**rule.body)
+ except (RuntimeError, TypeError, ValueError) as exc:
+ logger.warning(
+ "bad ast-grep rule %r for %s: %s", rule.rule_id, file_path, exc
+ )
+ return
+ for node in matches:
+ self._emit_finding(rule, node, module_qn, relative_path)
+
+ def _emit_finding(
+ self, rule: _Rule, node: Any, module_qn: str, relative_path: str
+ ) -> None:
+ node_range = node.range()
+ start_line = node_range.start.line + 1
+ end_line = node_range.end.line + 1
+ # qn scopes the finding to file+line+column+rule so two matches of one
+ # rule on one line stay distinct, while re-indexing merges the site.
+ qualified_name = cs.SEPARATOR_DOT.join(
+ [module_qn, str(start_line), str(node_range.start.column), rule.rule_id]
+ )
+ snippet = node.text()
+ if len(snippet) > _SNIPPET_MAX:
+ snippet = snippet[:_SNIPPET_MAX]
+ self._ingestor.ensure_node_batch(
+ rule.node_label,
+ {
+ cs.KEY_QUALIFIED_NAME: qualified_name,
+ cs.KEY_NAME: rule.rule_id,
+ cs.KEY_MESSAGE: rule.message,
+ cs.KEY_START_LINE: start_line,
+ cs.KEY_END_LINE: end_line,
+ cs.KEY_PATH: relative_path,
+ cs.KEY_SNIPPET: snippet,
+ },
+ )
+ self._ingestor.ensure_relationship_batch(
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ rule.rel_type,
+ (rule.node_label, cs.KEY_QUALIFIED_NAME, qualified_name),
+ )
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/c.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/c.yaml
new file mode 100644
index 000000000..505b2f6e5
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/c.yaml
@@ -0,0 +1,31 @@
+# ast-grep design-pattern rules for C (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. C has no classic OO patterns, so these are heuristic
+# function-name idioms recognised via the declarator identifier.
+ast_grep_id: c
+extensions: [.c, .h]
+rules:
+ - id: constructor_like
+ message: "Constructor idiom: function name starts with create/make/new/alloc"
+ rule:
+ kind: function_definition
+ has:
+ kind: function_declarator
+ stopBy: end
+ has: { field: declarator, kind: identifier, regex: '^(create|make|new|alloc)' }
+ - id: initializer_like
+ message: "Initializer idiom: function name starts with init"
+ rule:
+ kind: function_definition
+ has:
+ kind: function_declarator
+ stopBy: end
+ has: { field: declarator, kind: identifier, regex: '^init' }
+ - id: destructor_like
+ message: "Destructor idiom: function name starts with free/destroy/release"
+ rule:
+ kind: function_definition
+ has:
+ kind: function_declarator
+ stopBy: end
+ has: { field: declarator, kind: identifier, regex: '^(free|destroy|release)' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/cpp.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/cpp.yaml
new file mode 100644
index 000000000..08dae1721
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/cpp.yaml
@@ -0,0 +1,34 @@
+# ast-grep design-pattern rules for C++ (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: cpp
+extensions: [.cpp, .cc, .cxx, .hpp, .hh]
+rules:
+ - id: singleton
+ message: "Singleton: class exposes a static getInstance accessor"
+ rule:
+ kind: function_definition
+ all:
+ - has: { kind: storage_class_specifier, regex: '^static$', stopBy: end }
+ - has:
+ kind: function_declarator
+ has: { field: declarator, regex: '^getInstance$' }
+ stopBy: end
+ - id: factory_function
+ message: "Factory function: name starts with create/make/build"
+ rule:
+ kind: function_definition
+ has:
+ kind: function_declarator
+ has: { field: declarator, regex: '^(create|make|build)' }
+ stopBy: end
+ - id: abstract_class
+ message: "Abstract class: declares a pure virtual method (= 0)"
+ rule:
+ kind: class_specifier
+ has:
+ kind: field_declaration
+ all:
+ - has: { regex: '^virtual$', stopBy: end }
+ - regex: '=\s*0\s*;'
+ stopBy: end
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/csharp.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/csharp.yaml
new file mode 100644
index 000000000..7dada07db
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/csharp.yaml
@@ -0,0 +1,46 @@
+# ast-grep design-pattern rules for C# (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: csharp
+extensions: [.cs]
+rules:
+ - id: singleton
+ message: "Singleton: class exposes a static Instance member"
+ rule:
+ kind: class_declaration
+ has:
+ stopBy: end
+ any:
+ # static property named Instance / _instance
+ - kind: property_declaration
+ all:
+ - has: { kind: modifier, regex: '^static$', stopBy: end }
+ - has: { kind: identifier, regex: '^_?[Ii]nstance$', stopBy: end }
+ # static backing field named Instance / _instance
+ - kind: field_declaration
+ all:
+ - has: { kind: modifier, regex: '^static$', stopBy: end }
+ - has: { kind: identifier, regex: '^_?[Ii]nstance$', stopBy: end }
+ - id: factory_method
+ message: "Factory method: name starts with Create/Make/Build"
+ rule:
+ kind: method_declaration
+ has: { field: name, regex: '^(Create|Make|Build)' }
+ - id: builder
+ message: "Builder: class name ends with Builder"
+ rule:
+ kind: class_declaration
+ has: { field: name, regex: 'Builder$' }
+ - id: abstract_class
+ message: "Abstract base class: declared with the abstract modifier"
+ rule:
+ kind: class_declaration
+ has: { kind: modifier, regex: '^abstract$' }
+ - id: disposable
+ message: "Disposable: class defines a Dispose method"
+ rule:
+ kind: class_declaration
+ has:
+ stopBy: end
+ kind: method_declaration
+ has: { field: name, regex: '^Dispose$' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/dart.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/dart.yaml
new file mode 100644
index 000000000..93b686282
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/dart.yaml
@@ -0,0 +1,28 @@
+# ast-grep design-pattern rules for Dart (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: dart
+extensions: [.dart]
+rules:
+ - id: factory_constructor
+ message: "Factory: class exposes a factory constructor"
+ rule:
+ kind: factory_constructor_signature
+ - id: singleton
+ message: "Singleton: class holds a static instance field"
+ rule:
+ kind: class_declaration
+ has:
+ kind: static_final_declaration
+ stopBy: end
+ has: { kind: identifier, regex: '(?i)instance' }
+ - id: builder
+ message: "Builder: class name ends in Builder"
+ rule:
+ kind: class_declaration
+ has: { field: name, regex: 'Builder$' }
+ - id: abstract_class
+ message: "Abstract class declares an interface for subclasses"
+ rule:
+ kind: class_declaration
+ regex: '^abstract\s'
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/go.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/go.yaml
new file mode 100644
index 000000000..9e9a6c80e
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/go.yaml
@@ -0,0 +1,25 @@
+# ast-grep design-pattern rules for Go (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: go
+extensions: [.go]
+rules:
+ - id: constructor
+ message: "Constructor: function name starts with New"
+ rule:
+ kind: function_declaration
+ has: { field: name, regex: '^New' }
+ - id: factory_function
+ message: "Factory function: name starts with New/Make/Build/Create"
+ rule:
+ kind: function_declaration
+ has: { field: name, regex: '^(New|Make|Build|Create)' }
+ - id: singleton
+ message: "Singleton: struct embeds sync.Once to guard one-time init"
+ rule:
+ kind: struct_type
+ has: { kind: qualified_type, regex: 'sync\.Once', stopBy: end }
+ - id: interface_declaration
+ message: "Interface: a behavioural contract declared for satisfaction"
+ rule:
+ kind: interface_type
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/java.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/java.yaml
new file mode 100644
index 000000000..423183ab1
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/java.yaml
@@ -0,0 +1,37 @@
+# ast-grep design-pattern rules for Java (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: java
+extensions: [.java]
+rules:
+ - id: singleton
+ message: "Singleton: class exposes a static getInstance accessor"
+ rule:
+ kind: class_declaration
+ has:
+ kind: method_declaration
+ stopBy: end
+ all:
+ - has: { kind: modifiers, has: { regex: '^static$' } }
+ - has: { field: name, regex: '^getInstance$' }
+ - id: factory_method
+ message: "Factory method: name starts with create/make/build/newInstance"
+ rule:
+ kind: method_declaration
+ has: { field: name, regex: '^(create|make|build|newInstance)' }
+ - id: builder
+ message: "Builder: class name ends with Builder"
+ rule:
+ kind: class_declaration
+ has: { field: name, regex: 'Builder$' }
+ - id: fluent_builder
+ message: "Fluent builder: instance method returns this"
+ rule:
+ kind: method_declaration
+ has:
+ kind: block
+ has:
+ kind: return_statement
+ has: { kind: this }
+ stopBy: end
+ stopBy: end
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/javascript.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/javascript.yaml
new file mode 100644
index 000000000..62088a29d
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/javascript.yaml
@@ -0,0 +1,28 @@
+# ast-grep design-pattern rules for JavaScript (issue #413).
+ast_grep_id: javascript
+extensions: [.js, .jsx, .mjs, .cjs]
+rules:
+ - id: singleton
+ message: "Singleton: class exposes a static getInstance()"
+ rule:
+ kind: method_definition
+ has: { field: name, regex: '^getInstance$' }
+ - id: promise_constructor
+ message: "Promise constructor: manual new Promise usage"
+ rule:
+ pattern: "new Promise($$$)"
+ - id: factory_function
+ message: "Factory function: name starts with create/make/build"
+ rule:
+ any:
+ - kind: function_declaration
+ has: { field: name, regex: '^(create|make|build)' }
+ - kind: variable_declarator
+ all:
+ - has: { field: name, regex: '^(create|make|build)' }
+ - has:
+ field: value
+ any:
+ - kind: arrow_function
+ - kind: function_expression
+ - kind: generator_function
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/lua.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/lua.yaml
new file mode 100644
index 000000000..21b5cc2e7
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/lua.yaml
@@ -0,0 +1,56 @@
+# ast-grep design-pattern rules for Lua (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: lua
+extensions: [.lua]
+rules:
+ - id: metatable_oop
+ message: "Metatable OOP: setmetatable() wires a table to its class table"
+ rule:
+ kind: function_call
+ has: { kind: identifier, regex: '^setmetatable$' }
+ - id: index_metatable
+ message: "Class table: __index assigned for prototype-style inheritance"
+ rule:
+ kind: assignment_statement
+ has:
+ kind: dot_index_expression
+ has: { field: field, regex: '^__index$' }
+ stopBy: end
+ - id: factory_function
+ # Decide on the function NAME only. The old `stopBy: end` scanned the whole
+ # body, so a function merely calling `results.new(...)` was flagged. Match a
+ # bare name (`function create()`) or the last segment of a dotted name
+ # (`function results.new()`), never an identifier deep in the body.
+ message: "Factory function: name starts with new/create/make"
+ rule:
+ kind: function_declaration
+ has:
+ any:
+ - all:
+ - kind: identifier
+ - regex: '^(new|create|make)'
+ - all:
+ - kind: dot_index_expression
+ - has: { field: field, regex: '^(new|create|make)' }
+ # Colon syntax `function Class:new()` is the idiomatic Lua constructor;
+ # its name is a method_index_expression, not a dot_index_expression.
+ - all:
+ - kind: method_index_expression
+ - has: { field: method, regex: '^(new|create|make)' }
+ - id: module_return
+ # The idiomatic module export is the TOP-LEVEL trailing `return M`, not
+ # every `return x` inside a function; exclude returns nested in a function.
+ message: "Module pattern: chunk returns a single table variable"
+ rule:
+ kind: return_statement
+ all:
+ - has:
+ kind: expression_list
+ has: { kind: identifier, nthChild: 1 }
+ - not:
+ inside:
+ any:
+ - kind: function_declaration
+ - kind: function_definition
+ stopBy: end
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/php.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/php.yaml
new file mode 100644
index 000000000..1a1060ef8
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/php.yaml
@@ -0,0 +1,31 @@
+# ast-grep design-pattern rules for PHP (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: php
+extensions: [.php]
+rules:
+ - id: singleton
+ message: "Singleton: class exposes a static getInstance accessor"
+ rule:
+ kind: class_declaration
+ has:
+ kind: method_declaration
+ stopBy: end
+ all:
+ - has: { kind: static_modifier }
+ - has: { field: name, regex: '^getInstance$' }
+ - id: factory_function
+ message: "Factory function: name starts with create/make/build"
+ rule:
+ kind: function_definition
+ has: { field: name, regex: '^(create|make|build)' }
+ - id: factory_method
+ message: "Factory method: name starts with create/make/build"
+ rule:
+ kind: method_declaration
+ has: { field: name, regex: '^(create|make|build)' }
+ - id: abstract_class
+ message: "Abstract class: declared with the abstract modifier"
+ rule:
+ kind: class_declaration
+ has: { kind: abstract_modifier }
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/python.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/python.yaml
new file mode 100644
index 000000000..562e12101
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/python.yaml
@@ -0,0 +1,49 @@
+# ast-grep design-pattern rules for Python (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: python
+extensions: [.py]
+rules:
+ - id: singleton
+ message: "Singleton: class holds a private _instance"
+ rule:
+ kind: class_definition
+ has:
+ pattern: "_instance = None"
+ stopBy: end
+ - id: context_manager
+ message: "Context manager: class defines __enter__ and __exit__"
+ rule:
+ kind: class_definition
+ all:
+ - has:
+ kind: function_definition
+ has: { field: name, regex: '^__enter__$' }
+ stopBy: end
+ - has:
+ kind: function_definition
+ has: { field: name, regex: '^__exit__$' }
+ stopBy: end
+ - id: iterator
+ message: "Iterator: class defines __iter__ and __next__"
+ rule:
+ kind: class_definition
+ all:
+ - has:
+ kind: function_definition
+ has: { field: name, regex: '^__iter__$' }
+ stopBy: end
+ - has:
+ kind: function_definition
+ has: { field: name, regex: '^__next__$' }
+ stopBy: end
+ - id: abstract_base
+ message: "Abstract base class: inherits ABC"
+ rule:
+ kind: class_definition
+ has: { field: superclasses, regex: 'ABC' }
+ - id: factory_function
+ message: "Factory function: name starts with make_/create_/build_"
+ rule:
+ kind: function_definition
+ has: { field: name, regex: '^(make|create|build)_' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/ruby.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/ruby.yaml
new file mode 100644
index 000000000..e3028372b
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/ruby.yaml
@@ -0,0 +1,29 @@
+# ast-grep design-pattern rules for Ruby (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: ruby
+extensions: [.rb]
+rules:
+ - id: singleton
+ message: "Singleton: class mixes in the Singleton module"
+ rule:
+ pattern: "include Singleton"
+ - id: factory_method
+ message: "Factory: class method named create/build/make"
+ rule:
+ any:
+ # def self.create ...
+ - kind: singleton_method
+ has: { field: name, regex: '^(create|build|make)' }
+ # class << self; def create ...
+ - kind: method
+ all:
+ - has: { field: name, regex: '^(create|build|make)' }
+ - inside: { kind: singleton_class, stopBy: end }
+ - id: module_mixin
+ message: "Module mixin: behavior shared via include/extend"
+ rule:
+ kind: call
+ has:
+ field: method
+ regex: '^(include|extend|prepend)$'
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/rust.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/rust.yaml
new file mode 100644
index 000000000..70d8f3395
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/rust.yaml
@@ -0,0 +1,27 @@
+# ast-grep design-pattern rules for Rust (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: rust
+extensions: [.rs]
+rules:
+ - id: constructor_new
+ message: "Constructor: associated fn new returning Self"
+ rule:
+ kind: function_item
+ all:
+ - has: { field: name, regex: '^new$' }
+ - has: { field: return_type, regex: '^Self$' }
+ - id: factory_function
+ message: "Factory function: name starts with new/build/create/make"
+ rule:
+ kind: function_item
+ has: { field: name, regex: '^(new|build|create|make)' }
+ - id: trait_impl
+ message: "Trait implementation: impl Trait for Type"
+ rule:
+ kind: impl_item
+ has: { field: trait, regex: '.' }
+ - id: trait_definition
+ message: "Trait definition declares an interface"
+ rule:
+ kind: trait_item
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/scala.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/scala.yaml
new file mode 100644
index 000000000..258ba9571
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/scala.yaml
@@ -0,0 +1,25 @@
+# ast-grep design-pattern rules for Scala (issue #413).
+# Each match becomes a Pattern node linked to the file's Module via
+# IMPLEMENTS_PATTERN. Rules are heuristic and syntactic.
+ast_grep_id: scala
+extensions: [.scala, .sc]
+rules:
+ - id: singleton_object
+ message: "Singleton: object declares a single shared instance"
+ rule:
+ kind: object_definition
+ - id: case_class
+ message: "Case class: immutable value type with generated equality"
+ rule:
+ kind: class_definition
+ regex: '^case\s+class'
+ - id: companion_factory
+ message: "Factory: apply method builds instances of the type"
+ rule:
+ kind: function_definition
+ has: { field: name, regex: '^apply$' }
+ - id: factory_method
+ message: "Factory method: name starts with make/create/build"
+ rule:
+ kind: function_definition
+ has: { field: name, regex: '^(make|create|build)' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/tsx.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/tsx.yaml
new file mode 100644
index 000000000..96a54fec5
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/tsx.yaml
@@ -0,0 +1,28 @@
+# ast-grep design-pattern rules for TSX (issue #413).
+ast_grep_id: tsx
+extensions: [.tsx]
+rules:
+ - id: singleton
+ message: "Singleton: class exposes a static getInstance()"
+ rule:
+ kind: method_definition
+ has: { field: name, regex: '^getInstance$' }
+ - id: promise_constructor
+ message: "Promise constructor: manual new Promise usage"
+ rule:
+ pattern: "new Promise($$$)"
+ - id: factory_function
+ message: "Factory function: name starts with create/make/build"
+ rule:
+ any:
+ - kind: function_declaration
+ has: { field: name, regex: '^(create|make|build)' }
+ - kind: variable_declarator
+ all:
+ - has: { field: name, regex: '^(create|make|build)' }
+ - has:
+ field: value
+ any:
+ - kind: arrow_function
+ - kind: function_expression
+ - kind: generator_function
diff --git a/codebase_rag/analyzers/ast_grep_rules/patterns/typescript.yaml b/codebase_rag/analyzers/ast_grep_rules/patterns/typescript.yaml
new file mode 100644
index 000000000..a2c4ff9f9
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/patterns/typescript.yaml
@@ -0,0 +1,28 @@
+# ast-grep design-pattern rules for TypeScript (issue #413).
+ast_grep_id: typescript
+extensions: [.ts, .mts, .cts]
+rules:
+ - id: singleton
+ message: "Singleton: class exposes a static getInstance()"
+ rule:
+ kind: method_definition
+ has: { field: name, regex: '^getInstance$' }
+ - id: promise_constructor
+ message: "Promise constructor: manual new Promise usage"
+ rule:
+ pattern: "new Promise($$$)"
+ - id: factory_function
+ message: "Factory function: name starts with create/make/build"
+ rule:
+ any:
+ - kind: function_declaration
+ has: { field: name, regex: '^(create|make|build)' }
+ - kind: variable_declarator
+ all:
+ - has: { field: name, regex: '^(create|make|build)' }
+ - has:
+ field: value
+ any:
+ - kind: arrow_function
+ - kind: function_expression
+ - kind: generator_function
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/c.yaml b/codebase_rag/analyzers/ast_grep_rules/security/c.yaml
new file mode 100644
index 000000000..07cb635a4
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/c.yaml
@@ -0,0 +1,35 @@
+# ast-grep security rules for C (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules favour recall; treat findings as review prompts.
+ast_grep_id: c
+extensions: [.c, .h]
+rules:
+ - id: dangerous_string_fn
+ message: "Unbounded string function (gets/strcpy/strcat/sprintf) risks buffer overflow"
+ rule:
+ kind: call_expression
+ has: { field: function, regex: '^(gets|strcpy|strcat|sprintf|vsprintf|scanf)$' }
+ - id: command_execution
+ message: "Possible command injection via system/popen/exec* call"
+ rule:
+ kind: call_expression
+ has: { field: function, regex: '^(system|popen|execl|execlp|execle|execv|execvp|execvpe)$' }
+ - id: hardcoded_secret
+ # Security rule: favour recall. Target name matches a credential word and the
+ # value is a non-trivial string literal (>=10 chars) that is not a printf-style
+ # format template. Handles both `char *password = "..."` (pointer_declarator)
+ # and `T token = "..."` (plain identifier) via a stopBy-end identifier probe.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: init_declarator
+ all:
+ - has: { stopBy: end, kind: identifier, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: value
+ all:
+ - kind: string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/cpp.yaml b/codebase_rag/analyzers/ast_grep_rules/security/cpp.yaml
new file mode 100644
index 000000000..b91e89dc5
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/cpp.yaml
@@ -0,0 +1,53 @@
+# ast-grep security rules for C++ (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: cpp
+extensions: [.cpp, .cc, .cxx, .hpp, .hh]
+rules:
+ - id: system_call
+ message: "Possible command injection: system() executes a shell command"
+ rule:
+ kind: call_expression
+ has: { field: function, regex: '^system$' }
+ - id: unsafe_string_function
+ message: "Unsafe C string function; can overflow the destination buffer"
+ rule:
+ kind: call_expression
+ has:
+ field: function
+ regex: '^(strcpy|strcat|sprintf|gets|scanf|strncpy)$'
+ - id: hardcoded_secret
+ # Security rule: favour recall. Value must be a non-trivial string literal
+ # (>=10 chars incl. quotes) that is not a format/message template (a
+ # printf %s/%d specifier or a {..} placeholder). Braces and % conversions
+ # are the only exclusions so real secrets that contain %, spaces, URLs with
+ # embedded credentials, or SCREAMING_SNAKE shapes are still caught. Without
+ # entropy scoring, identifier/word literals still surface as review prompts;
+ # that residual is the known limit of regex-only secret detection.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ any:
+ - kind: init_declarator
+ all:
+ - has: { field: declarator, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: value
+ all:
+ - kind: string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
+ - kind: assignment_expression
+ all:
+ - has: { field: left, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: right
+ all:
+ - kind: string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/csharp.yaml b/codebase_rag/analyzers/ast_grep_rules/security/csharp.yaml
new file mode 100644
index 000000000..653cd57b9
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/csharp.yaml
@@ -0,0 +1,83 @@
+# ast-grep security rules for C# (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: csharp
+extensions: [.cs]
+rules:
+ - id: process_start
+ message: "Process.Start can execute external commands; validate inputs"
+ rule:
+ pattern: "Process.Start($$$)"
+ - id: sql_injection
+ message: "Possible SQL injection: SqlCommand text built with concatenation or interpolation"
+ rule:
+ any:
+ # new SqlCommand("SELECT " + x)
+ - kind: object_creation_expression
+ all:
+ - has: { kind: identifier, regex: 'SqlCommand', stopBy: end }
+ - has:
+ stopBy: end
+ any:
+ - kind: binary_expression
+ - kind: interpolated_string_expression
+ # cmd.CommandText = "SELECT * FROM ..." + x; (require a real SQL shape so
+ # an unrelated obj.CommandText = "Choose " + x is not misflagged)
+ - kind: assignment_expression
+ all:
+ - has: { regex: 'CommandText', stopBy: end }
+ - has:
+ stopBy: end
+ any:
+ - kind: binary_expression
+ - kind: interpolated_string_expression
+ - has:
+ stopBy: end
+ regex: '(?i)(select\b.*\bfrom\b|insert\b.*\binto\b|update\b.*\bset\b|delete\b.*\bfrom\b)'
+ - id: hardcoded_secret
+ # Security rule: favour recall. A credential-named field, property, or
+ # assignment target bound to a non-trivial (>=8 char) plain string literal.
+ # Name regex intentionally broadened beyond the literal api_key spelling to
+ # the idiomatic C# ApiKey/apiKey forms; still contains password|secret|
+ # api_key|token. Format templates ({..} placeholders, printf %s/%d) are the
+ # only value exclusions so URLs-with-credentials and SCREAMING_SNAKE shapes
+ # still surface. Identifier/word literals remain the known regex-only limit.
+ message: "Hardcoded secret assigned to a credential-named member"
+ rule:
+ any:
+ # field: private string password = "....";
+ - kind: variable_declarator
+ all:
+ - has: { kind: identifier, regex: '(?i)(password|passwd|secret|api_?key|token)' }
+ - has:
+ kind: string_literal
+ all:
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
+ # property: public string ApiKey { get; set; } = "....";
+ - kind: property_declaration
+ all:
+ - has: { field: name, regex: '(?i)(password|passwd|secret|api_?key|token)' }
+ - has:
+ kind: string_literal
+ all:
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
+ # assignment: token = "....";
+ - kind: assignment_expression
+ all:
+ - has: { kind: identifier, regex: '(?i)(password|passwd|secret|api_?key|token)' }
+ - has:
+ kind: string_literal
+ all:
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/dart.yaml b/codebase_rag/analyzers/ast_grep_rules/security/dart.yaml
new file mode 100644
index 000000000..a35d58551
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/dart.yaml
@@ -0,0 +1,45 @@
+# ast-grep security rules for Dart (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: dart
+extensions: [.dart]
+rules:
+ - id: process_exec
+ # Dart's ast-grep pattern parser does not resolve receiver.method() call
+ # patterns, so match the call_expression by its source text instead.
+ message: "Process.run/start may enable command injection"
+ rule:
+ kind: call_expression
+ regex: '^Process\.(run|start|runSync|startSync)\s*\('
+ - id: sqli_concat
+ # Anchored to an actual database call (rawQuery/execute/query/...) whose
+ # argument is built by string concatenation. Tying the rule to a sink, rather
+ # than to SQL-looking string content, keeps ordinary prose that merely
+ # contains words like "select ... from" from being misflagged.
+ message: "Possible SQL injection: query built with string concatenation"
+ rule:
+ kind: call_expression
+ all:
+ - regex: '(?i)\b(rawQuery|rawInsert|rawUpdate|rawDelete|execute|query)\s*\('
+ - has:
+ kind: additive_expression
+ has: { kind: string_literal, stopBy: end }
+ stopBy: end
+ - id: hardcoded_secret
+ # Security rule: favour recall. A field or local variable whose name matches
+ # a credential shape bound to a non-trivial (>=8 char incl. quotes) string
+ # literal. Both the class-field form (initialized_identifier) and the local
+ # form (initialized_variable_definition) are covered. Without entropy
+ # scoring, config-name-shaped literals still surface as review prompts; that
+ # residual is the known limit of regex-only secret detection, not a defect.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ any:
+ - kind: initialized_identifier
+ all:
+ - has: { kind: identifier, regex: '(?i)(password|secret|api_key|token)' }
+ - has: { kind: string_literal, regex: '^.{10,}$' }
+ - kind: initialized_variable_definition
+ all:
+ - has: { kind: identifier, regex: '(?i)(password|secret|api_key|token)' }
+ - has: { kind: string_literal, regex: '^.{10,}$' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/go.yaml b/codebase_rag/analyzers/ast_grep_rules/security/go.yaml
new file mode 100644
index 000000000..fb3491486
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/go.yaml
@@ -0,0 +1,127 @@
+# ast-grep security rules for Go (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic and favour recall; treat findings as
+# review prompts.
+ast_grep_id: go
+extensions: [.go]
+rules:
+ - id: exec_command
+ message: "External command execution via os/exec.Command"
+ rule:
+ kind: call_expression
+ has:
+ field: function
+ kind: selector_expression
+ all:
+ - has: { field: operand, regex: '^exec$' }
+ - has: { field: field, regex: '^Command' }
+ - id: sqli_concat
+ # Sink-anchored: the Query/Exec selector must be THIS call's function (a
+ # direct child, not a nested one) and the `+` concatenation must live in
+ # THIS call's argument_list. Without that anchoring a benign
+ # `c.Header().Set("d", a + url.QueryEscape(b))` matched (QueryEscape is a
+ # net/url false friend for `^Query`, and its `+` belonged to Set). The
+ # explicit exclusion of Query(Escape|Unescape) covers the direct
+ # `url.QueryEscape("a" + b)` case since the Rust regex engine has no
+ # lookaround.
+ message: "Possible SQL injection: Query/Exec argument built with string concatenation"
+ rule:
+ kind: call_expression
+ all:
+ - has:
+ kind: selector_expression
+ has:
+ field: field
+ all:
+ - regex: '^(Query|Exec)'
+ - not: { regex: '^Query(Escape|Unescape)' }
+ - has:
+ kind: argument_list
+ has:
+ kind: binary_expression
+ has: { field: operator, regex: '\+', stopBy: end }
+ stopBy: end
+ - id: hardcoded_secret
+ # Security rule: favour recall. Value must be a non-trivial literal (>=8
+ # content chars) that is not a format/message template (a Sprintf {..}
+ # placeholder or a printf %s/%d specifier). Braces and % conversions are the
+ # only exclusions so real secrets that legitimately contain %, spaces, or
+ # URLs with embedded credentials are still caught. Covers :=, =, and
+ # var/const declarations. Ceiling: without entropy scoring, plain word/URL
+ # literals still surface as review prompts; that residual is the known limit
+ # of regex-only secret detection, not a defect.
+ # The string literal must be the ASSIGNED VALUE itself, reached from the
+ # right-hand expression_list without crossing a call boundary
+ # (`stopBy: {kind: call_expression}`). This traverses through
+ # parenthesized_expression wrappers so `token := ("literal")` is caught,
+ # while killing the false-positive class where the value is a function call
+ # carrying a long string argument, e.g. `token := getEnv("SOME_LONG_DEFAULT")`.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ any:
+ - kind: short_var_declaration
+ all:
+ - has: { field: left, regex: '(?i)(password|passwd|secret|api_?key|apikey|token|access_?key)' }
+ - has:
+ field: right
+ kind: expression_list
+ has:
+ stopBy: { kind: call_expression }
+ all:
+ - any:
+ - kind: interpreted_string_literal
+ - kind: raw_string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoqvtbc]'
+ - kind: assignment_statement
+ all:
+ - has: { field: left, regex: '(?i)(password|passwd|secret|api_?key|apikey|token|access_?key)' }
+ - has:
+ field: right
+ kind: expression_list
+ has:
+ stopBy: { kind: call_expression }
+ all:
+ - any:
+ - kind: interpreted_string_literal
+ - kind: raw_string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoqvtbc]'
+ - kind: var_spec
+ all:
+ - has: { field: name, regex: '(?i)(password|passwd|secret|api_?key|apikey|token|access_?key)' }
+ - has:
+ kind: expression_list
+ has:
+ stopBy: { kind: call_expression }
+ all:
+ - any:
+ - kind: interpreted_string_literal
+ - kind: raw_string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoqvtbc]'
+ - kind: const_spec
+ all:
+ - has: { field: name, regex: '(?i)(password|passwd|secret|api_?key|apikey|token|access_?key)' }
+ - has:
+ kind: expression_list
+ has:
+ stopBy: { kind: call_expression }
+ all:
+ - any:
+ - kind: interpreted_string_literal
+ - kind: raw_string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoqvtbc]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/java.yaml b/codebase_rag/analyzers/ast_grep_rules/security/java.yaml
new file mode 100644
index 000000000..24fa5394d
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/java.yaml
@@ -0,0 +1,57 @@
+# ast-grep security rules for Java (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: java
+extensions: [.java]
+rules:
+ - id: runtime_exec
+ # Require a Runtime receiver so an unrelated obj.exec() is not misflagged.
+ message: "Runtime.exec() can enable command injection"
+ rule:
+ kind: method_invocation
+ all:
+ - has: { field: name, regex: '^exec$' }
+ # Constrain to the receiver so a string arg mentioning "runtime" or an
+ # unrelated obj.exec() is not misflagged.
+ - has: { field: object, regex: '(?i)runtime', stopBy: end }
+ - id: process_builder
+ message: "ProcessBuilder spawns an external process; validate its input"
+ rule:
+ pattern: "new ProcessBuilder($$$)"
+ - id: sqli_concat
+ message: "Possible SQL injection: query built with string concatenation"
+ rule:
+ kind: method_invocation
+ all:
+ - has: { field: name, regex: '^(executeQuery|executeUpdate|execute|prepareStatement)$' }
+ - has:
+ kind: binary_expression
+ has: { kind: string_literal, stopBy: end }
+ stopBy: end
+ - id: hardcoded_secret
+ # Security rule: favour recall. Any variable/field whose name matches a
+ # credential word and is bound to a non-trivial string literal (>=8 inner
+ # chars, i.e. >=10 including the surrounding quotes) surfaces as a review
+ # prompt. Format/message templates (a "{..}" placeholder or a printf
+ # %s/%d specifier) are excluded so only literal-looking secrets remain.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: variable_declarator
+ all:
+ - has: { field: name, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: value
+ all:
+ - kind: string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
+ - id: md5_digest
+ # Both the uppercase and lowercase spellings are common in the wild.
+ message: "Weak hash algorithm (MD5) requested"
+ rule:
+ any:
+ - pattern: 'MessageDigest.getInstance("MD5")'
+ - pattern: 'MessageDigest.getInstance("md5")'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/javascript.yaml b/codebase_rag/analyzers/ast_grep_rules/security/javascript.yaml
new file mode 100644
index 000000000..411d1211f
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/javascript.yaml
@@ -0,0 +1,35 @@
+# ast-grep security rules for JavaScript (issue #413).
+ast_grep_id: javascript
+extensions: [.js, .jsx, .mjs, .cjs]
+rules:
+ - id: innerhtml_xss
+ message: "Possible XSS: assignment to innerHTML/outerHTML"
+ rule:
+ any:
+ - pattern: "$EL.innerHTML = $X"
+ - pattern: "$EL.innerHTML += $X"
+ - pattern: "$EL.outerHTML = $X"
+ - pattern: "$EL.outerHTML += $X"
+ - id: eval_use
+ message: "eval() executes arbitrary code"
+ rule:
+ pattern: "eval($$$)"
+ - id: document_write
+ message: "document.write() can enable XSS"
+ rule:
+ pattern: "document.write($$$)"
+ - id: hardcoded_secret
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: variable_declarator
+ all:
+ - has: { field: name, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: value
+ all:
+ - kind: string
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/lua.yaml b/codebase_rag/analyzers/ast_grep_rules/security/lua.yaml
new file mode 100644
index 000000000..9e8cc4869
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/lua.yaml
@@ -0,0 +1,47 @@
+# ast-grep security rules for Lua (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: lua
+extensions: [.lua]
+rules:
+ - id: dynamic_code
+ message: "load()/loadstring() executes arbitrary code at runtime"
+ rule:
+ kind: function_call
+ has: { kind: identifier, regex: '^(load|loadstring)$' }
+ - id: os_command
+ message: "Possible command injection: os.execute()/io.popen() runs a shell command"
+ rule:
+ kind: function_call
+ has:
+ kind: dot_index_expression
+ regex: '^(os\.execute|io\.popen)$'
+ - id: dofile_use
+ message: "dofile()/loadfile() loads and runs an external file"
+ rule:
+ kind: function_call
+ has: { kind: identifier, regex: '^(dofile|loadfile)$' }
+ - id: hardcoded_secret
+ # Security rule: favour recall. Target name matches a credential keyword and
+ # the value is a non-trivial string literal. Covers both `local secret = ...`
+ # (assignment wrapped in variable_declaration) and bare `secret = ...` globals
+ # because both share the assignment_statement node.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: assignment_statement
+ all:
+ - has:
+ kind: variable_list
+ has: { kind: identifier, regex: '(?i)(password|secret|api_key|token)', stopBy: end }
+ # The string must be the assigned VALUE, reached from the
+ # expression_list without crossing a function_call boundary. This
+ # traverses parenthesized_expression wrappers (so `password = ("...")`
+ # is caught) while rejecting a string buried in a call argument, e.g.
+ # the destructuring `password = creds:match("([^:]*):(.*)")` whose
+ # pattern string lives inside the match() arguments, not the value.
+ - has:
+ kind: expression_list
+ has:
+ kind: string
+ regex: '^.{8,}$'
+ stopBy: { kind: function_call }
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/php.yaml b/codebase_rag/analyzers/ast_grep_rules/security/php.yaml
new file mode 100644
index 000000000..03630e8e6
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/php.yaml
@@ -0,0 +1,59 @@
+# ast-grep security rules for PHP (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic and favour recall; treat findings as
+# review prompts.
+ast_grep_id: php
+extensions: [.php]
+rules:
+ - id: eval_use
+ message: "eval() executes arbitrary code"
+ rule:
+ kind: function_call_expression
+ has: { field: function, regex: '^eval$' }
+ - id: command_exec
+ # Shell/process execution sinks: user-controlled args risk command injection.
+ message: "Command execution sink (system/exec/shell_exec/passthru/proc_open/popen)"
+ rule:
+ kind: function_call_expression
+ has:
+ field: function
+ regex: '^(system|exec|shell_exec|passthru|proc_open|popen)$'
+ - id: sqli_concat
+ # SQL built by string concatenation (binary '.') passed to a query sink.
+ # Favour recall: match either a bare mysqli_query call or any ->query() method
+ # whose argument list contains a concatenation.
+ message: "Possible SQL injection: query built with string concatenation"
+ rule:
+ any:
+ - kind: function_call_expression
+ all:
+ - has: { field: function, regex: '(?i)^(mysqli_query|pg_query|sqlsrv_query|db_query)$' }
+ - has: { kind: binary_expression, stopBy: end }
+ - kind: member_call_expression
+ all:
+ - has: { field: name, regex: '(?i)^query$' }
+ - has: { kind: binary_expression, stopBy: end }
+ - id: hardcoded_secret
+ # Security rule: favour recall. Value must be a non-trivial literal (>=10
+ # chars) that is not a format/message template (a {..} placeholder or a
+ # printf-style %s/%d specifier). Those two exclusions are the only ones, so
+ # real secrets containing %, spaces, or URLs with embedded credentials still
+ # surface. Ceiling: without entropy scoring, ordinary word/URL literals bound
+ # to a credential-named variable still surface as review prompts; that
+ # residual is the known limit of regex-only secret detection, not a defect.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: assignment_expression
+ all:
+ - has: { field: left, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: right
+ all:
+ - any:
+ - kind: encapsed_string # double-quoted
+ - kind: string # single-quoted
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/python.yaml b/codebase_rag/analyzers/ast_grep_rules/security/python.yaml
new file mode 100644
index 000000000..b87795219
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/python.yaml
@@ -0,0 +1,71 @@
+# ast-grep security rules for Python (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: python
+extensions: [.py]
+rules:
+ - id: sqli_concat
+ message: "Possible SQL injection: query built with string concatenation or % formatting"
+ rule:
+ any:
+ - pattern: "$OBJ.execute($A + $B)"
+ # (%) matches only when a string literal is present, so a numeric modulo
+ # like execute(index % shard_count) is not mistaken for query formatting.
+ - all:
+ - pattern: "$OBJ.execute($A % $B)"
+ - has: { kind: string, stopBy: end }
+ - id: sqli_fstring
+ message: "Possible SQL injection: execute() called with an f-string"
+ rule:
+ kind: call
+ all:
+ - has: { pattern: "execute", stopBy: end }
+ - has: { kind: string, regex: '^f', stopBy: end }
+ - id: hardcoded_secret
+ # Security rule: favour recall. Value must be a non-trivial literal (>=8
+ # chars) that is not a format/message template (an f-string/.format {..}
+ # placeholder or a printf %s/%d specifier). Braces and % conversions are the
+ # only exclusions so real secrets that legitimately contain %, spaces, URLs
+ # with embedded credentials, or SCREAMING_SNAKE shapes are still caught.
+ # Ceiling: without entropy scoring, identifier/word/URL-name literals (e.g.
+ # "OPENAI_API_KEY", "cl100k_base") still surface as review prompts; that
+ # residual is the known limit of regex-only secret detection, not a defect.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: assignment
+ all:
+ - has: { field: left, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: right
+ all:
+ - kind: string
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
+ - id: eval_use
+ message: "eval() executes arbitrary code"
+ rule:
+ pattern: "eval($$$)"
+ - id: exec_use
+ message: "exec() executes arbitrary code"
+ rule:
+ pattern: "exec($$$)"
+ - id: os_system_concat
+ message: "Possible command injection: os.system() with concatenation"
+ rule:
+ pattern: "os.system($A + $B)"
+ - id: subprocess_shell
+ message: "subprocess called with shell=True enables command injection"
+ rule:
+ pattern: "subprocess.$M($$$)"
+ has: { regex: 'shell\s*=\s*True', stopBy: end }
+ - id: yaml_load
+ message: "yaml.load() without SafeLoader can execute arbitrary code"
+ rule:
+ pattern: "yaml.load($$$)"
+ - id: pickle_loads
+ message: "pickle.loads() on untrusted data executes arbitrary code"
+ rule:
+ pattern: "pickle.loads($$$)"
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/ruby.yaml b/codebase_rag/analyzers/ast_grep_rules/security/ruby.yaml
new file mode 100644
index 000000000..fd09e68af
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/ruby.yaml
@@ -0,0 +1,65 @@
+# ast-grep security rules for Ruby (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic and favour recall; treat findings as
+# review prompts.
+ast_grep_id: ruby
+extensions: [.rb]
+rules:
+ - id: eval_use
+ message: "eval/instance_eval/class_eval executes arbitrary code"
+ rule:
+ kind: call
+ has:
+ field: method
+ regex: '^(eval|instance_eval|class_eval|module_eval)$'
+ - id: os_command
+ message: "Possible command injection: shell command via system/exec/backticks/%x"
+ rule:
+ any:
+ # backticks `...` and %x{...} both parse as subshell
+ - kind: subshell
+ - kind: call
+ has:
+ field: method
+ regex: '^(system|exec|spawn)$'
+ - id: sql_injection
+ message: "Possible SQL injection: query built with interpolation or concatenation"
+ rule:
+ kind: call
+ all:
+ - has:
+ field: method
+ regex: '^(execute|exec_query|select_all|find_by_sql|query|where|find_by)$'
+ - has:
+ stopBy: end
+ any:
+ # string with #{...} interpolation
+ - kind: interpolation
+ # "..." + var concatenation
+ - kind: binary
+ has:
+ kind: string
+ stopBy: end
+ - id: hardcoded_secret
+ # Security rule: favour recall. Target name matches a credential keyword and
+ # the value is a non-trivial (>=8 char) string literal that is not a
+ # format/message template (an #{..} interpolation placeholder or a printf
+ # %s/%d specifier). Those two exclusions keep real secrets that contain %,
+ # spaces, URLs with embedded credentials, or SCREAMING_SNAKE shapes.
+ # Ceiling: without entropy scoring, identifier/word-shaped literals still
+ # surface as review prompts; that residual is the known limit of regex-only
+ # secret detection, not a defect.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: assignment
+ all:
+ - has: { field: left, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: right
+ all:
+ - kind: string
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - has: { kind: interpolation, stopBy: end }
+ - regex: '%[sdrifeEgGxXoc]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/rust.yaml b/codebase_rag/analyzers/ast_grep_rules/security/rust.yaml
new file mode 100644
index 000000000..47ff3ee58
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/rust.yaml
@@ -0,0 +1,78 @@
+# ast-grep security rules for Rust (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: rust
+extensions: [.rs]
+rules:
+ - id: command_new
+ # Anchor to the call's own `function` field so only the call whose callee IS
+ # Command::new fires. The old `stopBy: end` searched every descendant, so a
+ # call that merely wraps a Command::new argument (`foo(Command::new(x))`) or
+ # any ancestor in a builder chain matched too, emitting a phantom finding on
+ # the outer call. A parenthesized callee (`(Command::new)("ls")`) is a
+ # parenthesized_expression, so accept that shape too (recall). A method-chain
+ # callee is a field_expression, which is deliberately NOT matched, so
+ # `.arg(...)` ancestors do not re-fire. std::process::Command::new still
+ # matches via the substring regex.
+ message: "Spawning an external process (Command::new) can enable command injection"
+ rule:
+ kind: call_expression
+ has:
+ field: function
+ any:
+ - kind: scoped_identifier
+ regex: 'Command\s*::\s*new'
+ - kind: parenthesized_expression
+ has:
+ kind: scoped_identifier
+ regex: 'Command\s*::\s*new'
+ - id: unsafe_block
+ message: "unsafe block bypasses Rust memory-safety guarantees"
+ rule:
+ kind: unsafe_block
+ - id: hardcoded_secret
+ # Security rule: favour recall. Target name matches a credential keyword and
+ # the value is a non-trivial string literal (>=10 chars incl. quotes) that is
+ # not a format template (a {..} placeholder or a printf-style specifier).
+ # Covers both `let` bindings and reassignments. Identifier/word/URL-name
+ # literals still surface as review prompts; that residual is the known limit
+ # of regex-only secret detection, not a defect.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ any:
+ - kind: let_declaration
+ all:
+ - has: { field: pattern, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: value
+ all:
+ - kind: string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
+ - kind: assignment_expression
+ all:
+ - has: { field: left, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: right
+ all:
+ - kind: string_literal
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
+ - id: sql_format
+ message: "Possible SQL injection: query built with format! passed to execute"
+ rule:
+ kind: call_expression
+ all:
+ - has:
+ kind: field_expression
+ has: { field: field, regex: '^(execute|query|fetch)' }
+ - has:
+ kind: macro_invocation
+ has: { field: macro, regex: '^format$' }
+ stopBy: end
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/scala.yaml b/codebase_rag/analyzers/ast_grep_rules/security/scala.yaml
new file mode 100644
index 000000000..a3faa4586
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/scala.yaml
@@ -0,0 +1,34 @@
+# ast-grep security rules for Scala (issue #413).
+# Each match becomes a SecurityIssue node linked to the file's Module via
+# HAS_VULNERABILITY. Rules are heuristic; treat findings as review prompts.
+ast_grep_id: scala
+extensions: [.scala, .sc]
+rules:
+ - id: os_command_exec
+ message: "Possible command injection: external process launched from code"
+ rule:
+ any:
+ # A bare $X.exec matched any method named exec; require a Runtime receiver.
+ - pattern: "Runtime.getRuntime.exec($$$)"
+ - pattern: "Process($$$)"
+ - pattern: "scala.sys.process.Process($$$)"
+ - id: sql_string_concat
+ # Security rule: favour recall. A SQL keyword string literal joined with +
+ # is a classic injection shape.
+ message: "Possible SQL injection: query built by string concatenation"
+ rule:
+ kind: infix_expression
+ all:
+ - has: { kind: string, regex: '(?i)\b(select|insert|update|delete)\b' }
+ - has: { kind: operator_identifier, regex: '^\+$' }
+ - id: hardcoded_secret
+ # Security rule: favour recall. A val/var whose name matches a credential
+ # keyword bound to a non-trivial (>=10 char) string literal.
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ all:
+ - any:
+ - kind: val_definition
+ - kind: var_definition
+ - has: { kind: identifier, regex: '(?i)(password|secret|api_key|token)' }
+ - has: { kind: string, regex: '^.{12,}$' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/tsx.yaml b/codebase_rag/analyzers/ast_grep_rules/security/tsx.yaml
new file mode 100644
index 000000000..d5c2d4cdd
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/tsx.yaml
@@ -0,0 +1,35 @@
+# ast-grep security rules for TSX (issue #413).
+ast_grep_id: tsx
+extensions: [.tsx]
+rules:
+ - id: innerhtml_xss
+ message: "Possible XSS: assignment to innerHTML/outerHTML"
+ rule:
+ any:
+ - pattern: "$EL.innerHTML = $X"
+ - pattern: "$EL.innerHTML += $X"
+ - pattern: "$EL.outerHTML = $X"
+ - pattern: "$EL.outerHTML += $X"
+ - id: eval_use
+ message: "eval() executes arbitrary code"
+ rule:
+ pattern: "eval($$$)"
+ - id: document_write
+ message: "document.write() can enable XSS"
+ rule:
+ pattern: "document.write($$$)"
+ - id: hardcoded_secret
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: variable_declarator
+ all:
+ - has: { field: name, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: value
+ all:
+ - kind: string
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/security/typescript.yaml b/codebase_rag/analyzers/ast_grep_rules/security/typescript.yaml
new file mode 100644
index 000000000..441446982
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/security/typescript.yaml
@@ -0,0 +1,35 @@
+# ast-grep security rules for TypeScript (issue #413).
+ast_grep_id: typescript
+extensions: [.ts, .mts, .cts]
+rules:
+ - id: innerhtml_xss
+ message: "Possible XSS: assignment to innerHTML/outerHTML"
+ rule:
+ any:
+ - pattern: "$EL.innerHTML = $X"
+ - pattern: "$EL.innerHTML += $X"
+ - pattern: "$EL.outerHTML = $X"
+ - pattern: "$EL.outerHTML += $X"
+ - id: eval_use
+ message: "eval() executes arbitrary code"
+ rule:
+ pattern: "eval($$$)"
+ - id: document_write
+ message: "document.write() can enable XSS"
+ rule:
+ pattern: "document.write($$$)"
+ - id: hardcoded_secret
+ message: "Hardcoded secret assigned to a credential-named variable"
+ rule:
+ kind: variable_declarator
+ all:
+ - has: { field: name, regex: '(?i)(password|secret|api_key|token)' }
+ - has:
+ field: value
+ all:
+ - kind: string
+ - regex: '^.{10,}$'
+ - not:
+ any:
+ - regex: '\{[^{}]*\}'
+ - regex: '%[sdrifeEgGxXoc(]'
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/c.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/c.yaml
new file mode 100644
index 000000000..e45882793
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/c.yaml
@@ -0,0 +1,20 @@
+# ast-grep code-smell rules for C (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: c
+extensions: [.c, .h]
+rules:
+ - id: goto_statement
+ message: "goto complicates control flow"
+ rule:
+ kind: goto_statement
+ - id: debug_print
+ message: "Leftover debug printf/fprintf output"
+ rule:
+ kind: call_expression
+ has: { field: function, regex: '^(printf|fprintf)$' }
+ - id: infinite_loop
+ message: "Infinite loop while(1)/for(;;) has no visible exit condition"
+ rule:
+ any:
+ - pattern: "while (1) $$$"
+ - pattern: "for (;;) $$$"
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/cpp.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/cpp.yaml
new file mode 100644
index 000000000..e75dfba67
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/cpp.yaml
@@ -0,0 +1,24 @@
+# ast-grep code-smell rules for C++ (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: cpp
+extensions: [.cpp, .cc, .cxx, .hpp, .hh]
+rules:
+ - id: using_namespace_std
+ message: "using namespace std pollutes the global namespace"
+ rule:
+ pattern: "using namespace std;"
+ - id: cout_debug_output
+ # Match both std::cout and the bare cout common under `using namespace std`.
+ message: "Debug output via std::cout left in source"
+ rule:
+ any:
+ - pattern: "std::cout"
+ - pattern: "cout"
+ - id: raw_new
+ message: "Raw new expression; prefer smart pointers"
+ rule:
+ kind: new_expression
+ - id: c_array_buffer
+ message: "Fixed-size C array; prefer std::array or std::vector"
+ rule:
+ kind: array_declarator
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/csharp.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/csharp.yaml
new file mode 100644
index 000000000..d17eaaea4
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/csharp.yaml
@@ -0,0 +1,27 @@
+# ast-grep code-smell rules for C# (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: csharp
+extensions: [.cs]
+rules:
+ - id: console_writeline
+ message: "Console.WriteLine left in code reads like debug output"
+ rule:
+ any:
+ - pattern: "Console.WriteLine($$$)"
+ - pattern: "Console.Write($$$)"
+ - id: empty_catch
+ message: "Empty catch block swallows the exception silently"
+ rule:
+ kind: catch_clause
+ has: { kind: block, regex: '^\{\s*\}$' }
+ - id: broad_catch
+ message: "catch (Exception ...) catches too much"
+ rule:
+ kind: catch_clause
+ has:
+ kind: catch_declaration
+ has: { kind: identifier, regex: '^Exception$' }
+ - id: thread_sleep
+ message: "Thread.Sleep blocks the thread; prefer async waiting"
+ rule:
+ pattern: "Thread.Sleep($$$)"
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/dart.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/dart.yaml
new file mode 100644
index 000000000..d11e0359f
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/dart.yaml
@@ -0,0 +1,29 @@
+# ast-grep code-smell rules for Dart (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: dart
+extensions: [.dart]
+rules:
+ - id: debug_print
+ message: "Leftover print() debug statement"
+ rule:
+ kind: call_expression
+ regex: '^print\s*\('
+ - id: empty_catch
+ message: "Empty catch block swallows the exception"
+ rule:
+ kind: block
+ regex: '^\{\s*\}$'
+ inside: { kind: try_statement, stopBy: end }
+ follows: { kind: catch_clause }
+ - id: dynamic_type
+ message: "dynamic type defeats static type checking"
+ rule:
+ kind: type_identifier
+ regex: '^dynamic$'
+ - id: double_equals_null
+ # Catch both orders: `x == null` and the Yoda form `null == x`.
+ message: "Prefer null-aware operators over explicit == null checks scattered in code"
+ rule:
+ kind: equality_expression
+ # Word boundaries so `null` matches only the literal, not `nullValue` etc.
+ regex: '(?:==\s*\bnull\b|\bnull\b\s*==)'
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/go.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/go.yaml
new file mode 100644
index 000000000..4b069372f
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/go.yaml
@@ -0,0 +1,30 @@
+# ast-grep code-smell rules for Go (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: go
+extensions: [.go]
+rules:
+ - id: debug_print
+ message: "Leftover debug output: fmt.Print/Printf/Println call"
+ rule:
+ kind: call_expression
+ has:
+ field: function
+ kind: selector_expression
+ all:
+ - has: { field: operand, regex: '^fmt$' }
+ - has: { field: field, regex: '^Print' }
+ - id: panic_call
+ message: "panic() aborts the program; prefer returning an error"
+ rule:
+ kind: call_expression
+ has: { field: function, kind: identifier, regex: '^panic$' }
+ - id: ignored_error_shortvar
+ message: "Return value discarded with blank identifier in := declaration"
+ rule:
+ kind: short_var_declaration
+ has: { field: left, regex: '_\s*$' }
+ - id: ignored_error_assign
+ message: "Return value discarded with blank identifier in = assignment"
+ rule:
+ kind: assignment_statement
+ has: { field: left, regex: '_\s*$' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/java.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/java.yaml
new file mode 100644
index 000000000..db806b63c
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/java.yaml
@@ -0,0 +1,32 @@
+# ast-grep code-smell rules for Java (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: java
+extensions: [.java]
+rules:
+ - id: system_out_println
+ message: "Debug print left in code: System.out.println"
+ rule:
+ pattern: "System.out.println($$$)"
+ - id: empty_catch
+ message: "Empty catch block swallows the exception"
+ rule:
+ kind: catch_clause
+ has:
+ field: body
+ kind: block
+ regex: '^\{\s*\}$'
+ - id: broad_catch
+ message: "Broad catch of Exception/Throwable catches too much"
+ rule:
+ kind: catch_clause
+ has:
+ kind: catch_formal_parameter
+ has:
+ kind: type_identifier
+ regex: '^(Exception|Throwable)$'
+ stopBy: end
+ - id: print_stack_trace
+ message: "printStackTrace used instead of a logger"
+ rule:
+ kind: method_invocation
+ has: { field: name, regex: '^printStackTrace$' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/javascript.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/javascript.yaml
new file mode 100644
index 000000000..06bb9eb1a
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/javascript.yaml
@@ -0,0 +1,22 @@
+# ast-grep code-smell rules for JavaScript (issue #413).
+ast_grep_id: javascript
+extensions: [.js, .jsx, .mjs, .cjs]
+rules:
+ - id: var_usage
+ message: "var declaration; prefer let/const"
+ rule:
+ kind: variable_declaration
+ - id: loose_equality
+ message: "Loose equality/inequality (== or !=); prefer === / !=="
+ rule:
+ any:
+ - pattern: "$A == $B"
+ - pattern: "$A != $B"
+ - id: console_log
+ message: "Leftover console.log"
+ rule:
+ pattern: "console.log($$$)"
+ - id: debugger_statement
+ message: "Leftover debugger statement"
+ rule:
+ kind: debugger_statement
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/lua.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/lua.yaml
new file mode 100644
index 000000000..1f3cab7bf
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/lua.yaml
@@ -0,0 +1,35 @@
+# ast-grep code-smell rules for Lua (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: lua
+extensions: [.lua]
+rules:
+ - id: debug_print
+ message: "Leftover print() call looks like debug output"
+ rule:
+ kind: function_call
+ has: { kind: identifier, regex: '^print$' }
+ - id: global_assignment
+ # Require at least one bare-identifier target. A field/index write
+ # (`M.const = ...`, `t[k] = ...`) targets an existing table and never leaks
+ # a global, but a MIXED target (`leaked, M.field = 1, 2`) still leaks the
+ # bare `leaked`, so match on the presence of a bare identifier rather than
+ # the absence of an index write. Distinguishing a local reassignment from a
+ # true global still needs scope analysis ast-grep cannot do; that residual
+ # is a known recall-first ceiling.
+ message: "Assignment without local leaks a global"
+ rule:
+ kind: assignment_statement
+ all:
+ - not: { inside: { kind: variable_declaration } }
+ - has:
+ kind: variable_list
+ has: { kind: identifier }
+ - id: long_param_list
+ message: "Very long parameter list is hard to call correctly"
+ rule:
+ kind: parameters
+ has: { kind: identifier, nthChild: 8 }
+ - id: goto_statement
+ message: "goto makes control flow hard to follow"
+ rule:
+ kind: goto_statement
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/php.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/php.yaml
new file mode 100644
index 000000000..b96f07050
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/php.yaml
@@ -0,0 +1,24 @@
+# ast-grep code-smell rules for PHP (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: php
+extensions: [.php]
+rules:
+ - id: debug_output
+ message: "Debug output call (var_dump/print_r/var_export) left in code"
+ rule:
+ kind: function_call_expression
+ has: { field: function, regex: '^(var_dump|print_r|var_export)$' }
+ - id: error_suppression
+ message: "Error suppression operator @ hides failures"
+ rule:
+ kind: error_suppression_expression
+ - id: loose_equality
+ # Loose == / != coerce operand types; anchored regex skips strict === / !==.
+ message: "Loose equality (== or !=) performs type coercion"
+ rule:
+ kind: binary_expression
+ has: { field: operator, regex: '^(==|!=)$' }
+ - id: global_declaration
+ message: "global declaration couples the function to global state"
+ rule:
+ kind: global_declaration
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/python.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/python.yaml
new file mode 100644
index 000000000..c1e6ac591
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/python.yaml
@@ -0,0 +1,33 @@
+# ast-grep code-smell rules for Python (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: python
+extensions: [.py]
+rules:
+ - id: bare_except
+ message: "Bare except swallows every exception"
+ rule:
+ kind: except_clause
+ regex: '^except\s*:'
+ - id: broad_except
+ message: "Broad except Exception catches too much"
+ rule:
+ kind: except_clause
+ regex: 'except\s+Exception'
+ - id: mutable_default_list
+ message: "Mutable default argument (list) is shared across calls"
+ rule:
+ kind: default_parameter
+ has: { field: value, kind: list }
+ - id: mutable_default_dict
+ message: "Mutable default argument (dict) is shared across calls"
+ rule:
+ kind: default_parameter
+ has: { field: value, kind: dictionary }
+ - id: wildcard_import
+ message: "Wildcard import pollutes the namespace"
+ rule:
+ pattern: "from $M import *"
+ - id: global_statement
+ message: "global statement couples the function to module state"
+ rule:
+ kind: global_statement
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/ruby.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/ruby.yaml
new file mode 100644
index 000000000..d0d68fe24
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/ruby.yaml
@@ -0,0 +1,40 @@
+# ast-grep code-smell rules for Ruby (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: ruby
+extensions: [.rb]
+rules:
+ - id: debug_print
+ message: "Leftover debug output (puts/p/pp/print)"
+ rule:
+ kind: call
+ has:
+ field: method
+ regex: '^(puts|pp|print|p)$'
+ - id: swallowed_rescue
+ message: "rescue that is empty or only returns nil hides errors"
+ rule:
+ kind: rescue
+ any:
+ # rescue with no handler body at all
+ - not:
+ has:
+ kind: then
+ # rescue whose body IS bare nil. The nil must be the direct child of the
+ # body: `stopBy: end` searched every descendant, so a real recovery
+ # (`app, data = nil`) or a nil buried in a call (`log(nil)`) was
+ # mis-flagged. A parenthesized body (`(nil)`) wraps the nil in a
+ # parenthesized_statements node, so accept a nil that is its direct child
+ # too (recall) while still rejecting `(app = nil)`. An empty body is
+ # covered by the branch above.
+ - has:
+ kind: then
+ has:
+ any:
+ - kind: nil
+ - kind: parenthesized_statements
+ has:
+ kind: nil
+ - id: global_variable
+ message: "Global variable ($var) couples code to shared mutable state"
+ rule:
+ kind: global_variable
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/rust.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/rust.yaml
new file mode 100644
index 000000000..cc810b259
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/rust.yaml
@@ -0,0 +1,23 @@
+# ast-grep code-smell rules for Rust (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: rust
+extensions: [.rs]
+rules:
+ - id: unwrap_call
+ message: "unwrap() panics instead of handling the error case"
+ rule:
+ pattern: "$X.unwrap()"
+ - id: expect_call
+ message: "expect() panics instead of handling the error case"
+ rule:
+ pattern: "$X.expect($$$)"
+ - id: dbg_macro
+ message: "dbg! macro left in code"
+ rule:
+ kind: macro_invocation
+ has: { field: macro, regex: '^dbg$' }
+ - id: panic_macro
+ message: "panic!/unimplemented!/todo!/unreachable! macro aborts at runtime"
+ rule:
+ kind: macro_invocation
+ has: { field: macro, regex: '^(panic|unimplemented|todo|unreachable)$' }
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/scala.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/scala.yaml
new file mode 100644
index 000000000..a3e0ca3a1
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/scala.yaml
@@ -0,0 +1,22 @@
+# ast-grep code-smell rules for Scala (issue #413).
+# Each match becomes a CodeSmell node linked to the file's Module via HAS_SMELL.
+ast_grep_id: scala
+extensions: [.scala, .sc]
+rules:
+ - id: debug_println
+ message: "println left in code; use a logger instead"
+ rule:
+ pattern: "println($$$)"
+ - id: mutable_var
+ message: "Mutable var declaration; prefer val where possible"
+ rule:
+ kind: var_definition
+ - id: null_usage
+ message: "null literal; prefer Option to represent absence"
+ rule:
+ kind: null_literal
+ - id: instanceof_cast
+ message: "asInstanceOf cast bypasses the type system"
+ rule:
+ kind: generic_function
+ regex: 'asInstanceOf'
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/tsx.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/tsx.yaml
new file mode 100644
index 000000000..0bfe11be2
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/tsx.yaml
@@ -0,0 +1,22 @@
+# ast-grep code-smell rules for TSX (issue #413).
+ast_grep_id: tsx
+extensions: [.tsx]
+rules:
+ - id: var_usage
+ message: "var declaration; prefer let/const"
+ rule:
+ kind: variable_declaration
+ - id: loose_equality
+ message: "Loose equality/inequality (== or !=); prefer === / !=="
+ rule:
+ any:
+ - pattern: "$A == $B"
+ - pattern: "$A != $B"
+ - id: console_log
+ message: "Leftover console.log"
+ rule:
+ pattern: "console.log($$$)"
+ - id: debugger_statement
+ message: "Leftover debugger statement"
+ rule:
+ kind: debugger_statement
diff --git a/codebase_rag/analyzers/ast_grep_rules/smells/typescript.yaml b/codebase_rag/analyzers/ast_grep_rules/smells/typescript.yaml
new file mode 100644
index 000000000..e5086e9d3
--- /dev/null
+++ b/codebase_rag/analyzers/ast_grep_rules/smells/typescript.yaml
@@ -0,0 +1,22 @@
+# ast-grep code-smell rules for TypeScript (issue #413).
+ast_grep_id: typescript
+extensions: [.ts, .mts, .cts]
+rules:
+ - id: var_usage
+ message: "var declaration; prefer let/const"
+ rule:
+ kind: variable_declaration
+ - id: loose_equality
+ message: "Loose equality/inequality (== or !=); prefer === / !=="
+ rule:
+ any:
+ - pattern: "$A == $B"
+ - pattern: "$A != $B"
+ - id: console_log
+ message: "Leftover console.log"
+ rule:
+ pattern: "console.log($$$)"
+ - id: debugger_statement
+ message: "Leftover debugger statement"
+ rule:
+ kind: debugger_statement
diff --git a/codebase_rag/ast_cache.py b/codebase_rag/ast_cache.py
new file mode 100644
index 000000000..2b9860b1c
--- /dev/null
+++ b/codebase_rag/ast_cache.py
@@ -0,0 +1,92 @@
+# LRU AST cache bounded by both entry count and estimated memory. A miss
+# re-parses from disk via the loader so an eviction cannot lose an AST that
+# type inference still needs (see load()).
+
+import sys
+from collections import OrderedDict
+from collections.abc import Callable, ItemsView
+from pathlib import Path
+
+from tree_sitter import Node
+
+from . import constants as cs
+from .config import settings
+
+
+class BoundedASTCache:
+ __slots__ = ("cache", "loader", "max_entries", "max_memory_bytes")
+
+ def __init__(
+ self,
+ max_entries: int | None = None,
+ max_memory_mb: int | None = None,
+ loader: Callable[[Path], tuple[Node, cs.SupportedLanguage] | None]
+ | None = None,
+ ):
+ self.cache: OrderedDict[Path, tuple[Node, cs.SupportedLanguage]] = OrderedDict()
+ self.loader = loader
+ self.max_entries = (
+ max_entries if max_entries is not None else settings.CACHE_MAX_ENTRIES
+ )
+ max_mem = (
+ max_memory_mb if max_memory_mb is not None else settings.CACHE_MAX_MEMORY_MB
+ )
+ self.max_memory_bytes = max_mem * cs.BYTES_PER_MB
+
+ def load(self, key: Path) -> tuple[Node, cs.SupportedLanguage] | None:
+ # Cache read that survives eviction: a miss re-parses from disk via the
+ # loader and re-inserts (bounded). Type inference reads OTHER modules'
+ # ASTs long after Pass 2 parsed them; on a repo larger than max_entries
+ # a plain __getitem__ would drop the inferred type (django:
+ # urls/resolvers.py evicted before admindocs resolves get_resolver()).
+ if key in self.cache:
+ return self[key]
+ if self.loader is None or not (entry := self.loader(key)):
+ return None
+ self[key] = entry
+ return entry
+
+ def __setitem__(self, key: Path, value: tuple[Node, cs.SupportedLanguage]) -> None:
+ if key in self.cache:
+ del self.cache[key]
+
+ self.cache[key] = value
+
+ self._enforce_limits()
+
+ def __getitem__(self, key: Path) -> tuple[Node, cs.SupportedLanguage]:
+ value = self.cache[key]
+ self.cache.move_to_end(key)
+ return value
+
+ def __delitem__(self, key: Path) -> None:
+ if key in self.cache:
+ del self.cache[key]
+
+ def __contains__(self, key: Path) -> bool:
+ return key in self.cache
+
+ def items(self) -> ItemsView[Path, tuple[Node, cs.SupportedLanguage]]:
+ return self.cache.items()
+
+ def _enforce_limits(self) -> None:
+ while len(self.cache) > self.max_entries:
+ self.cache.popitem(last=False)
+
+ if self._should_evict_for_memory():
+ entries_to_remove = max(
+ 1, len(self.cache) // settings.CACHE_EVICTION_DIVISOR
+ )
+ for _ in range(entries_to_remove):
+ if self.cache:
+ self.cache.popitem(last=False)
+
+ def _should_evict_for_memory(self) -> bool:
+ try:
+ cache_size = sum(sys.getsizeof(v) for v in self.cache.values())
+ return cache_size > self.max_memory_bytes
+ except Exception:
+ return (
+ len(self.cache)
+ > self.max_entries * settings.CACHE_MEMORY_THRESHOLD_RATIO
+ )
diff --git a/codebase_rag/capture.py b/codebase_rag/capture.py
new file mode 100644
index 000000000..3f7b1a7f1
--- /dev/null
+++ b/codebase_rag/capture.py
@@ -0,0 +1,136 @@
+from __future__ import annotations
+
+from collections.abc import Iterable
+from dataclasses import dataclass
+
+from loguru import logger
+
+from . import constants as cs
+from . import logs as ls
+
+# Relationships that are usually meaningless without a companion still
+# enabled. Dropping the companion is obeyed, but warned about.
+_SOFT_DEPENDENCIES: dict[cs.RelationshipType, cs.RelationshipType] = {
+ cs.RelationshipType.OVERRIDES: cs.RelationshipType.INHERITS,
+}
+
+
+@dataclass(frozen=True)
+class CaptureSelection:
+ enabled_rels: frozenset[cs.RelationshipType]
+ enabled_node_labels: frozenset[cs.NodeLabel]
+
+ def rel_enabled(self, rel: cs.RelationshipType) -> bool:
+ return rel in self.enabled_rels
+
+ def node_enabled(self, label: cs.NodeLabel) -> bool:
+ return label in self.enabled_node_labels
+
+ @property
+ def io_enabled(self) -> bool:
+ return (
+ cs.RelationshipType.READS_FROM in self.enabled_rels
+ or cs.RelationshipType.WRITES_TO in self.enabled_rels
+ )
+
+
+def _node_labels_for(
+ enabled_rels: frozenset[cs.RelationshipType],
+) -> frozenset[cs.NodeLabel]:
+ owner_of: dict[cs.NodeLabel, cs.CaptureGroup] = {
+ label: group
+ for group, labels in cs.CAPTURE_GROUP_NODE_LABELS.items()
+ for label in labels
+ }
+ enabled: set[cs.NodeLabel] = set()
+ for label in cs.NodeLabel:
+ owner = owner_of.get(label)
+ if owner is None or cs.CAPTURE_GROUP_RELS[owner] & enabled_rels:
+ enabled.add(label)
+ return frozenset(enabled)
+
+
+def _selection_for(
+ enabled_rels: frozenset[cs.RelationshipType],
+) -> CaptureSelection:
+ for rel, companion in _SOFT_DEPENDENCIES.items():
+ if rel in enabled_rels and companion not in enabled_rels:
+ logger.warning(ls.CAPTURE_DEPENDENCY_GAP.format(rel=rel, missing=companion))
+ return CaptureSelection(
+ enabled_rels=enabled_rels,
+ enabled_node_labels=_node_labels_for(enabled_rels),
+ )
+
+
+_ALL_RELS: frozenset[cs.RelationshipType] = frozenset(cs.RelationshipType)
+
+ALL_ENABLED = _selection_for(_ALL_RELS)
+
+
+def _resolve_token(token: str) -> frozenset[cs.RelationshipType] | None:
+ try:
+ return cs.CAPTURE_GROUP_RELS[cs.CaptureGroup(token.lower())]
+ except ValueError:
+ pass
+ try:
+ return frozenset({cs.RelationshipType(token.upper())})
+ except ValueError:
+ return None
+
+
+def _base_rels(groups: Iterable[cs.CaptureGroup]) -> set[cs.RelationshipType]:
+ enabled: set[cs.RelationshipType] = set()
+ for group in groups:
+ enabled |= cs.CAPTURE_GROUP_RELS[group]
+ return enabled
+
+
+def resolve_capture(tokens: Iterable[str]) -> CaptureSelection:
+ # Tokens apply left-to-right so later ones win: a `none`/`all` base
+ # clears everything before it, and add-drops after a base survive. So
+ # `--capture none` overrides an inherited `CGR_CAPTURE=io`.
+ enabled = _base_rels(cs.DEFAULT_CAPTURE_GROUPS)
+ for token in tokens:
+ token = token.strip()
+ if not token:
+ continue
+ low = token.lower()
+ if low == cs.CAPTURE_TOKEN_NONE:
+ enabled = set()
+ continue
+ if low == cs.CAPTURE_TOKEN_ALL:
+ enabled = set(_ALL_RELS)
+ continue
+ drop = token.startswith(cs.CAPTURE_DROP_PREFIX)
+ name = token.lstrip(cs.CAPTURE_DROP_PREFIX + cs.CAPTURE_ADD_PREFIX)
+ rels = _resolve_token(name)
+ if rels is None:
+ logger.warning(ls.CAPTURE_UNKNOWN_TOKEN.format(token=token))
+ continue
+ if drop:
+ enabled -= rels
+ else:
+ enabled |= rels
+
+ return _selection_for(frozenset(enabled))
+
+
+def split_spec(spec: str) -> list[str]:
+ out: list[str] = []
+ token = ""
+ for ch in spec:
+ if ch in cs.CAPTURE_TOKEN_SEPARATORS:
+ if token:
+ out.append(token)
+ token = ""
+ else:
+ token += ch
+ if token:
+ out.append(token)
+ return out
+
+
+def default_capture() -> CaptureSelection:
+ from .config import settings
+
+ return resolve_capture(split_spec(settings.CGR_CAPTURE))
diff --git a/codebase_rag/cgr_state.py b/codebase_rag/cgr_state.py
new file mode 100644
index 000000000..703672a64
--- /dev/null
+++ b/codebase_rag/cgr_state.py
@@ -0,0 +1,57 @@
+from __future__ import annotations
+
+import json
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import TypedDict
+
+from loguru import logger
+
+from .config import settings
+
+STATE_FILENAME = "state.json"
+
+
+class _StateShape(TypedDict, total=False):
+ last_sync: dict[str, str]
+
+
+def state_path(home: Path | None = None) -> Path:
+ base = (home or settings.CGR_HOME).expanduser()
+ return base / STATE_FILENAME
+
+
+def _load(path: Path) -> _StateShape:
+ if not path.exists():
+ return _StateShape()
+ try:
+ with path.open(encoding="utf-8") as f:
+ data = json.load(f)
+ if isinstance(data, dict):
+ return _StateShape(last_sync=data.get("last_sync", {}))
+ except (OSError, json.JSONDecodeError) as e:
+ logger.warning(f"Failed to load cgr state from {path}: {e}")
+ return _StateShape()
+
+
+def _save(path: Path, data: _StateShape) -> None:
+ try:
+ path.parent.mkdir(parents=True, exist_ok=True)
+ with path.open("w", encoding="utf-8") as f:
+ json.dump(data, f, indent=2)
+ except OSError as e:
+ logger.warning(f"Failed to save cgr state to {path}: {e}")
+
+
+def record_sync(project_name: str, home: Path | None = None) -> None:
+ path = state_path(home)
+ state = _load(path)
+ last_sync = state.get("last_sync", {})
+ last_sync[project_name] = datetime.now(UTC).isoformat()
+ state["last_sync"] = last_sync
+ _save(path, state)
+
+
+def read_sync_timestamps(home: Path | None = None) -> dict[str, str]:
+ state = _load(state_path(home))
+ return dict(state.get("last_sync", {}))
diff --git a/codebase_rag/cli.py b/codebase_rag/cli.py
index 87f9a5379..4d20ed4f2 100644
--- a/codebase_rag/cli.py
+++ b/codebase_rag/cli.py
@@ -1,39 +1,75 @@
+"""Command-line entry point wiring cgr subcommands to their handlers."""
+
import asyncio
+import json
+import time
+from collections.abc import Callable
+from fnmatch import fnmatch
+from functools import partial
+from importlib.metadata import version as get_version
from pathlib import Path
+import click
import typer
from loguru import logger
+from rich.console import Console
from rich.panel import Panel
from rich.table import Table
+from . import cgr_state
from . import cli_help as ch
from . import constants as cs
from . import logs as ls
-from .config import load_cgrignore_patterns, settings
+from .capture import CaptureSelection, resolve_capture, split_spec
+from .config import load_ignore_patterns, settings
from .graph_updater import GraphUpdater
from .main import (
+ _create_configuration_table,
app_context,
connect_memgraph,
export_graph_to_file,
main_async,
main_optimize_async,
+ main_single_query,
prompt_for_unignored_directories,
style,
update_model_settings,
)
from .parser_loader import load_parsers
+from .services.graph_service import MemgraphIngestor
from .services.protobuf_service import ProtobufFileIngestor
+from .stack import StackManager
+from .stack.cli import cli as daemon_cli
+from .stack.constants import StackState
+from .stack.manager import StackError
from .tools.health_checker import HealthChecker
from .tools.language import cli as language_cli
+from .types_defs import DeadCodeConfig, DeadCodeRow, ResultRow
+from .utils.path_utils import derive_project_name, resolve_repo_path
+from .vector_store import clear_all_embeddings, delete_project_embeddings
+from .workspaces import WorkspaceConfig, WorkspaceError, load_workspace
+from .workspaces.cli import cli as workspace_cli
app = typer.Typer(
- name="code-graph-rag",
+ name=cs.PACKAGE_NAME,
help=ch.APP_DESCRIPTION,
+ epilog=ch.APP_EPILOG,
no_args_is_help=True,
add_completion=False,
)
+def _version_callback(value: bool) -> None:
+ if value:
+ app_context.console.print(
+ cs.CLI_MSG_VERSION.format(
+ package=cs.PACKAGE_NAME, version=get_version(cs.PACKAGE_NAME)
+ ),
+ highlight=False,
+ )
+ raise typer.Exit()
+
+
def validate_models_early() -> None:
try:
orchestrator_config = settings.active_orchestrator_config
@@ -58,11 +94,19 @@ def _update_and_validate_models(orchestrator: str | None, cypher: str | None) ->
@app.callback()
def _global_options(
+ version: bool | None = typer.Option(
+ None,
+ "--version",
+ "-v",
+ help=ch.HELP_VERSION,
+ callback=_version_callback,
+ is_eager=True,
+ ),
quiet: bool = typer.Option(
False,
"--quiet",
"-q",
- help="Suppress non-essential output (progress messages, banners, informational logs).",
+ help=ch.HELP_QUIET,
is_eager=True,
),
) -> None:
@@ -77,7 +121,207 @@ def _info(msg: str) -> None:
app_context.console.print(msg)
-@app.command(help=ch.CMD_START)
+def _load_workspace_or_exit(workspace: str | None) -> WorkspaceConfig | None:
+ if workspace is None:
+ return None
+ try:
+ return load_workspace(workspace)
+ except WorkspaceError as e:
+ app_context.console.print(style(str(e), cs.Color.RED))
+ raise typer.Exit(1) from e
+
+
+def _sync_workspace(
+ config: WorkspaceConfig,
+ batch_size: int,
+ exclude: list[str] | None,
+ capture: list[str] | None = None,
+ skip_embeddings: bool | None = None,
+) -> None:
+ total = len(config.repos)
+ if total == 0:
+ _info(
+ style(cs.CLI_MSG_WORKSPACE_EMPTY.format(name=config.name), cs.Color.YELLOW)
+ )
+ return
+ _info(
+ style(
+ cs.CLI_MSG_WORKSPACE_SYNCING.format(name=config.name, count=total),
+ cs.Color.CYAN,
+ )
+ )
+ for idx, repo in enumerate(config.repos, start=1):
+ repo_path = repo.repo_path()
+ _info(
+ style(
+ cs.CLI_MSG_WORKSPACE_SYNC_REPO.format(
+ idx=idx,
+ total=total,
+ path=repo_path,
+ project_name=repo.project_name,
+ ),
+ cs.Color.CYAN,
+ )
+ )
+ _run_graph_sync(
+ repo=repo_path,
+ project_name=repo.project_name,
+ batch_size=batch_size,
+ exclude=exclude,
+ interactive_setup=False,
+ capture=capture,
+ skip_embeddings=skip_embeddings,
+ )
+
+
+def _resolve_active_projects(projects: str | None, default_project: str) -> list[str]:
+ if projects:
+ parsed = [p.strip() for p in projects.split(",") if p.strip()]
+ if parsed:
+ return parsed
+ return [default_project]
+
+
+def _maybe_start_stack() -> None:
+ mgr = StackManager()
+ if mgr.status().state == StackState.RUNNING:
+ return
+ try:
+ mgr.ensure_running()
+ except StackError as e:
+ app_context.console.print(style(str(e), cs.Color.RED))
+ raise typer.Exit(1) from e
+
+
+def _capture_selection(capture: list[str] | None) -> CaptureSelection:
+ # Env CGR_CAPTURE is the sticky baseline; --capture tokens are appended so
+ # a single run can override it (later tokens win in the resolver).
+ return resolve_capture([*split_spec(settings.CGR_CAPTURE), *(capture or [])])
+
+
+def _run_graph_sync(
+ repo: Path,
+ project_name: str,
+ batch_size: int,
+ exclude: list[str] | None,
+ interactive_setup: bool,
+ clean: bool = False,
+ output: str | None = None,
+ capture: list[str] | None = None,
+ skip_embeddings: bool | None = None,
+) -> None:
+ cgrignore = load_ignore_patterns(repo)
+ cli_excludes = frozenset(exclude) if exclude else frozenset()
+ exclude_paths = cli_excludes | cgrignore.exclude or None
+ unignore_paths: frozenset[str] | None
+ if interactive_setup:
+ unignore_paths = prompt_for_unignored_directories(repo, exclude)
+ else:
+ unignore_paths = cgrignore.unignore or None
+
+ elapsed = time.monotonic()
+ with connect_memgraph(batch_size) as ingestor:
+ if clean:
+ _info(style(cs.CLI_MSG_CLEANING_DB, cs.Color.YELLOW))
+ ingestor.clean_database()
+ _delete_hash_cache(repo)
+ # Stale vectors keyed by recycled node ids would crowd out live
+ # hits and can map onto unrelated nodes in the rebuilt graph.
+ clear_all_embeddings()
+
+ ingestor.ensure_constraints()
+
+ parsers, queries = load_parsers()
+
+ updater = GraphUpdater(
+ ingestor=ingestor,
+ repo_path=repo,
+ parsers=parsers,
+ queries=queries,
+ unignore_paths=unignore_paths,
+ exclude_paths=exclude_paths,
+ project_name=project_name,
+ capture=_capture_selection(capture),
+ skip_embeddings=skip_embeddings,
+ )
+ updater.run()
+ cgr_state.record_sync(project_name)
+
+ if output:
+ _info(style(cs.CLI_MSG_EXPORTING_TO.format(path=output), cs.Color.CYAN))
+ if not export_graph_to_file(ingestor, output):
+ raise typer.Exit(1)
+ elapsed = time.monotonic() - elapsed
+ if updater.skipped_because_in_sync:
+ app_context.console.print(
+ style(
+ cs.CLI_MSG_SYNC_SKIPPED.format(project=project_name, elapsed=elapsed),
+ cs.Color.CYAN,
+ cs.StyleModifier.DIM,
+ )
+ )
+ else:
+ app_context.console.print(
+ style(
+ cs.CLI_MSG_SYNC_DONE.format(project=project_name, elapsed=elapsed),
+ cs.Color.CYAN,
+ cs.StyleModifier.NONE,
+ )
+ )
+
+
+def _delete_hash_cache(repo_path: Path) -> None:
+ cache_path = repo_path / cs.HASH_CACHE_FILENAME
+ if cache_path.exists():
+ _info(
+ style(
+ cs.CLI_MSG_CLEANING_HASH_CACHE.format(path=cache_path),
+ cs.Color.YELLOW,
+ )
+ )
+ cache_path.unlink(missing_ok=True)
+ (repo_path / cs.DIR_MTIMES_FILENAME).unlink(missing_ok=True)
+ (repo_path / cs.PARSER_FINGERPRINT_FILENAME).unlink(missing_ok=True)
+
+
+def _resolve_and_validate_repo(repo_path: str | None) -> Path:
+ resolved = resolve_repo_path(repo_path, settings.TARGET_REPO_PATH)
+ if not resolved.exists():
+ app_context.console.print(
+ style(cs.CLI_ERR_PATH_NOT_EXISTS.format(path=resolved), cs.Color.RED)
+ )
+ raise typer.Exit(1)
+ if not resolved.is_dir():
+ app_context.console.print(
+ style(cs.CLI_ERR_PATH_NOT_DIR.format(path=resolved), cs.Color.RED)
+ )
+ raise typer.Exit(1)
+ if not (resolved / cs.GIT_DIR_NAME).exists():
+ app_context.console.print(
+ style(cs.CLI_WARN_NOT_GIT_REPO.format(path=resolved), cs.Color.YELLOW)
+ )
+ return resolved
+
+
+def _cleanup_project_embeddings(ingestor: MemgraphIngestor, project_name: str) -> None:
+ rows = ingestor.fetch_all(
+ cs.CYPHER_QUERY_PROJECT_NODE_IDS,
+ {cs.KEY_PROJECT_NAME: project_name},
+ )
+ node_ids: list[int] = []
+ for row in rows:
+ node_id = row.get(cs.KEY_NODE_ID)
+ if isinstance(node_id, int):
+ node_ids.append(node_id)
+ delete_project_embeddings(project_name, node_ids)
+
+
+@app.command(
+ help=ch.CMD_START,
+ short_help=ch.CMD_START,
+ epilog=ch.EXAMPLES_START,
+ rich_help_panel=ch.PANEL_USE,
+)
def start(
repo_path: str | None = typer.Option(
None, "--repo-path", help=ch.HELP_REPO_PATH_RETRIEVAL
@@ -113,26 +357,86 @@ def start(
"--no-confirm",
help=ch.HELP_NO_CONFIRM,
),
+ no_instructions: bool = typer.Option(
+ False,
+ "--no-instructions",
+ help=ch.HELP_NO_INSTRUCTIONS,
+ ),
batch_size: int | None = typer.Option(
None,
"--batch-size",
min=1,
help=ch.HELP_BATCH_SIZE,
),
+ project_name: str | None = typer.Option(
+ None,
+ "--project-name",
+ help=ch.HELP_PROJECT_NAME,
+ ),
exclude: list[str] | None = typer.Option(
None,
"--exclude",
help=ch.HELP_EXCLUDE_PATTERNS,
),
+ capture: list[str] | None = typer.Option(
+ None,
+ "--capture",
+ help=ch.HELP_CAPTURE,
+ ),
interactive_setup: bool = typer.Option(
False,
"--interactive-setup",
help=ch.HELP_INTERACTIVE_SETUP,
),
+ ask_agent: str | None = typer.Option(
+ None,
+ "-a",
+ "--ask-agent",
+ help=ch.HELP_ASK_AGENT,
+ ),
+ output_format: cs.QueryFormat = typer.Option(
+ cs.QueryFormat.TABLE,
+ "--output-format",
+ help=ch.HELP_QUERY_OUTPUT_FORMAT,
+ ),
+ no_start_stack: bool = typer.Option(
+ False,
+ "--no-start-stack",
+ help=ch.HELP_NO_START_STACK,
+ ),
+ no_sync: bool = typer.Option(
+ False,
+ "--no-sync",
+ help=ch.HELP_NO_SYNC,
+ ),
+ no_embeddings: bool = typer.Option(
+ False,
+ "--no-embeddings",
+ help=ch.HELP_NO_EMBEDDINGS,
+ ),
+ projects: str | None = typer.Option(
+ None,
+ "--projects",
+ help=ch.HELP_PROJECTS,
+ ),
+ workspace: str | None = typer.Option(
+ None,
+ "--workspace",
+ help=ch.HELP_WORKSPACE,
+ ),
) -> None:
app_context.session.confirm_edits = not no_confirm
+ app_context.session.load_cgr_instructions = not no_instructions
- target_repo_path = repo_path or settings.TARGET_REPO_PATH
+ if output_format == cs.QueryFormat.JSON and not ask_agent:
+ app_context.console.print(
+ style(cs.CLI_ERR_JSON_REQUIRES_ASK_AGENT, cs.Color.RED)
+ )
+ raise typer.Exit(1)
+
+ resolved_repo = _resolve_and_validate_repo(repo_path)
+ target_repo_path = str(resolved_repo)
+ resolved_project_name = project_name or derive_project_name(resolved_repo)
if output and not update_graph:
app_context.console.print(
@@ -140,54 +444,105 @@ def start(
)
raise typer.Exit(1)
- _update_and_validate_models(orchestrator, cypher)
+ if not no_start_stack:
+ _maybe_start_stack()
effective_batch_size = settings.resolve_batch_size(batch_size)
+ if clean and not update_graph:
+ repo_to_clean = Path(target_repo_path)
+ with connect_memgraph(effective_batch_size) as ingestor:
+ _info(style(cs.CLI_MSG_CLEANING_DB, cs.Color.YELLOW))
+ ingestor.clean_database()
+
+ clear_all_embeddings()
+ _delete_hash_cache(repo_to_clean)
+ _info(style(cs.CLI_MSG_CLEAN_DONE, cs.Color.GREEN))
+ return
+
+ _update_and_validate_models(orchestrator, cypher)
+
+ if not ask_agent and not update_graph:
+ app_context.console.print(_create_configuration_table(target_repo_path))
+
if update_graph:
- repo_to_update = Path(target_repo_path)
_info(
- style(cs.CLI_MSG_UPDATING_GRAPH.format(path=repo_to_update), cs.Color.GREEN)
+ style(cs.CLI_MSG_UPDATING_GRAPH.format(path=resolved_repo), cs.Color.GREEN)
)
-
- cgrignore = load_cgrignore_patterns(repo_to_update)
- cli_excludes = frozenset(exclude) if exclude else frozenset()
- exclude_paths = cli_excludes | cgrignore.exclude or None
- unignore_paths: frozenset[str] | None = None
- if interactive_setup:
- unignore_paths = prompt_for_unignored_directories(repo_to_update, exclude)
- else:
+ if not interactive_setup:
_info(style(cs.CLI_MSG_AUTO_EXCLUDE, cs.Color.YELLOW))
- unignore_paths = cgrignore.unignore or None
+ _run_graph_sync(
+ repo=resolved_repo,
+ project_name=resolved_project_name,
+ batch_size=effective_batch_size,
+ exclude=exclude,
+ interactive_setup=interactive_setup,
+ clean=clean,
+ output=output,
+ capture=capture,
+ skip_embeddings=no_embeddings or None,
+ )
+ _info(style(cs.CLI_MSG_GRAPH_UPDATED, cs.Color.GREEN))
+ return
- with connect_memgraph(effective_batch_size) as ingestor:
- if clean:
- _info(style(cs.CLI_MSG_CLEANING_DB, cs.Color.YELLOW))
- ingestor.clean_database()
- ingestor.ensure_constraints()
-
- parsers, queries = load_parsers()
-
- updater = GraphUpdater(
- ingestor,
- repo_to_update,
- parsers,
- queries,
- unignore_paths,
- exclude_paths,
+ workspace_config = _load_workspace_or_exit(workspace)
+
+ sync_task: Callable[[], None] | None = None
+ sync_message = cs.MSG_SYNCING_KNOWLEDGE_GRAPH
+ if not no_sync:
+ if workspace_config is not None:
+ sync_task = partial(
+ _sync_workspace,
+ workspace_config,
+ effective_batch_size,
+ exclude,
+ capture=capture,
+ skip_embeddings=no_embeddings or None,
+ )
+ sync_message = cs.MSG_SYNCING_WORKSPACE.format(
+ name=workspace_config.name, count=len(workspace_config.repos)
+ )
+ else:
+ sync_task = partial(
+ _run_graph_sync,
+ repo=resolved_repo,
+ project_name=resolved_project_name,
+ batch_size=effective_batch_size,
+ exclude=exclude,
+ interactive_setup=interactive_setup,
+ capture=capture,
+ skip_embeddings=no_embeddings or None,
)
- updater.run()
-
- if output:
- _info(style(cs.CLI_MSG_EXPORTING_TO.format(path=output), cs.Color.CYAN))
- if not export_graph_to_file(ingestor, output):
- raise typer.Exit(1)
- _info(style(cs.CLI_MSG_GRAPH_UPDATED, cs.Color.GREEN))
- return
+ if workspace_config is not None:
+ active_projects = workspace_config.project_names()
+ if projects:
+ active_projects = _resolve_active_projects(projects, active_projects[0])
+ else:
+ active_projects = _resolve_active_projects(projects, resolved_project_name)
try:
- asyncio.run(main_async(target_repo_path, effective_batch_size))
+ if ask_agent:
+ if sync_task is not None:
+ sync_task()
+ main_single_query(
+ target_repo_path,
+ effective_batch_size,
+ ask_agent,
+ active_projects=active_projects,
+ output_format=output_format,
+ )
+ else:
+ asyncio.run(
+ main_async(
+ target_repo_path,
+ effective_batch_size,
+ active_projects=active_projects,
+ show_config_table=False,
+ pre_chat_sync=sync_task,
+ pre_chat_sync_message=sync_message,
+ )
+ )
except KeyboardInterrupt:
app_context.console.print(style(cs.CLI_MSG_APP_TERMINATED, cs.Color.RED))
except ValueError as e:
@@ -196,7 +551,12 @@ def start(
)
-@app.command(help=ch.CMD_INDEX)
+@app.command(
+ help=ch.CMD_INDEX,
+ short_help=ch.CMD_INDEX,
+ epilog=ch.EXAMPLES_INDEX,
+ rich_help_panel=ch.PANEL_GRAPH,
+)
def index(
repo_path: str | None = typer.Option(
None, "--repo-path", help=ch.HELP_REPO_PATH_INDEX
@@ -217,19 +577,23 @@ def index(
"--exclude",
help=ch.HELP_EXCLUDE_PATTERNS,
),
+ capture: list[str] | None = typer.Option(
+ None,
+ "--capture",
+ help=ch.HELP_CAPTURE,
+ ),
interactive_setup: bool = typer.Option(
False,
"--interactive-setup",
help=ch.HELP_INTERACTIVE_SETUP,
),
) -> None:
- target_repo_path = repo_path or settings.TARGET_REPO_PATH
- repo_to_index = Path(target_repo_path)
+ repo_to_index = _resolve_and_validate_repo(repo_path)
_info(style(cs.CLI_MSG_INDEXING_AT.format(path=repo_to_index), cs.Color.GREEN))
_info(style(cs.CLI_MSG_OUTPUT_TO.format(path=output_proto_dir), cs.Color.CYAN))
- cgrignore = load_cgrignore_patterns(repo_to_index)
+ cgrignore = load_ignore_patterns(repo_to_index)
cli_excludes = frozenset(exclude) if exclude else frozenset()
exclude_paths = cli_excludes | cgrignore.exclude or None
unignore_paths: frozenset[str] | None = None
@@ -245,7 +609,13 @@ def index(
)
parsers, queries = load_parsers()
updater = GraphUpdater(
- ingestor, repo_to_index, parsers, queries, unignore_paths, exclude_paths
+ ingestor=ingestor,
+ repo_path=repo_to_index,
+ parsers=parsers,
+ queries=queries,
+ unignore_paths=unignore_paths,
+ exclude_paths=exclude_paths,
+ capture=_capture_selection(capture),
)
updater.run()
@@ -259,7 +629,12 @@ def index(
raise typer.Exit(1) from e
-@app.command(help=ch.CMD_EXPORT)
+@app.command(
+ help=ch.CMD_EXPORT,
+ short_help=ch.CMD_EXPORT,
+ epilog=ch.EXAMPLES_EXPORT,
+ rich_help_panel=ch.PANEL_GRAPH,
+)
def export(
output: str = typer.Option(..., "-o", "--output", help=ch.HELP_OUTPUT_PATH),
format_json: bool = typer.Option(
@@ -295,7 +670,12 @@ def export(
raise typer.Exit(1) from e
-@app.command(help=ch.CMD_OPTIMIZE)
+@app.command(
+ help=ch.CMD_OPTIMIZE,
+ short_help=ch.CMD_OPTIMIZE,
+ epilog=ch.EXAMPLES_OPTIMIZE,
+ rich_help_panel=ch.PANEL_USE,
+)
def optimize(
language: str = typer.Argument(
...,
@@ -324,6 +704,11 @@ def optimize(
"--no-confirm",
help=ch.HELP_NO_CONFIRM,
),
+ no_instructions: bool = typer.Option(
+ False,
+ "--no-instructions",
+ help=ch.HELP_NO_INSTRUCTIONS,
+ ),
batch_size: int | None = typer.Option(
None,
"--batch-size",
@@ -332,8 +717,9 @@ def optimize(
),
) -> None:
app_context.session.confirm_edits = not no_confirm
+ app_context.session.load_cgr_instructions = not no_instructions
- target_repo_path = repo_path or settings.TARGET_REPO_PATH
+ target_repo_path = str(_resolve_and_validate_repo(repo_path))
_update_and_validate_models(orchestrator, cypher)
@@ -356,12 +742,31 @@ def optimize(
)
-@app.command(name=ch.CLICommandName.MCP_SERVER, help=ch.CMD_MCP_SERVER)
-def mcp_server() -> None:
+@app.command(
+ name=ch.CLICommandName.MCP_SERVER,
+ help=ch.CMD_MCP_SERVER,
+ short_help=ch.CMD_MCP_SERVER,
+ epilog=ch.EXAMPLES_MCP_SERVER,
+ rich_help_panel=ch.PANEL_USE,
+)
+def mcp_server(
+ transport: cs.MCPTransport = typer.Option(
+ cs.MCPTransport.STDIO, help=ch.HELP_MCP_TRANSPORT
+ ),
+ host: str = typer.Option(None, help=ch.HELP_MCP_HTTP_HOST),
+ port: int = typer.Option(None, help=ch.HELP_MCP_HTTP_PORT),
+) -> None:
try:
- from codebase_rag.mcp import main as mcp_main
+ if transport == cs.MCPTransport.HTTP:
+ from codebase_rag.mcp import serve_http
+
+ resolved_host = host or settings.MCP_HTTP_HOST
+ resolved_port = port or settings.MCP_HTTP_PORT
+ asyncio.run(serve_http(host=resolved_host, port=resolved_port))
+ else:
+ from codebase_rag.mcp import serve_stdio
- asyncio.run(mcp_main())
+ asyncio.run(serve_stdio())
except KeyboardInterrupt:
app_context.console.print(style(cs.CLI_MSG_APP_TERMINATED, cs.Color.RED))
except ValueError as e:
@@ -369,14 +774,19 @@ def mcp_server() -> None:
style(cs.CLI_ERR_CONFIG.format(error=e), cs.Color.RED)
)
_info(style(cs.CLI_MSG_HINT_TARGET_REPO, cs.Color.YELLOW))
-
except Exception as e:
app_context.console.print(
style(cs.CLI_ERR_MCP_SERVER.format(error=e), cs.Color.RED)
)
-@app.command(name=ch.CLICommandName.GRAPH_LOADER, help=ch.CMD_GRAPH_LOADER)
+@app.command(
+ name=ch.CLICommandName.GRAPH_LOADER,
+ help=ch.CMD_GRAPH_LOADER,
+ short_help=ch.CMD_GRAPH_LOADER,
+ epilog=ch.EXAMPLES_GRAPH_LOADER,
+ rich_help_panel=ch.PANEL_GRAPH,
+)
def graph_loader_command(
graph_file: str = typer.Argument(..., help=ch.HELP_GRAPH_FILE),
) -> None:
@@ -408,16 +818,141 @@ def graph_loader_command(
raise typer.Exit(1) from e
+_DELEGATED_GROUP_CONTEXT = {
+ "allow_extra_args": True,
+ "allow_interspersed_args": False,
+ "ignore_unknown_options": True,
+}
+
+
+def _run_delegated_group(group: click.Group, ctx: typer.Context) -> None:
+ group.main(
+ args=list(ctx.args),
+ prog_name=ctx.command_path,
+ standalone_mode=False,
+ )
+
+
@app.command(
name=ch.CLICommandName.LANGUAGE,
help=ch.CMD_LANGUAGE,
- context_settings={"allow_extra_args": True, "allow_interspersed_args": False},
+ short_help=ch.CMD_LANGUAGE,
+ add_help_option=False,
+ context_settings=_DELEGATED_GROUP_CONTEXT,
+ rich_help_panel=ch.PANEL_MANAGE,
)
def language_command(ctx: typer.Context) -> None:
- language_cli(ctx.args, standalone_mode=False)
+ _run_delegated_group(language_cli, ctx)
-@app.command(name=ch.CLICommandName.DOCTOR, help=ch.CMD_DOCTOR)
+@app.command(
+ name=ch.CLICommandName.DAEMON,
+ help=ch.CMD_DAEMON,
+ short_help=ch.CMD_DAEMON,
+ add_help_option=False,
+ context_settings=_DELEGATED_GROUP_CONTEXT,
+ rich_help_panel=ch.PANEL_MANAGE,
+)
+def daemon_command(ctx: typer.Context) -> None:
+ _run_delegated_group(daemon_cli, ctx)
+
+
+@app.command(
+ name=ch.CLICommandName.WORKSPACE,
+ help=ch.CMD_WORKSPACE,
+ short_help=ch.CMD_WORKSPACE,
+ add_help_option=False,
+ context_settings=_DELEGATED_GROUP_CONTEXT,
+ rich_help_panel=ch.PANEL_MANAGE,
+)
+def workspace_command(ctx: typer.Context) -> None:
+ _run_delegated_group(workspace_cli, ctx)
+
+
+@app.command(
+ name=ch.CLICommandName.HELP,
+ help=ch.CMD_HELP,
+ short_help=ch.CMD_HELP,
+ epilog=ch.EXAMPLES_HELP,
+ rich_help_panel=ch.PANEL_HELP,
+)
+def help_command(
+ ctx: typer.Context,
+ command: list[str] | None = typer.Argument(None, help=ch.HELP_COMMAND),
+) -> None:
+ root_context = ctx.find_root()
+ requested = command or []
+ if not requested:
+ typer.echo(root_context.get_help())
+ return
+
+ root_command = root_context.command
+ command_name, *command_args = requested
+ if not isinstance(root_command, click.Group):
+ raise typer.Exit(1)
+
+ target = root_command.get_command(root_context, command_name)
+ if target is None:
+ typer.echo(f"cgr: '{command_name}' is not a cgr command.", err=True)
+ typer.echo("See 'cgr help'.", err=True)
+ raise typer.Exit(2)
+
+ try:
+ target.main(
+ args=[*command_args, "--help"],
+ prog_name=f"{root_context.command_path} {command_name}",
+ standalone_mode=False,
+ )
+ except click.ClickException as error:
+ error.show()
+ raise typer.Exit(error.exit_code) from error
+
+
+@app.command(
+ name=ch.CLICommandName.STOP,
+ help=ch.CMD_STOP,
+ short_help=ch.CMD_STOP,
+ rich_help_panel=ch.PANEL_MANAGE,
+)
+def stop_command() -> None:
+ mgr = StackManager()
+ try:
+ mgr.down()
+ except StackError as e:
+ app_context.console.print(style(str(e), cs.Color.RED))
+ raise typer.Exit(1) from e
+ _info(style("stack stopped", cs.Color.GREEN))
+
+
+@app.command(
+ name=ch.CLICommandName.STATUS,
+ help=ch.CMD_STATUS,
+ short_help=ch.CMD_STATUS,
+ rich_help_panel=ch.PANEL_MANAGE,
+)
+def status_command() -> None:
+ status = StackManager().status()
+ app_context.console.print(
+ f"stack: {status.state.value} "
+ f"(memgraph={status.memgraph_endpoint} reachable={status.memgraph_reachable}, "
+ f"qdrant={status.qdrant_endpoint} reachable={status.qdrant_reachable})"
+ )
+ app_context.console.print(f"compose: {status.compose_file}")
+ timestamps = cgr_state.read_sync_timestamps()
+ if not timestamps:
+ app_context.console.print("syncs: (no projects synced via cgr yet)")
+ return
+ app_context.console.print("syncs:")
+ for project, ts in sorted(timestamps.items()):
+ app_context.console.print(f" - {project}: last sync {ts}")
+
+
+@app.command(
+ name=ch.CLICommandName.DOCTOR,
+ help=ch.CMD_DOCTOR,
+ short_help=ch.CMD_DOCTOR,
+ rich_help_panel=ch.PANEL_MANAGE,
+)
def doctor() -> None:
checker = HealthChecker()
results = checker.run_all_checks()
@@ -465,5 +1000,362 @@ def doctor() -> None:
raise typer.Exit(1)
+def _build_stats_table(
+ title: str,
+ col_label: str,
+ rows: list[ResultRow],
+ get_label: Callable[[ResultRow], str],
+ total_label: str,
+) -> Table:
+ table = Table(
+ title=style(title, cs.Color.GREEN),
+ show_header=True,
+ header_style=f"{cs.StyleModifier.BOLD} {cs.Color.MAGENTA}",
+ )
+ table.add_column(col_label, style=cs.Color.CYAN)
+ table.add_column(cs.CLI_STATS_COL_COUNT, style=cs.Color.YELLOW, justify="right")
+ total = 0
+ for row in rows:
+ raw_count = row.get("count", 0)
+ count = int(raw_count) if isinstance(raw_count, int | float) else 0
+ total += count
+ table.add_row(get_label(row), f"{count:,}")
+ table.add_section()
+ table.add_row(
+ style(total_label, cs.Color.GREEN),
+ style(f"{total:,}", cs.Color.GREEN),
+ )
+ return table
+
+
+@app.command(
+ name=ch.CLICommandName.STATS,
+ help=ch.CMD_STATS,
+ short_help=ch.CMD_STATS,
+ rich_help_panel=ch.PANEL_GRAPH,
+)
+def stats() -> None:
+ from .cypher_queries import (
+ CYPHER_STATS_NODE_COUNTS,
+ CYPHER_STATS_RELATIONSHIP_COUNTS,
+ )
+
+ app_context.console.print(style(cs.CLI_MSG_CONNECTING_STATS, cs.Color.CYAN))
+
+ try:
+ with connect_memgraph(batch_size=1) as ingestor:
+ node_results = ingestor.fetch_all(CYPHER_STATS_NODE_COUNTS)
+ rel_results = ingestor.fetch_all(CYPHER_STATS_RELATIONSHIP_COUNTS)
+
+ app_context.console.print(
+ _build_stats_table(
+ cs.CLI_STATS_NODE_TITLE,
+ cs.CLI_STATS_COL_NODE_TYPE,
+ node_results,
+ lambda r: ":".join(r.get("labels", [])) or cs.CLI_STATS_UNKNOWN,
+ cs.CLI_STATS_TOTAL_NODES,
+ )
+ )
+ app_context.console.print()
+ app_context.console.print(
+ _build_stats_table(
+ cs.CLI_STATS_REL_TITLE,
+ cs.CLI_STATS_COL_REL_TYPE,
+ rel_results,
+ lambda r: str(r.get("type", cs.CLI_STATS_UNKNOWN)),
+ cs.CLI_STATS_TOTAL_RELS,
+ )
+ )
+
+ except Exception as e:
+ app_context.console.print(
+ style(cs.CLI_ERR_STATS_FAILED.format(error=e), cs.Color.RED)
+ )
+ logger.exception(ls.STATS_ERROR.format(error=e))
+ raise typer.Exit(1) from e
+
+
+def _resolve_dead_code_project(
+ project_name: str | None, projects: list[str]
+) -> str | None:
+ if project_name:
+ return project_name.strip()
+ if len(projects) == 1:
+ return projects[0]
+ return None
+
+
+def _dead_code_config(
+ include_tests: bool,
+ include_classes: bool,
+ entry_points: list[str],
+ decorator_roots: list[str],
+) -> DeadCodeConfig:
+ # test_patterns is always set: included tests become roots; excluded, it
+ # filters test modules out of module-load roots so test-only code stays dead.
+ return DeadCodeConfig(
+ include_tests=include_tests,
+ include_classes=include_classes,
+ root_decorators=frozenset(
+ {d.lower() for d in cs.DEFAULT_ROOT_DECORATORS}
+ | {d.lower() for d in decorator_roots}
+ ),
+ entry_points=tuple(entry_points),
+ test_patterns=tuple(cs.TEST_PATH_PATTERNS),
+ )
+
+
+def _filter_excluded_rows(rows: list[ResultRow], exclude: list[str]) -> list[ResultRow]:
+ # Drop candidates whose file path matches an exclude glob (generated dirs
+ # like client/core or *.gen.* have no in-repo caller, so every symbol reports
+ # as dead). fnmatch's '*' spans '/', so '*client/core*' matches at any depth.
+ if not exclude:
+ return rows
+ return [
+ row
+ for row in rows
+ if not any(
+ fnmatch(str(row.get(cs.KEY_PATH) or ""), pattern) for pattern in exclude
+ )
+ ]
+
+
+def _to_dead_code_row(row: ResultRow) -> DeadCodeRow:
+ start = row.get(cs.KEY_START_LINE, 0)
+ end = row.get(cs.KEY_END_LINE, 0)
+ return DeadCodeRow(
+ label=str(row.get(cs.KEY_LABEL, "")),
+ name=str(row.get(cs.KEY_NAME, "")),
+ qualified_name=str(row.get(cs.KEY_QUALIFIED_NAME, "")),
+ start_line=int(start) if isinstance(start, int | float) else 0,
+ end_line=int(end) if isinstance(end, int | float) else 0,
+ )
+
+
+def _build_dead_code_table(candidates: list[DeadCodeRow], project_name: str) -> Table:
+ table = Table(
+ title=style(
+ cs.CLI_DEADCODE_TABLE_TITLE.format(project_name=project_name),
+ cs.Color.GREEN,
+ ),
+ show_header=True,
+ header_style=f"{cs.StyleModifier.BOLD} {cs.Color.MAGENTA}",
+ )
+ table.add_column(cs.CLI_DEADCODE_COL_KIND, style=cs.Color.MAGENTA)
+ table.add_column(cs.CLI_DEADCODE_COL_QUALIFIED_NAME, style=cs.Color.CYAN)
+ table.add_column(cs.CLI_DEADCODE_COL_LINES, style=cs.Color.YELLOW, justify="right")
+ for row in candidates:
+ table.add_row(
+ row["label"],
+ row["qualified_name"],
+ cs.CLI_DEADCODE_LINE_RANGE.format(
+ start=row["start_line"], end=row["end_line"]
+ ),
+ )
+ return table
+
+
+def _emit_dead_code(
+ candidates: list[DeadCodeRow],
+ output_format: cs.DeadCodeFormat,
+ output: Path | None,
+ project_name: str,
+) -> None:
+ if output_format == cs.DeadCodeFormat.JSON:
+ payload = json.dumps(candidates, indent=2)
+ if output is not None:
+ output.write_text(payload, encoding=cs.ENCODING_UTF8)
+ app_context.console.print(
+ style(
+ cs.CLI_DEADCODE_WRITTEN.format(count=len(candidates), path=output),
+ cs.Color.GREEN,
+ )
+ )
+ return
+ typer.echo(payload)
+ return
+
+ table = _build_dead_code_table(candidates, project_name)
+ if output is not None:
+ with output.open("w", encoding=cs.ENCODING_UTF8) as fh:
+ Console(file=fh).print(table)
+ app_context.console.print(
+ style(
+ cs.CLI_DEADCODE_WRITTEN.format(count=len(candidates), path=output),
+ cs.Color.GREEN,
+ )
+ )
+ return
+
+ if not candidates:
+ app_context.console.print(style(cs.CLI_DEADCODE_NONE, cs.Color.GREEN))
+ return
+ app_context.console.print(table)
+ app_context.console.print(
+ style(cs.CLI_DEADCODE_SUMMARY.format(count=len(candidates)), cs.Color.GREEN)
+ )
+
+
+@app.command(
+ name=ch.CLICommandName.DEAD_CODE,
+ help=ch.CMD_DEAD_CODE,
+ short_help=ch.CMD_DEAD_CODE,
+ epilog=ch.EXAMPLES_DEAD_CODE,
+ rich_help_panel=ch.PANEL_GRAPH,
+)
+def dead_code(
+ project_name: str | None = typer.Option(
+ None, "--project-name", "-n", help=ch.HELP_DEADCODE_PROJECT_NAME
+ ),
+ entry_point: list[str] = typer.Option(
+ [], "--entry-point", "-e", help=ch.HELP_DEADCODE_ENTRY_POINT
+ ),
+ decorator_root: list[str] = typer.Option(
+ [], "--decorator-root", help=ch.HELP_DEADCODE_DECORATOR_ROOT
+ ),
+ exclude: list[str] = typer.Option([], "--exclude", help=ch.HELP_DEADCODE_EXCLUDE),
+ include_tests: bool = typer.Option(
+ True,
+ "--include-tests/--no-include-tests",
+ help=ch.HELP_DEADCODE_INCLUDE_TESTS,
+ ),
+ include_classes: bool = typer.Option(
+ False,
+ "--classes/--no-classes",
+ help=ch.HELP_DEADCODE_CLASSES,
+ ),
+ output_format: cs.DeadCodeFormat = typer.Option(
+ cs.DeadCodeFormat.TABLE, "--format", help=ch.HELP_DEADCODE_FORMAT
+ ),
+ output: Path | None = typer.Option(
+ None, "--output", "-o", help=ch.HELP_DEADCODE_OUTPUT
+ ),
+ fail_on_found: bool = typer.Option(
+ False, "--fail-on-found", help=ch.HELP_DEADCODE_FAIL_ON_FOUND
+ ),
+) -> None:
+ from .dead_code import collect_dead_code
+
+ show_progress = output_format == cs.DeadCodeFormat.TABLE and output is None
+ if show_progress:
+ app_context.console.print(style(cs.CLI_DEADCODE_CONNECTING, cs.Color.CYAN))
+
+ projects: list[str] = []
+ resolved: str | None = None
+ rows: list[ResultRow] = []
+ try:
+ with connect_memgraph(batch_size=1) as ingestor:
+ projects = ingestor.list_projects()
+ resolved = _resolve_dead_code_project(project_name, projects)
+ if resolved is not None:
+ logger.info(ls.DEADCODE_SCANNING.format(project_name=resolved))
+ rows = collect_dead_code(
+ ingestor,
+ resolved,
+ _dead_code_config(
+ include_tests, include_classes, entry_point, decorator_root
+ ),
+ )
+ except Exception as e:
+ app_context.console.print(
+ style(cs.CLI_ERR_DEADCODE_FAILED.format(error=e), cs.Color.RED)
+ )
+ logger.exception(ls.DEADCODE_ERROR.format(error=e))
+ raise typer.Exit(1) from e
+
+ if resolved is None:
+ message = (
+ cs.CLI_ERR_DEADCODE_NO_PROJECTS
+ if not projects
+ else cs.CLI_ERR_DEADCODE_AMBIGUOUS_PROJECT.format(projects=projects)
+ )
+ app_context.console.print(style(message, cs.Color.RED))
+ raise typer.Exit(1)
+
+ candidates = [
+ _to_dead_code_row(row) for row in _filter_excluded_rows(rows, exclude)
+ ]
+ _emit_dead_code(candidates, output_format, output, resolved)
+
+ if fail_on_found and candidates:
+ raise typer.Exit(1)
+
+
+@app.command(
+ name=ch.CLICommandName.DELETE_PROJECT,
+ help=ch.CMD_DELETE_PROJECT,
+ short_help=ch.CMD_DELETE_PROJECT,
+ epilog=ch.EXAMPLES_DELETE_PROJECT,
+ rich_help_panel=ch.PANEL_GRAPH,
+)
+def delete_project(
+ name: str = typer.Option(
+ ...,
+ "--name",
+ "-n",
+ help=ch.HELP_DELETE_PROJECT_NAME,
+ ),
+ repo_path: str | None = typer.Option(
+ None,
+ "--repo-path",
+ help=ch.HELP_DELETE_PROJECT_REPO_PATH,
+ ),
+) -> None:
+ project_name = name.strip()
+ if not project_name:
+ app_context.console.print(style(cs.CLI_ERR_PROJECT_NAME_REQUIRED, cs.Color.RED))
+ raise typer.Exit(1)
+
+ effective_batch_size = settings.resolve_batch_size(None)
+
+ try:
+ with connect_memgraph(effective_batch_size) as ingestor:
+ projects = ingestor.list_projects()
+ if project_name not in projects:
+ app_context.console.print(
+ style(
+ cs.CLI_ERR_PROJECT_NOT_FOUND.format(
+ project_name=project_name, projects=projects
+ ),
+ cs.Color.RED,
+ )
+ )
+ raise typer.Exit(1)
+
+ _info(
+ style(
+ cs.CLI_MSG_DELETING_PROJECT.format(project_name=project_name),
+ cs.Color.YELLOW,
+ )
+ )
+ _cleanup_project_embeddings(ingestor, project_name)
+ ingestor.delete_project(project_name)
+ except typer.Exit:
+ raise
+ except Exception as e:
+ app_context.console.print(
+ style(
+ cs.CLI_ERR_DELETE_PROJECT_FAILED.format(
+ project_name=project_name, error=e
+ ),
+ cs.Color.RED,
+ )
+ )
+ logger.exception(
+ cs.CLI_ERR_DELETE_PROJECT_FAILED.format(project_name=project_name, error=e)
+ )
+ raise typer.Exit(1) from e
+
+ if repo_path:
+ _delete_hash_cache(Path(repo_path))
+
+ _info(
+ style(
+ cs.CLI_MSG_PROJECT_DELETED.format(project_name=project_name),
+ cs.Color.GREEN,
+ )
+ )
+
+
if __name__ == "__main__":
app()
diff --git a/codebase_rag/cli_help.py b/codebase_rag/cli_help.py
index 96e816d9a..c13b23aa1 100644
--- a/codebase_rag/cli_help.py
+++ b/codebase_rag/cli_help.py
@@ -10,84 +10,229 @@ class CLICommandName(StrEnum):
GRAPH_LOADER = "graph-loader"
LANGUAGE = "language"
DOCTOR = "doctor"
+ STATS = "stats"
+ DEAD_CODE = "dead-code"
+ DELETE_PROJECT = "delete-project"
+ DAEMON = "daemon"
+ WORKSPACE = "workspace"
+ STOP = "stop"
+ STATUS = "status"
+ HELP = "help"
APP_DESCRIPTION = (
- "An accurate Retrieval-Augmented Generation (RAG) system that analyzes "
- "multi-language codebases using Tree-sitter, builds comprehensive knowledge "
- "graphs, and enables natural language querying of codebase structure and relationships."
-)
-
-CMD_START = "Start interactive chat session with your codebase"
-CMD_INDEX = "Index codebase to protobuf files for offline use"
-CMD_EXPORT = "Export knowledge graph from Memgraph to JSON file"
-CMD_OPTIMIZE = "AI-guided codebase optimization session"
-CMD_MCP_SERVER = "Start the MCP server for Claude Code integration"
-CMD_GRAPH_LOADER = "Load and display summary of exported graph JSON"
-CMD_LANGUAGE = "Manage language grammars (add, remove, list)"
-CMD_DOCTOR = "Verify that all dependencies and configurations are properly set up"
-
-CMD_LANGUAGE_GROUP = "CLI for managing language grammars"
-CMD_LANGUAGE_ADD = "Add a new language grammar to the project."
-CMD_LANGUAGE_LIST = "List all currently configured languages."
-CMD_LANGUAGE_REMOVE = "Remove a language from the project."
-CMD_LANGUAGE_CLEANUP = "Clean up orphaned git modules that weren't properly removed."
-
-HELP_BATCH_SIZE = "Number of buffered nodes/relationships before flushing to Memgraph"
-HELP_MEMGRAPH_HOST = "Memgraph host"
-HELP_MEMGRAPH_PORT = "Memgraph port"
+ "Analyse source code with Tree-sitter, store its structure in a shared "
+ "knowledge graph, and query it in natural language."
+)
+APP_EPILOG = "Run 'cgr help COMMAND' for details about a command."
+
+PANEL_USE = "Query and improve code"
+PANEL_GRAPH = "Build and inspect the graph"
+PANEL_MANAGE = "Manage cgr"
+PANEL_HELP = "Help"
+
+CMD_START = "Open the code assistant for a repository or workspace"
+CMD_INDEX = "Write an offline protobuf index for a repository"
+CMD_EXPORT = "Export the shared graph database to JSON"
+CMD_OPTIMIZE = "Run a language-focused code optimisation session"
+CMD_MCP_SERVER = "Serve cgr tools over stdio or HTTP"
+CMD_GRAPH_LOADER = "Summarise an exported graph JSON file"
+CMD_LANGUAGE = "Manage language grammars and parser metadata"
+CMD_DOCTOR = "Check dependencies, services, and configuration"
+CMD_STATS = "Show graph node and relationship counts"
+CMD_DEAD_CODE = "Report code that appears unreachable from known entry points"
+CMD_DELETE_PROJECT = "Delete one project without changing other indexed projects"
+CMD_HELP = "Show help for a command"
+
+CMD_LANGUAGE_GROUP = CMD_LANGUAGE
+CMD_LANGUAGE_ADD = "Add and register a Tree-sitter grammar"
+CMD_LANGUAGE_LIST = "List configured languages and their node mappings"
+CMD_LANGUAGE_REMOVE = "Remove a language from cgr configuration"
+CMD_LANGUAGE_CLEANUP = "Remove orphaned grammar entries under .git/modules"
+
+CMD_DAEMON = "Manage the shared Memgraph and Qdrant stack"
+CMD_DAEMON_GROUP = CMD_DAEMON
+CMD_DAEMON_UP = "Start the shared stack and wait until it is healthy"
+CMD_DAEMON_DOWN = "Stop the shared stack and preserve its data volumes"
+CMD_DAEMON_STATUS = "Show stack state and service reachability"
+CMD_DAEMON_LOGS = "Show Docker Compose logs for the shared stack"
+CMD_DAEMON_RESTART = "Restart the shared stack and wait for health checks"
+
+CMD_WORKSPACE = "Manage named groups of repositories"
+CMD_WORKSPACE_GROUP = CMD_WORKSPACE
+CMD_WORKSPACE_LIST = "List saved workspaces"
+CMD_WORKSPACE_CREATE = "Create an empty workspace definition"
+CMD_WORKSPACE_DELETE = "Delete a workspace definition but keep indexed graph data"
+CMD_WORKSPACE_SHOW = "Show the repositories and project names in a workspace"
+CMD_WORKSPACE_ADD_REPO = "Add a repository to a workspace"
+CMD_WORKSPACE_REMOVE_REPO = "Remove a repository from a workspace by path"
+
+CMD_STOP = "Stop the shared stack (alias for cgr daemon down)"
+CMD_STATUS = "Show stack state and the last sync time for each project"
+
+EXAMPLES_START = (
+ "EXAMPLES\n\n"
+ " cgr start --repo-path ./my-repo\n\n"
+ " cgr start --workspace backend\n\n"
+ ' cgr start --ask-agent "Where is authentication handled?"'
+)
+EXAMPLES_INDEX = "EXAMPLE\n\n cgr index --repo-path ./my-repo -o ./index-out"
+EXAMPLES_EXPORT = "EXAMPLE\n\n cgr export -o graph.json"
+EXAMPLES_OPTIMIZE = "EXAMPLE\n\n cgr optimize python --repo-path ./my-repo"
+EXAMPLES_MCP_SERVER = (
+ "EXAMPLES\n\n cgr mcp-server\n\n cgr mcp-server --transport http --port 8080"
+)
+EXAMPLES_GRAPH_LOADER = "EXAMPLE\n\n cgr graph-loader graph.json"
+EXAMPLES_LANGUAGE_ADD = "EXAMPLE\n\n cgr language add-grammar ruby"
+EXAMPLES_LANGUAGE_REMOVE = "EXAMPLE\n\n cgr language remove-language ruby"
+EXAMPLES_DEAD_CODE = (
+ "EXAMPLE\n\n cgr dead-code --project-name my-project --format json"
+)
+EXAMPLES_DELETE_PROJECT = "EXAMPLE\n\n cgr delete-project --name my-project"
+EXAMPLES_HELP = "EXAMPLES\n\n cgr help start\n\n cgr help daemon logs"
+
+EPILOG_LANGUAGE = "Run 'cgr help language COMMAND' for command-specific help."
+EPILOG_DAEMON = "Run 'cgr help daemon COMMAND' for command-specific help."
+EPILOG_WORKSPACE = "Run 'cgr help workspace COMMAND' for command-specific help."
+
+HELP_WORKSPACE_DESCRIPTION = "Optional short description for the workspace."
+HELP_WORKSPACE_FORCE = "Overwrite an existing workspace with the same name."
+HELP_WORKSPACE_REPO_PROJECT_NAME = (
+ "Project name to use for this repo. Defaults to the derived repo name."
+)
+
+MSG_NO_WORKSPACES = "(no workspaces; create one with 'cgr workspace create ')"
+
+HELP_DAEMON_LOGS_FOLLOW = "Continue printing new log entries until interrupted."
+HELP_DAEMON_LOGS_SERVICE = (
+ "Show only SERVICE logs (memgraph, qdrant, or lab). By default, show all services."
+)
+HELP_NO_START_STACK = "Do not start the shared stack automatically."
+HELP_NO_SYNC = "Do not synchronise the graph before starting the assistant."
+HELP_NO_EMBEDDINGS = (
+ "Do not generate semantic embeddings during sync. Graph nodes and relationships "
+ "are still updated. Equivalent env: CGR_SKIP_EMBEDDINGS=1."
+)
+HELP_PROJECTS = (
+ "Limit queries to comma-separated project names. Overrides --project-name; "
+ "defaults to the selected repository or workspace."
+)
+HELP_WORKSPACE = "Query every project defined in workspace NAME."
+
+HELP_BATCH_SIZE = "Flush to Memgraph after this many buffered nodes or relationships."
+HELP_MEMGRAPH_HOST = "Memgraph host."
+HELP_MEMGRAPH_PORT = "Memgraph port."
HELP_ORCHESTRATOR = (
- "Specify orchestrator as provider:model "
- "(e.g., ollama:llama3.2, openai:gpt-4, google:gemini-2.5-pro)"
-)
-HELP_CYPHER_MODEL = (
- "Specify cypher model as provider:model "
- "(e.g., ollama:codellama, google:gemini-2.5-flash)"
-)
-HELP_NO_CONFIRM = "Disable confirmation prompts for edit operations (YOLO mode)"
-
-HELP_REPO_PATH_RETRIEVAL = "Path to the target repository for code retrieval"
-HELP_REPO_PATH_INDEX = "Path to the target repository to index."
-HELP_REPO_PATH_OPTIMIZE = "Path to the repository to optimize"
-HELP_REPO_PATH_WATCH = "Path to the repository to watch."
-
-HELP_UPDATE_GRAPH = "Update the knowledge graph by parsing the repository"
-HELP_CLEAN_DB = "Clean the database before updating (use when adding first repo)"
-HELP_OUTPUT_GRAPH = "Export graph to JSON file after updating (requires --update-graph)"
-HELP_OUTPUT_PATH = "Output file path for the exported graph"
-HELP_OUTPUT_PROTO_DIR = (
- "Required. Path to the output directory for the protobuf index file(s)."
-)
-HELP_SPLIT_INDEX = "Write index to separate nodes.bin and relationships.bin files."
-HELP_FORMAT_JSON = "Export in JSON format"
-HELP_LANGUAGE_ARG = (
- "Programming language to optimize for (e.g., python, java, javascript, cpp)"
-)
-HELP_REFERENCE_DOC = "Path to reference document/book for optimization guidance"
-HELP_GRAPH_FILE = "Path to the exported graph JSON file"
+ "Model for the planning assistant, in provider:model form "
+ "(for example openai:gpt-4 or ollama:llama3.2)."
+)
+HELP_CYPHER_MODEL = "Model used to generate Cypher, in provider:model form."
+HELP_NO_CONFIRM = "Skip edit confirmation prompts."
+HELP_NO_INSTRUCTIONS = (
+ "Do not load ~/.cgr.md or /.cgr.md into the session prompt."
+)
+
+HELP_REPO_PATH_RETRIEVAL = "Repository to open. Defaults to the current directory."
+HELP_REPO_PATH_INDEX = "Repository to index. Defaults to the current directory."
+HELP_REPO_PATH_OPTIMIZE = "Repository to optimise. Defaults to the current directory."
+HELP_REPO_PATH_WATCH = "Repository to watch."
+HELP_VERSION = "Show the version and exit."
+HELP_QUIET = "Suppress progress, banners, and informational logs."
+
+HELP_DEBOUNCE = "Debounce delay in seconds. Set to 0 to disable debouncing."
+HELP_MAX_WAIT = (
+ "Maximum wait time in seconds before forcing an update during continuous edits."
+)
+
+HELP_UPDATE_GRAPH = "Parse the repository and sync its graph before continuing."
+HELP_CLEAN_DB = (
+ "Delete every project from the shared graph and clear the selected repository's "
+ "sync cache. With --update-graph, rebuild after deletion."
+)
+HELP_OUTPUT_GRAPH = "Write the updated graph to PATH as JSON. Requires --update-graph."
+HELP_OUTPUT_PATH = "Write the exported graph to PATH."
+HELP_OUTPUT_PROTO_DIR = "Write protobuf index files under DIRECTORY."
+HELP_SPLIT_INDEX = "Write separate nodes.bin and relationships.bin files."
+HELP_FORMAT_JSON = "Use JSON output. Other export formats are not supported."
+HELP_LANGUAGE_ARG = "Language to optimise, such as python, java, javascript, or cpp."
+HELP_REFERENCE_DOC = "Reference document to use during optimisation."
+HELP_GRAPH_FILE = "Exported graph JSON file to load."
HELP_EXPORTED_GRAPH_FILE = "Path to the exported_graph.json file."
HELP_GRAMMAR_URL = (
- "URL to the tree-sitter grammar repository. If not provided, "
- "will use https://github.com/tree-sitter/tree-sitter-"
+ "Tree-sitter grammar repository URL. Defaults to "
+ "https://github.com/tree-sitter/tree-sitter-."
+)
+HELP_KEEP_SUBMODULE = (
+ "Keep the grammar git submodule when removing the language. By default, remove it."
)
-HELP_KEEP_SUBMODULE = "Keep the git submodule (default: remove it)"
+HELP_PROJECT_NAME = (
+ "Project name to store in the graph. Defaults to the repo directory name."
+)
HELP_EXCLUDE_PATTERNS = (
- "Additional directories to exclude from indexing. Can be specified multiple times."
+ "Exclude paths matching PATTERN from indexing. Repeat the option to add patterns."
)
-HELP_INTERACTIVE_SETUP = (
- "Show interactive prompt to select which detected directories to keep. "
- "Without this flag, all directories matching ignore patterns are automatically excluded."
+HELP_INTERACTIVE_SETUP = "Choose which detected directories remain included."
+HELP_CAPTURE = (
+ "Capture GROUP (structure, calls, types, imports, io), all/none, or a +TYPE/-TYPE "
+ "override. Repeatable; later values override CGR_CAPTURE."
)
+HELP_ASK_AGENT = "Ask one question, write the answer to stdout, and exit."
+
+HELP_QUERY_OUTPUT_FORMAT = "Format --ask-agent output as table or json."
+
+HELP_MCP_TRANSPORT = "Transport to serve: stdio or http."
+HELP_MCP_HTTP_HOST = "HTTP bind host. Used only with --transport http."
+HELP_MCP_HTTP_PORT = "HTTP bind port. Used only with --transport http."
+
+HELP_DEADCODE_PROJECT_NAME = (
+ "Project to scan. If omitted, cgr uses the only indexed project."
+)
+HELP_DEADCODE_ENTRY_POINT = (
+ "Mark symbols ending with this qualified-name suffix as entry points. Repeatable."
+)
+HELP_DEADCODE_DECORATOR_ROOT = (
+ "Mark symbols with this decorator as entry points. Extends the built-in set."
+)
+HELP_DEADCODE_EXCLUDE = (
+ "Exclude symbols whose file path matches GLOB. '*' spans directories. Repeatable."
+)
+HELP_DEADCODE_INCLUDE_TESTS = (
+ "Treat test code as reachable so exercised production code is not reported."
+)
+HELP_DEADCODE_CLASSES = (
+ "Also report unreachable classes. This can include false positives for "
+ "types used only by annotations or dynamic lookups."
+)
+HELP_DEADCODE_FORMAT = "Report format: table or json."
+HELP_DEADCODE_OUTPUT = "Write the report to this file instead of stdout."
+HELP_DEADCODE_FAIL_ON_FOUND = (
+ "Exit with status 1 when any candidate is found. Useful in CI."
+)
+
+HELP_DELETE_PROJECT_NAME = "Project name to delete from the graph."
+HELP_DELETE_PROJECT_REPO_PATH = (
+ "Optional repo path. If set, the local hash cache is removed too."
+)
+HELP_COMMAND = "Command path to document, such as 'start' or 'daemon logs'."
+
CLI_COMMANDS: dict[CLICommandName, str] = {
CLICommandName.START: CMD_START,
- CLICommandName.INDEX: CMD_INDEX,
- CLICommandName.EXPORT: CMD_EXPORT,
CLICommandName.OPTIMIZE: CMD_OPTIMIZE,
CLICommandName.MCP_SERVER: CMD_MCP_SERVER,
+ CLICommandName.INDEX: CMD_INDEX,
+ CLICommandName.EXPORT: CMD_EXPORT,
CLICommandName.GRAPH_LOADER: CMD_GRAPH_LOADER,
+ CLICommandName.STATS: CMD_STATS,
+ CLICommandName.DEAD_CODE: CMD_DEAD_CODE,
+ CLICommandName.DELETE_PROJECT: CMD_DELETE_PROJECT,
CLICommandName.LANGUAGE: CMD_LANGUAGE,
+ CLICommandName.DAEMON: CMD_DAEMON,
+ CLICommandName.WORKSPACE: CMD_WORKSPACE,
+ CLICommandName.STOP: CMD_STOP,
+ CLICommandName.STATUS: CMD_STATUS,
CLICommandName.DOCTOR: CMD_DOCTOR,
+ CLICommandName.HELP: CMD_HELP,
}
diff --git a/codebase_rag/config.py b/codebase_rag/config.py
index 31848e4d1..eeae6ae52 100644
--- a/codebase_rag/config.py
+++ b/codebase_rag/config.py
@@ -1,5 +1,8 @@
+"""Runtime configuration: settings, model providers, and environment loading."""
+
from __future__ import annotations
+import os
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import TypedDict, Unpack
@@ -44,10 +47,10 @@ class ApiKeyInfoEntry(TypedDict):
"url": "https://portal.azure.com/",
"name": "Azure OpenAI",
},
- cs.Provider.COHERE: {
- "env_var": "COHERE_API_KEY",
- "url": "https://dashboard.cohere.com/api-keys",
- "name": "Cohere",
+ cs.Provider.MINIMAX: {
+ "env_var": "MINIMAX_API_KEY",
+ "url": "https://platform.minimax.io/user-center/basic-information/interface-key",
+ "name": "MiniMax",
},
}
@@ -94,6 +97,9 @@ def format_missing_api_key_errors(
return error_msg
+LOCAL_PROVIDERS = frozenset({cs.Provider.OLLAMA})
+
+
@dataclass
class ModelConfig:
provider: str
@@ -113,8 +119,21 @@ def to_update_kwargs(self) -> ModelConfigKwargs:
return ModelConfigKwargs(**result)
def validate_api_key(self, role: str = cs.DEFAULT_MODEL_ROLE) -> None:
- local_providers = {cs.Provider.OLLAMA, cs.Provider.LOCAL, cs.Provider.VLLM}
- if self.provider.lower() in local_providers:
+ provider_lower = self.provider.lower()
+ provider_env_keys = {
+ cs.Provider.ANTHROPIC: cs.ENV_ANTHROPIC_API_KEY,
+ cs.Provider.AZURE: cs.ENV_AZURE_API_KEY,
+ cs.Provider.MINIMAX: cs.ENV_MINIMAX_API_KEY,
+ }
+ env_key = provider_env_keys.get(provider_lower)
+ if (
+ provider_lower in LOCAL_PROVIDERS
+ or (
+ provider_lower == cs.Provider.GOOGLE
+ and self.provider_type == cs.GoogleProviderType.VERTEX
+ )
+ or (env_key and os.environ.get(env_key))
+ ):
return
if (
not self.api_key
@@ -127,7 +146,7 @@ def validate_api_key(self, role: str = cs.DEFAULT_MODEL_ROLE) -> None:
class AppConfig(BaseSettings):
"""
- (H) All settings are loaded from environment variables or a .env file.
+ All settings are loaded from environment variables or a .env file.
"""
model_config = SettingsConfigDict(
@@ -139,6 +158,8 @@ class AppConfig(BaseSettings):
MEMGRAPH_HOST: str = "localhost"
MEMGRAPH_PORT: int = 7687
MEMGRAPH_HTTP_PORT: int = 7444
+ MEMGRAPH_USERNAME: str | None = None
+ MEMGRAPH_PASSWORD: str | None = None
LAB_PORT: int = 3000
MEMGRAPH_BATCH_SIZE: int = 1000
AGENT_RETRIES: int = 3
@@ -150,7 +171,7 @@ class AppConfig(BaseSettings):
ORCHESTRATOR_ENDPOINT: str | None = None
ORCHESTRATOR_PROJECT_ID: str | None = None
ORCHESTRATOR_REGION: str = cs.DEFAULT_REGION
- ORCHESTRATOR_PROVIDER_TYPE: str | None = None
+ ORCHESTRATOR_PROVIDER_TYPE: cs.GoogleProviderType | None = None
ORCHESTRATOR_THINKING_BUDGET: int | None = None
ORCHESTRATOR_SERVICE_ACCOUNT_FILE: str | None = None
@@ -160,7 +181,7 @@ class AppConfig(BaseSettings):
CYPHER_ENDPOINT: str | None = None
CYPHER_PROJECT_ID: str | None = None
CYPHER_REGION: str = cs.DEFAULT_REGION
- CYPHER_PROVIDER_TYPE: str | None = None
+ CYPHER_PROVIDER_TYPE: cs.GoogleProviderType | None = None
CYPHER_THINKING_BUDGET: int | None = None
CYPHER_SERVICE_ACCOUNT_FILE: str | None = None
@@ -171,6 +192,19 @@ def ollama_endpoint(self) -> str:
return f"{self.OLLAMA_BASE_URL.rstrip('/')}/v1"
TARGET_REPO_PATH: str = "."
+ # HYBRID degrades to pure tree-sitter when libclang or compile_commands.json
+ # is missing, so it is a safe default and strictly better (macros, includes,
+ # expansion calls) with one.
+ CPP_FRONTEND: cs.CppFrontend = cs.CppFrontend.HYBRID
+ # Opt-in Roslyn semantic layer for C#. Defaults to pure tree-sitter because
+ # HYBRID needs a dotnet SDK + a restorable .csproj/.sln and degrades without
+ # them. HYBRID augments (base-vs-interface, overload and extension binding,
+ # partial-class identity); tree-sitter stays the standalone-correct backbone.
+ CSHARP_FRONTEND: cs.CSharpFrontend = cs.CSharpFrontend.AUTO
+ CAPTURE_FUNCTION_LOCAL_DEFINITIONS: bool = Field(
+ True, validation_alias="CGR_CAPTURE_LOCAL_DEFINITIONS"
+ )
+ CGR_HOME: Path = Field(default_factory=lambda: Path.home() / ".cgr")
SHELL_COMMAND_TIMEOUT: int = 30
SHELL_COMMAND_ALLOWLIST: frozenset[str] = frozenset(
{
@@ -235,24 +269,72 @@ def ollama_endpoint(self) -> str:
)
QDRANT_DB_PATH: str = "./.qdrant_code_embeddings"
+ QDRANT_URL: str | None = None
QDRANT_COLLECTION_NAME: str = "code_embeddings"
QDRANT_VECTOR_DIM: int = 768
QDRANT_TOP_K: int = 5
+ QDRANT_UPSERT_RETRIES: int = Field(default=3, gt=0)
+ QDRANT_RETRY_BASE_DELAY: float = Field(default=0.5, gt=0)
+ QDRANT_BATCH_SIZE: int = Field(default=50, gt=0)
+ VECTOR_STORE_BACKEND: cs.VectorStoreBackend = Field(
+ cs.VectorStoreBackend.QDRANT, validation_alias="CGR_VECTOR_STORE_BACKEND"
+ )
+ MILVUS_URI: str = "./.milvus_code_embeddings.db"
+ MILVUS_TOKEN: str | None = None
+ MILVUS_DB_NAME: str | None = None
+ MILVUS_COLLECTION_NAME: str = "code_embeddings"
+ MILVUS_VECTOR_DIM: int = 768
+ MILVUS_TOP_K: int = 5
+ MILVUS_CONSISTENCY_LEVEL: str = "Strong"
+ EMBEDDING_PROVIDER: cs.EmbeddingProvider = Field(
+ cs.EmbeddingProvider.UNIXCODER, validation_alias="CGR_EMBEDDING_PROVIDER"
+ )
+ OPENAI_EMBEDDING_BASE_URL: str = cs.OPENAI_DEFAULT_ENDPOINT
+ OPENAI_EMBEDDING_MODEL: str = cs.OPENAI_EMBEDDING_DEFAULT_MODEL
+ OPENAI_EMBEDDING_API_KEY: str | None = None
+ OPENAI_EMBEDDING_DIMENSIONS: int | None = Field(default=None, gt=0)
+ OPENAI_EMBEDDING_BATCH_SIZE: int = Field(default=128, gt=0)
+ OPENAI_EMBEDDING_TIMEOUT: float = Field(default=60.0, gt=0)
EMBEDDING_MAX_LENGTH: int = 512
EMBEDDING_PROGRESS_INTERVAL: int = 10
+ SKIP_EMBEDDINGS: bool = Field(False, validation_alias="CGR_SKIP_EMBEDDINGS")
+ EMBEDDING_DEVICE: cs.EmbeddingDevice | None = Field(
+ None, validation_alias="CGR_EMBEDDING_DEVICE"
+ )
+
+ FLUSH_THREAD_POOL_SIZE: int = Field(default=4, gt=0)
+ FILE_FLUSH_INTERVAL: int = Field(default=500, gt=0)
CACHE_MAX_ENTRIES: int = 1000
CACHE_MAX_MEMORY_MB: int = 500
CACHE_EVICTION_DIVISOR: int = 10
CACHE_MEMORY_THRESHOLD_RATIO: float = 0.8
+ QUERY_RESULT_MAX_TOKENS: int = Field(default=16000, gt=0)
+ QUERY_RESULT_ROW_CAP: int = Field(default=500, gt=0)
+ QUERY_MEMORY_LIMIT_MB: int = Field(default=4096, gt=0)
+ QUERY_TIMEOUT_S: float = Field(default=60.0, gt=0)
+
OLLAMA_HEALTH_TIMEOUT: float = 5.0
+ LITELLM_HEALTH_TIMEOUT: float = 5.0
_active_orchestrator: ModelConfig | None = None
_active_cypher: ModelConfig | None = None
QUIET: bool = Field(False, validation_alias="CGR_QUIET")
+ CGR_CAPTURE: str = Field("", validation_alias="CGR_CAPTURE")
+
+ # Loopback by default: the StreamableHTTP endpoint has no built-in
+ # auth, so exposing it beyond the host must be an explicit operator
+ # choice via MCP_HTTP_HOST (issue #808).
+ MCP_HTTP_HOST: str = "127.0.0.1"
+ MCP_HTTP_PORT: int = 8080
+ MCP_HTTP_ENDPOINT_PATH: str = "/mcp"
+ # Bearer token for the HTTP MCP endpoint; unset means loopback-only
+ # (serve_http refuses a non-loopback bind without it).
+ MCP_HTTP_AUTH_TOKEN: str | None = None
+
def _get_default_config(self, role: str) -> ModelConfig:
role_upper = role.upper()
@@ -325,13 +407,13 @@ def resolve_batch_size(self, batch_size: int | None) -> int:
settings = AppConfig()
CGRIGNORE_FILENAME = ".cgrignore"
+GITIGNORE_FILENAME = ".gitignore"
EMPTY_CGRIGNORE = CgrignorePatterns(exclude=frozenset(), unignore=frozenset())
-def load_cgrignore_patterns(repo_path: Path) -> CgrignorePatterns:
- ignore_file = repo_path / CGRIGNORE_FILENAME
+def _load_ignore_file(ignore_file: Path) -> CgrignorePatterns:
if not ignore_file.is_file():
return EMPTY_CGRIGNORE
@@ -359,6 +441,61 @@ def load_cgrignore_patterns(repo_path: Path) -> CgrignorePatterns:
exclude=frozenset(exclude),
unignore=frozenset(unignore),
)
- except OSError as e:
+ except (OSError, ValueError) as e:
logger.warning(logs.CGRIGNORE_READ_FAILED.format(path=ignore_file, error=e))
return EMPTY_CGRIGNORE
+
+
+def load_cgrignore_patterns(repo_path: Path) -> CgrignorePatterns:
+ return _load_ignore_file(repo_path / CGRIGNORE_FILENAME)
+
+
+def load_ignore_patterns(repo_path: Path) -> CgrignorePatterns:
+ # Merged exclude/unignore set for indexing: root .gitignore (gitignored
+ # paths are build artifacts and generated output that pollute the graph and
+ # dead-code report) plus .cgrignore, the authoritative cgr channel. The skip
+ # check gives excludes precedence, so a negation overrides a .gitignore
+ # exclude only by CANCELLING the exact pattern (`!generated/` drops
+ # `generated/`); .cgrignore excludes are never cancelled.
+ # ponytail: root .gitignore only, exact-string cancellation only; a
+ # finer-grained negation (`!dist/keep.py` under excluded `dist/`) still
+ # cannot rescue -- an ordered PathSpec soft layer in should_skip_path is
+ # the upgrade path if real repos need it.
+ cgr = _load_ignore_file(repo_path / CGRIGNORE_FILENAME)
+ git = _load_ignore_file(repo_path / GITIGNORE_FILENAME)
+ negations = cgr.unignore | git.unignore
+ return CgrignorePatterns(
+ exclude=cgr.exclude | (git.exclude - negations),
+ unignore=negations,
+ )
+
+
+CGR_INSTRUCTIONS_FILENAME = ".cgr.md"
+GLOBAL_CGR_INSTRUCTIONS_PATH = Path.home() / CGR_INSTRUCTIONS_FILENAME
+
+
+def _read_cgr_instructions_file(path: Path) -> str | None:
+ if not path.is_file():
+ return None
+ try:
+ with path.open(encoding="utf-8") as f:
+ body = f.read().strip()
+ except OSError as e:
+ logger.warning(logs.CGR_INSTRUCTIONS_READ_FAILED.format(path=path, error=e))
+ return None
+ if not body:
+ return None
+ logger.info(logs.CGR_INSTRUCTIONS_LOADED.format(path=path, chars=len(body)))
+ return body
+
+
+def load_cgr_instructions(repo_path: Path | None) -> str | None:
+ global_body = _read_cgr_instructions_file(GLOBAL_CGR_INSTRUCTIONS_PATH)
+ repo_body = (
+ _read_cgr_instructions_file(repo_path / CGR_INSTRUCTIONS_FILENAME)
+ if repo_path is not None
+ else None
+ )
+ if global_body and repo_body:
+ return f"{global_body}\n\n---\n\n{repo_body}"
+ return global_body or repo_body
diff --git a/codebase_rag/constants.py b/codebase_rag/constants.py
deleted file mode 100644
index 4ef971d8a..000000000
--- a/codebase_rag/constants.py
+++ /dev/null
@@ -1,2815 +0,0 @@
-from enum import StrEnum
-from typing import NamedTuple
-
-
-class PyInstallerPackage(NamedTuple):
- name: str
- collect_all: bool = False
- collect_data: bool = False
- hidden_import: str | None = None
-
-
-class ModelRole(StrEnum):
- ORCHESTRATOR = "orchestrator"
- CYPHER = "cypher"
-
-
-class Provider(StrEnum):
- OLLAMA = "ollama"
- ANTHROPIC = "anthropic"
- OPENAI = "openai"
- GOOGLE = "google"
- AZURE = "azure"
- COHERE = "cohere"
- LOCAL = "local"
- VLLM = "vllm"
-
-
-class Color(StrEnum):
- GREEN = "green"
- YELLOW = "yellow"
- CYAN = "cyan"
- RED = "red"
- MAGENTA = "magenta"
- BLUE = "blue"
-
-
-class KeyBinding(StrEnum):
- CTRL_J = "c-j"
- ENTER = "enter"
- CTRL_C = "c-c"
-
-
-class StyleModifier(StrEnum):
- BOLD = "bold"
- DIM = "dim"
- NONE = ""
-
-
-class FileAction(StrEnum):
- READ = "read"
- EDIT = "edit"
-
-
-DEFAULT_MODEL_ROLE = "model"
-
-BINARY_EXTENSIONS: frozenset[str] = frozenset(
- {
- ".pdf",
- ".png",
- ".jpg",
- ".jpeg",
- ".gif",
- ".bmp",
- ".ico",
- ".tiff",
- ".webp",
- }
-)
-
-# (H) Source file extensions by language
-EXT_PY = ".py"
-EXT_JS = ".js"
-EXT_JSX = ".jsx"
-EXT_TS = ".ts"
-EXT_TSX = ".tsx"
-EXT_RS = ".rs"
-EXT_GO = ".go"
-EXT_SCALA = ".scala"
-EXT_SC = ".sc"
-EXT_JAVA = ".java"
-EXT_CLASS = ".class"
-EXT_CPP = ".cpp"
-EXT_H = ".h"
-EXT_HPP = ".hpp"
-EXT_CC = ".cc"
-EXT_CXX = ".cxx"
-EXT_HXX = ".hxx"
-EXT_HH = ".hh"
-EXT_IXX = ".ixx"
-EXT_CPPM = ".cppm"
-EXT_CCM = ".ccm"
-EXT_CS = ".cs"
-EXT_PHP = ".php"
-EXT_LUA = ".lua"
-
-# (H) File extension tuples by language
-PY_EXTENSIONS = (EXT_PY,)
-JS_EXTENSIONS = (EXT_JS, EXT_JSX)
-TS_EXTENSIONS = (EXT_TS, EXT_TSX)
-RS_EXTENSIONS = (EXT_RS,)
-GO_EXTENSIONS = (EXT_GO,)
-SCALA_EXTENSIONS = (EXT_SCALA, EXT_SC)
-JAVA_EXTENSIONS = (EXT_JAVA,)
-CPP_EXTENSIONS = (
- EXT_CPP,
- EXT_H,
- EXT_HPP,
- EXT_CC,
- EXT_CXX,
- EXT_HXX,
- EXT_HH,
- EXT_IXX,
- EXT_CPPM,
- EXT_CCM,
-)
-CS_EXTENSIONS = (EXT_CS,)
-PHP_EXTENSIONS = (EXT_PHP,)
-LUA_EXTENSIONS = (EXT_LUA,)
-
-# (H) Package indicator files
-PKG_INIT_PY = "__init__.py"
-PKG_CARGO_TOML = "Cargo.toml"
-PKG_CMAKE_LISTS = "CMakeLists.txt"
-PKG_MAKEFILE = "Makefile"
-PKG_VCXPROJ_GLOB = "*.vcxproj"
-PKG_CONANFILE = "conanfile.txt"
-
-DEFAULT_REGION = "us-central1"
-DEFAULT_MODEL = "llama3.2"
-DEFAULT_API_KEY = "ollama"
-
-ENV_OPENAI_API_KEY = "OPENAI_API_KEY"
-ENV_GOOGLE_API_KEY = "GOOGLE_API_KEY"
-
-HELP_ARG = "help"
-
-
-class GoogleProviderType(StrEnum):
- GLA = "gla"
- VERTEX = "vertex"
-
-
-# (H) Provider endpoints
-OPENAI_DEFAULT_ENDPOINT = "https://api.openai.com/v1"
-OLLAMA_HEALTH_PATH = "/api/tags"
-GOOGLE_CLOUD_SCOPE = "https://www.googleapis.com/auth/cloud-platform"
-V1_PATH = "/v1"
-
-# (H) HTTP status codes
-HTTP_OK = 200
-
-UNIXCODER_MODEL = "microsoft/unixcoder-base"
-
-KEY_NODES = "nodes"
-KEY_RELATIONSHIPS = "relationships"
-KEY_NODE_ID = "node_id"
-KEY_LABELS = "labels"
-KEY_PROPERTIES = "properties"
-KEY_FROM_ID = "from_id"
-KEY_TO_ID = "to_id"
-KEY_TYPE = "type"
-KEY_METADATA = "metadata"
-KEY_TOTAL_NODES = "total_nodes"
-KEY_TOTAL_RELATIONSHIPS = "total_relationships"
-KEY_NODE_LABELS = "node_labels"
-KEY_RELATIONSHIP_TYPES = "relationship_types"
-KEY_EXPORTED_AT = "exported_at"
-KEY_PARSER = "parser"
-KEY_NAME = "name"
-KEY_QUALIFIED_NAME = "qualified_name"
-KEY_START_LINE = "start_line"
-KEY_END_LINE = "end_line"
-KEY_PATH = "path"
-KEY_EXTENSION = "extension"
-KEY_MODULE_TYPE = "module_type"
-KEY_IMPLEMENTS_MODULE = "implements_module"
-KEY_PROPS = "props"
-KEY_CREATED = "created"
-KEY_FROM_VAL = "from_val"
-KEY_TO_VAL = "to_val"
-KEY_VERSION_SPEC = "version_spec"
-KEY_PREFIX = "prefix"
-KEY_PROJECT_NAME = "project_name"
-KEY_IS_EXTERNAL = "is_external"
-
-ERR_SUBSTR_ALREADY_EXISTS = "already exists"
-ERR_SUBSTR_CONSTRAINT = "constraint"
-
-# (H) File names
-INIT_PY = "__init__.py"
-
-# (H) Encoding
-ENCODING_UTF8 = "utf-8"
-
-# (H) Protobuf file names
-PROTOBUF_INDEX_FILE = "index.bin"
-PROTOBUF_NODES_FILE = "nodes.bin"
-PROTOBUF_RELS_FILE = "relationships.bin"
-
-# (H) Protobuf oneof field names
-ONEOF_PROJECT = "project"
-ONEOF_PACKAGE = "package"
-ONEOF_FOLDER = "folder"
-ONEOF_MODULE = "module"
-ONEOF_CLASS = "class_node"
-ONEOF_FUNCTION = "function"
-ONEOF_METHOD = "method"
-ONEOF_FILE = "file"
-ONEOF_EXTERNAL_PACKAGE = "external_package"
-ONEOF_MODULE_IMPLEMENTATION = "module_implementation"
-ONEOF_MODULE_INTERFACE = "module_interface"
-
-# (H) CLI error and info messages
-CLI_ERR_OUTPUT_REQUIRES_UPDATE = (
- "Error: --output/-o option requires --update-graph to be specified."
-)
-CLI_ERR_ONLY_JSON = "Error: Currently only JSON format is supported."
-CLI_ERR_STARTUP = "Startup Error: {error}"
-CLI_ERR_CONFIG = "Configuration Error: {error}"
-CLI_ERR_INDEXING = "An error occurred during indexing: {error}"
-CLI_ERR_EXPORT_FAILED = "Failed to export graph: {error}"
-CLI_ERR_LOAD_GRAPH = "Failed to load graph: {error}"
-CLI_ERR_MCP_SERVER = "MCP Server Error: {error}"
-
-CLI_MSG_UPDATING_GRAPH = "Updating knowledge graph for: {path}"
-CLI_MSG_CLEANING_DB = "Cleaning database..."
-CLI_MSG_EXPORTING_TO = "Exporting graph to: {path}"
-CLI_MSG_GRAPH_UPDATED = "Graph update completed!"
-CLI_MSG_APP_TERMINATED = "\nApplication terminated by user."
-CLI_MSG_INDEXING_AT = "Indexing codebase at: {path}"
-CLI_MSG_OUTPUT_TO = "Output will be written to: {path}"
-CLI_MSG_INDEXING_DONE = "Indexing process completed successfully!"
-CLI_MSG_CONNECTING_MEMGRAPH = "Connecting to Memgraph to export graph..."
-CLI_MSG_EXPORTING_DATA = "Exporting graph data..."
-CLI_MSG_OPTIMIZATION_TERMINATED = "\nOptimization session terminated by user."
-CLI_MSG_MCP_TERMINATED = "\nMCP server terminated by user."
-CLI_MSG_HINT_TARGET_REPO = (
- "\nHint: Make sure TARGET_REPO_PATH environment variable is set."
-)
-CLI_MSG_GRAPH_SUMMARY = "Graph Summary:"
-CLI_MSG_AUTO_EXCLUDE = (
- "Auto-excluding common directories (venv, node_modules, .git, etc.). "
- "Use --interactive-setup to customize."
-)
-
-UI_DIFF_FILE_HEADER = "[bold cyan]File: {path}[/bold cyan]"
-UI_NEW_FILE_HEADER = "[bold cyan]New file: {path}[/bold cyan]"
-UI_SHELL_COMMAND_HEADER = "[bold cyan]Shell command:[/bold cyan]"
-UI_TOOL_APPROVAL = "[bold yellow]⚠️ Tool '{tool_name}' requires approval:[/bold yellow]"
-UI_FEEDBACK_PROMPT = (
- "[bold yellow]Feedback (why rejected, or press Enter to skip)[/bold yellow]"
-)
-UI_OPTIMIZATION_START = (
- "[bold green]Starting {language} optimization session...[/bold green]"
-)
-UI_OPTIMIZATION_PANEL = (
- "[bold yellow]The agent will analyze your codebase{document_info} and propose specific optimizations."
- " You'll be asked to approve each suggestion before implementation."
- " Type 'exit' or 'quit' to end the session.[/bold yellow]"
-)
-UI_OPTIMIZATION_INIT = "[bold cyan]Initializing optimization session for {language} codebase: {path}[/bold cyan]"
-UI_GRAPH_EXPORT_SUCCESS = (
- "[bold green]Graph exported successfully to: {path}[/bold green]"
-)
-UI_GRAPH_EXPORT_STATS = "[bold cyan]Export contains {nodes} nodes and {relationships} relationships[/bold cyan]"
-UI_ERR_UNEXPECTED = "[bold red]An unexpected error occurred: {error}[/bold red]"
-UI_ERR_EXPORT_FAILED = "[bold red]Failed to export graph: {error}[/bold red]"
-UI_MODEL_SWITCHED = "[bold green]Model switched to: {model}[/bold green]"
-UI_MODEL_CURRENT = "[bold cyan]Current model: {model}[/bold cyan]"
-UI_MODEL_SWITCH_ERROR = "[bold red]Failed to switch model: {error}[/bold red]"
-UI_MODEL_USAGE = "[bold yellow]Usage: /model (e.g., /model google:gemini-2.0-flash)[/bold yellow]"
-UI_HELP_COMMANDS = """[bold cyan]Available commands:[/bold cyan]
- /model - Switch to a different model
- /model - Show current model
- /help - Show this help
- exit, quit - Exit the session"""
-UI_TOOL_ARGS_FORMAT = " Arguments: {args}"
-UI_REFERENCE_DOC_INFO = " using the reference document: {reference_document}"
-UI_INPUT_PROMPT_HTML = (
- "{prompt}{hint}: "
-)
-
-# (H) ModelConfig field names
-FIELD_PROVIDER = "provider"
-FIELD_MODEL_ID = "model_id"
-FIELD_API_KEY = "api_key"
-FIELD_ENDPOINT = "endpoint"
-
-# (H) Tool argument field names
-ARG_TARGET_CODE = "target_code"
-ARG_REPLACEMENT_CODE = "replacement_code"
-ARG_FILE_PATH = "file_path"
-ARG_CONTENT = "content"
-ARG_COMMAND = "command"
-
-# (H) Qualified name separators
-SEPARATOR_DOT = "."
-SEPARATOR_SLASH = "/"
-
-# (H) Path navigation
-PATH_CURRENT_DIR = "."
-PATH_PARENT_DIR = ".."
-GLOB_ALL = "*"
-PATH_RELATIVE_PREFIX = "./"
-PATH_PARENT_PREFIX = "../"
-CPP_IMPORT_PARTITION_PREFIX = "import :"
-CPP_PARTITION_PREFIX = "partition_"
-
-# (H) Trie internal keys
-TRIE_TYPE_KEY = "__type__"
-TRIE_QN_KEY = "__qn__"
-TRIE_INTERNAL_PREFIX = "__"
-
-
-class UniqueKeyType(StrEnum):
- NAME = KEY_NAME
- PATH = KEY_PATH
- QUALIFIED_NAME = KEY_QUALIFIED_NAME
-
-
-class NodeLabel(StrEnum):
- PROJECT = "Project"
- PACKAGE = "Package"
- FOLDER = "Folder"
- FILE = "File"
- MODULE = "Module"
- CLASS = "Class"
- FUNCTION = "Function"
- METHOD = "Method"
- INTERFACE = "Interface"
- ENUM = "Enum"
- TYPE = "Type"
- UNION = "Union"
- MODULE_INTERFACE = "ModuleInterface"
- MODULE_IMPLEMENTATION = "ModuleImplementation"
- EXTERNAL_PACKAGE = "ExternalPackage"
-
-
-_NODE_LABEL_UNIQUE_KEYS: dict[NodeLabel, UniqueKeyType] = {
- NodeLabel.PROJECT: UniqueKeyType.NAME,
- NodeLabel.PACKAGE: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.FOLDER: UniqueKeyType.PATH,
- NodeLabel.FILE: UniqueKeyType.PATH,
- NodeLabel.MODULE: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.CLASS: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.FUNCTION: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.METHOD: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.INTERFACE: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.ENUM: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.TYPE: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.UNION: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.MODULE_INTERFACE: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.MODULE_IMPLEMENTATION: UniqueKeyType.QUALIFIED_NAME,
- NodeLabel.EXTERNAL_PACKAGE: UniqueKeyType.NAME,
-}
-
-_missing_keys = set(NodeLabel) - set(_NODE_LABEL_UNIQUE_KEYS.keys())
-if _missing_keys:
- raise RuntimeError(
- f"NodeLabel(s) missing from _NODE_LABEL_UNIQUE_KEYS: {_missing_keys}. "
- "Every NodeLabel MUST have a unique key defined."
- )
-
-
-class RelationshipType(StrEnum):
- CONTAINS_PACKAGE = "CONTAINS_PACKAGE"
- CONTAINS_FOLDER = "CONTAINS_FOLDER"
- CONTAINS_FILE = "CONTAINS_FILE"
- CONTAINS_MODULE = "CONTAINS_MODULE"
- DEFINES = "DEFINES"
- DEFINES_METHOD = "DEFINES_METHOD"
- IMPORTS = "IMPORTS"
- EXPORTS = "EXPORTS"
- EXPORTS_MODULE = "EXPORTS_MODULE"
- IMPLEMENTS_MODULE = "IMPLEMENTS_MODULE"
- INHERITS = "INHERITS"
- IMPLEMENTS = "IMPLEMENTS"
- OVERRIDES = "OVERRIDES"
- CALLS = "CALLS"
- DEPENDS_ON_EXTERNAL = "DEPENDS_ON_EXTERNAL"
-
-
-NODE_PROJECT = NodeLabel.PROJECT
-
-EXCLUDED_DEPENDENCY_NAMES = frozenset({"python", "php"})
-
-# (H) Byte size constants
-BYTES_PER_MB = 1024 * 1024
-
-# (H) Property keys
-KEY_PARAMETERS = "parameters"
-KEY_DECORATORS = "decorators"
-KEY_DOCSTRING = "docstring"
-KEY_IS_EXPORTED = "is_exported"
-
-# (H) Method signature formatting
-EMPTY_PARENS = "()"
-DOCSTRING_STRIP_CHARS = "'\" \n"
-
-# (H) Inline module path prefix
-INLINE_MODULE_PATH_PREFIX = "inline_module_"
-
-# (H) Dependency files
-DEPENDENCY_FILES = frozenset(
- {
- "pyproject.toml",
- "requirements.txt",
- "package.json",
- "cargo.toml",
- "go.mod",
- "gemfile",
- "composer.json",
- }
-)
-CSPROJ_SUFFIX = ".csproj"
-
-# (H) Cypher queries
-CYPHER_DEFAULT_LIMIT = 50
-
-CYPHER_QUERY_EMBEDDINGS = """
-MATCH (m:Module)-[:DEFINES]->(n)
-WHERE (n:Function OR n:Method)
- AND m.qualified_name STARTS WITH $project_name + '.'
-RETURN id(n) AS node_id, n.qualified_name AS qualified_name,
- n.start_line AS start_line, n.end_line AS end_line,
- m.path AS path
-"""
-
-
-class SupportedLanguage(StrEnum):
- PYTHON = "python"
- JS = "javascript"
- TS = "typescript"
- RUST = "rust"
- GO = "go"
- SCALA = "scala"
- JAVA = "java"
- CPP = "cpp"
- CSHARP = "c-sharp"
- PHP = "php"
- LUA = "lua"
-
-
-class LanguageStatus(StrEnum):
- FULL = "Fully Supported"
- DEV = "In Development"
-
-
-class LanguageMetadata(NamedTuple):
- status: LanguageStatus
- additional_features: str
- display_name: str
-
-
-LANGUAGE_METADATA: dict[SupportedLanguage, LanguageMetadata] = {
- SupportedLanguage.PYTHON: LanguageMetadata(
- LanguageStatus.FULL,
- "Type inference, decorators, nested functions",
- "Python",
- ),
- SupportedLanguage.JS: LanguageMetadata(
- LanguageStatus.FULL,
- "ES6 modules, CommonJS, prototype methods, object methods, arrow functions",
- "JavaScript",
- ),
- SupportedLanguage.TS: LanguageMetadata(
- LanguageStatus.FULL,
- "Interfaces, type aliases, enums, namespaces, ES6/CommonJS modules",
- "TypeScript",
- ),
- SupportedLanguage.CPP: LanguageMetadata(
- LanguageStatus.FULL,
- "Constructors, destructors, operator overloading, templates, lambdas, C++20 modules, namespaces",
- "C++",
- ),
- SupportedLanguage.LUA: LanguageMetadata(
- LanguageStatus.FULL,
- "Local/global functions, metatables, closures, coroutines",
- "Lua",
- ),
- SupportedLanguage.RUST: LanguageMetadata(
- LanguageStatus.FULL,
- "impl blocks, associated functions",
- "Rust",
- ),
- SupportedLanguage.JAVA: LanguageMetadata(
- LanguageStatus.FULL,
- "Generics, annotations, modern features (records/sealed classes), concurrency, reflection",
- "Java",
- ),
- SupportedLanguage.GO: LanguageMetadata(
- LanguageStatus.DEV,
- "Methods, type declarations",
- "Go",
- ),
- SupportedLanguage.SCALA: LanguageMetadata(
- LanguageStatus.DEV,
- "Case classes, objects",
- "Scala",
- ),
- SupportedLanguage.CSHARP: LanguageMetadata(
- LanguageStatus.DEV,
- "Classes, interfaces, generics (planned)",
- "C#",
- ),
- SupportedLanguage.PHP: LanguageMetadata(
- LanguageStatus.DEV,
- "Classes, functions, namespaces",
- "PHP",
- ),
-}
-
-
-# (H) Tree-sitter AST node type constants
-FUNCTION_NODES_BASIC = ("function_declaration", "function_definition")
-FUNCTION_NODES_LAMBDA = (
- "lambda_expression",
- "arrow_function",
- "anonymous_function",
- "closure_expression",
-)
-FUNCTION_NODES_METHOD = (
- "method_declaration",
- "constructor_declaration",
- "destructor_declaration",
-)
-FUNCTION_NODES_TEMPLATE = (
- "template_declaration",
- "function_signature_item",
- "function_signature",
-)
-FUNCTION_NODES_GENERATOR = ("generator_function_declaration", "function_expression")
-
-CLASS_NODES_BASIC = ("class_declaration", "class_definition")
-CLASS_NODES_STRUCT = ("struct_declaration", "struct_specifier", "struct_item")
-CLASS_NODES_INTERFACE = ("interface_declaration", "trait_declaration", "trait_item")
-CLASS_NODES_ENUM = ("enum_declaration", "enum_item", "enum_specifier")
-CLASS_NODES_TYPE_ALIAS = ("type_alias_declaration", "type_item")
-CLASS_NODES_UNION = ("union_specifier", "union_item")
-
-CALL_NODES_BASIC = ("call_expression", "function_call")
-CALL_NODES_METHOD = (
- "method_invocation",
- "member_call_expression",
- "field_expression",
-)
-CALL_NODES_OPERATOR = ("binary_expression", "unary_expression", "update_expression")
-CALL_NODES_SPECIAL = ("new_expression", "delete_expression", "macro_invocation")
-
-IMPORT_NODES_STANDARD = ("import_declaration", "import_statement")
-IMPORT_NODES_FROM = ("import_from_statement",)
-IMPORT_NODES_MODULE = ("lexical_declaration", "export_statement")
-IMPORT_NODES_INCLUDE = ("preproc_include",)
-IMPORT_NODES_USING = ("using_directive",)
-
-# (H) JS/TS specific node types
-JS_TS_FUNCTION_NODES = (
- "function_declaration",
- "generator_function_declaration",
- "function_expression",
- "arrow_function",
- "method_definition",
-)
-JS_TS_CLASS_NODES = ("class_declaration", "class")
-JS_TS_IMPORT_NODES = ("import_statement", "lexical_declaration", "export_statement")
-JS_TS_LANGUAGES = frozenset({SupportedLanguage.JS, SupportedLanguage.TS})
-
-# (H) C++ import node types
-CPP_IMPORT_NODES = ("preproc_include", "template_function", "declaration")
-
-# (H) Index file names
-INDEX_INIT = "__init__"
-INDEX_INDEX = "index"
-INDEX_MOD = "mod"
-
-# (H) AST field names for name extraction
-NAME_FIELDS = ("identifier", "name", "id")
-
-# (H) Tree-sitter field name constants for child_by_field_name
-FIELD_OBJECT = "object"
-FIELD_PROPERTY = "property"
-FIELD_NAME = "name"
-FIELD_ALIAS = "alias"
-FIELD_MODULE_NAME = "module_name"
-FIELD_ARGUMENTS = "arguments"
-FIELD_BODY = "body"
-FIELD_CONSTRUCTOR = "constructor"
-FIELD_DECLARATOR = "declarator"
-FIELD_PARAMETERS = "parameters"
-FIELD_TYPE = "type"
-FIELD_VALUE = "value"
-FIELD_LEFT = "left"
-FIELD_RIGHT = "right"
-FIELD_FIELD = "field"
-FIELD_SUPERCLASS = "superclass"
-FIELD_SUPERCLASSES = "superclasses"
-FIELD_INTERFACES = "interfaces"
-
-# (H) Method name constants for getattr/hasattr
-METHOD_FIND_WITH_PREFIX = "find_with_prefix"
-METHOD_ITEMS = "items"
-
-# (H) Image file extensions for chat image handling
-IMAGE_EXTENSIONS = (".png", ".jpg", ".jpeg", ".gif")
-
-# (H) CLI exit commands
-EXIT_COMMANDS = frozenset({"exit", "quit"})
-
-# (H) CLI commands
-MODEL_COMMAND_PREFIX = "/model"
-HELP_COMMAND = "/help"
-
-# (H) UI separators and formatting
-HORIZONTAL_SEPARATOR = "─" * 60
-
-# (H) Session log header
-SESSION_LOG_HEADER = "=== CODE-GRAPH RAG SESSION LOG ===\n\n"
-
-# (H) Logger format
-LOG_FORMAT = "{time:YYYY-MM-DD HH:mm:ss.SSS} | {message}"
-
-# (H) Temporary directory
-TMP_DIR = ".tmp"
-SESSION_LOG_PREFIX = "session_"
-SESSION_LOG_EXT = ".log"
-
-# (H) Session log prefixes
-SESSION_PREFIX_USER = "USER: "
-SESSION_PREFIX_ASSISTANT = "ASSISTANT: "
-
-# (H) Session context format
-SESSION_CONTEXT_START = (
- "\n\n[SESSION CONTEXT - Previous conversation in this session]:\n"
-)
-SESSION_CONTEXT_END = "\n[END SESSION CONTEXT]\n\n"
-
-# (H) Confirmation status display
-CONFIRM_ENABLED = "Enabled"
-CONFIRM_DISABLED = "Disabled (YOLO Mode)"
-
-# (H) Diff labels
-DIFF_LABEL_BEFORE = "before"
-DIFF_LABEL_AFTER = "after"
-DIFF_FALLBACK_PATH = "file"
-
-
-class DiffMarker:
- ADD = "+"
- DEL = "-"
- HUNK = "@"
- HEADER_ADD = "+++"
- HEADER_DEL = "---"
-
-
-# (H) Table column headers
-TABLE_COL_CONFIGURATION = "Configuration"
-TABLE_COL_VALUE = "Value"
-
-# (H) Table row labels
-TABLE_ROW_TARGET_LANGUAGE = "Target Language"
-TABLE_ROW_ORCHESTRATOR_MODEL = "Orchestrator Model"
-TABLE_ROW_CYPHER_MODEL = "Cypher Model"
-TABLE_ROW_OLLAMA_ENDPOINT = "Ollama Endpoint"
-TABLE_ROW_OLLAMA_ORCHESTRATOR = "Ollama Endpoint (Orchestrator)"
-TABLE_ROW_OLLAMA_CYPHER = "Ollama Endpoint (Cypher)"
-TABLE_ROW_EDIT_CONFIRMATION = "Edit Confirmation"
-TABLE_ROW_TARGET_REPOSITORY = "Target Repository"
-
-# (H) UI status messages
-MSG_CONNECTED_MEMGRAPH = "Successfully connected to Memgraph."
-MSG_THINKING_CANCELLED = "Thinking cancelled."
-MSG_TIMEOUT_FORMAT = "Operation timed out after {timeout} seconds."
-MSG_CHAT_INSTRUCTIONS = (
- "Ask questions about your codebase graph. Type 'exit' or 'quit' to end."
-)
-
-# (H) Default titles and prompts
-DEFAULT_TABLE_TITLE = "Code-Graph-RAG Initializing..."
-OPTIMIZATION_TABLE_TITLE = "Optimization Session Configuration"
-PROMPT_ASK_QUESTION = "Ask a question"
-PROMPT_YOUR_RESPONSE = "Your response"
-MULTILINE_INPUT_HINT = "(Press Ctrl+J to submit, Enter for new line)"
-
-# (H) Interactive setup prompt - grouped view
-INTERACTIVE_TITLE_GROUPED = "Detected Directories (will be excluded unless kept)"
-INTERACTIVE_TITLE_NESTED = "Nested paths in '{pattern}'"
-INTERACTIVE_COL_NUM = "#"
-INTERACTIVE_COL_PATTERN = "Pattern"
-INTERACTIVE_COL_NESTED = "Nested"
-INTERACTIVE_COL_PATH = "Path"
-INTERACTIVE_STYLE_DIM = "dim"
-INTERACTIVE_STATUS_DETECTED = "auto-detected"
-INTERACTIVE_STATUS_CLI = "--exclude"
-INTERACTIVE_STATUS_CGRIGNORE = ".cgrignore"
-INTERACTIVE_NESTED_SINGULAR = "{count} dir"
-INTERACTIVE_NESTED_PLURAL = "{count} dirs"
-INTERACTIVE_INSTRUCTIONS_GROUPED = (
- "These directories would normally be excluded. "
- "Options: 'all' (keep all), 'none' (keep none), "
- "numbers like '1,3' (keep groups), or '1e' to expand group 1"
-)
-INTERACTIVE_INSTRUCTIONS_NESTED = (
- "Select paths to keep from '{pattern}'. "
- "Options: 'all', 'none', or numbers like '1,3'"
-)
-INTERACTIVE_PROMPT_KEEP = "Keep"
-INTERACTIVE_KEEP_ALL = "all"
-INTERACTIVE_KEEP_NONE = "none"
-INTERACTIVE_EXPAND_SUFFIX = "e"
-INTERACTIVE_BFS_MAX_DEPTH = 10
-INTERACTIVE_DEFAULT_GROUP = "."
-
-# (H) JSON formatting
-JSON_INDENT = 2
-
-# (H) Parser loader paths and args
-GRAMMARS_DIR = "grammars"
-TREE_SITTER_PREFIX = "tree-sitter-"
-TREE_SITTER_MODULE_PREFIX = "tree_sitter_"
-BINDINGS_DIR = "bindings"
-SETUP_PY = "setup.py"
-BUILD_EXT_CMD = "build_ext"
-INPLACE_FLAG = "--inplace"
-LANG_ATTR_PREFIX = "language_"
-LANG_ATTR_TYPESCRIPT = "language_typescript"
-
-
-class TreeSitterModule(StrEnum):
- PYTHON = "tree_sitter_python"
- JS = "tree_sitter_javascript"
- TS = "tree_sitter_typescript"
- RUST = "tree_sitter_rust"
- GO = "tree_sitter_go"
- SCALA = "tree_sitter_scala"
- JAVA = "tree_sitter_java"
- CPP = "tree_sitter_cpp"
- LUA = "tree_sitter_lua"
-
-
-# (H) Query dict keys
-QUERY_FUNCTIONS = "functions"
-QUERY_CLASSES = "classes"
-QUERY_CALLS = "calls"
-QUERY_IMPORTS = "imports"
-QUERY_LOCALS = "locals"
-QUERY_CONFIG = "config"
-QUERY_LANGUAGE = "language"
-
-# (H) Query capture names
-CAPTURE_FUNCTION = "function"
-CAPTURE_CLASS = "class"
-CAPTURE_CALL = "call"
-CAPTURE_IMPORT = "import"
-CAPTURE_IMPORT_FROM = "import_from"
-
-# (H) Locals query patterns for JS/TS
-JS_LOCALS_PATTERN = """
-; Variable definitions
-(variable_declarator name: (identifier) @local.definition)
-(function_declaration name: (identifier) @local.definition)
-(class_declaration name: (identifier) @local.definition)
-
-; Variable references
-(identifier) @local.reference
-"""
-
-TS_LOCALS_PATTERN = """
-; Variable definitions (TypeScript has multiple declaration types)
-(variable_declarator name: (identifier) @local.definition)
-(lexical_declaration (variable_declarator name: (identifier) @local.definition))
-(variable_declaration (variable_declarator name: (identifier) @local.definition))
-
-; Function definitions
-(function_declaration name: (identifier) @local.definition)
-
-; Class definitions (uses type_identifier for class names)
-(class_declaration name: (type_identifier) @local.definition)
-
-; Variable references
-(identifier) @local.reference
-"""
-
-# (H) Patterns to detect at repo root and offer as exclude candidates (user selects which to exclude)
-IGNORE_PATTERNS = frozenset(
- {
- ".cache",
- ".claude",
- ".eclipse",
- ".eggs",
- ".env",
- ".git",
- ".gradle",
- ".hg",
- ".idea",
- ".maven",
- ".mypy_cache",
- ".nox",
- ".npm",
- ".nyc_output",
- ".pnpm-store",
- ".pytest_cache",
- ".qdrant_code_embeddings",
- ".ruff_cache",
- ".svn",
- ".tmp",
- ".tox",
- ".venv",
- ".vs",
- ".vscode",
- ".yarn",
- "__pycache__",
- "bin",
- "bower_components",
- "build",
- "coverage",
- "dist",
- "env",
- "htmlcov",
- "node_modules",
- "obj",
- "out",
- "Pods",
- "site-packages",
- "target",
- "temp",
- "tmp",
- "vendor",
- "venv",
- }
-)
-IGNORE_SUFFIXES = frozenset(
- {".tmp", "~", ".pyc", ".pyo", ".o", ".a", ".so", ".dll", ".class"}
-)
-
-PAYLOAD_NODE_ID = "node_id"
-PAYLOAD_QUALIFIED_NAME = "qualified_name"
-
-
-class EventType(StrEnum):
- MODIFIED = "modified"
- CREATED = "created"
-
-
-CYPHER_DELETE_MODULE = "MATCH (m:Module {path: $path})-[*0..]->(c) DETACH DELETE m, c"
-CYPHER_DELETE_CALLS = "MATCH ()-[r:CALLS]->() DELETE r"
-
-REALTIME_LOGGER_FORMAT = (
- "{time:YYYY-MM-DD HH:mm:ss.SSS} | "
- "{level: <8} | "
- "{name}:{function}:{line} - "
- "{message}"
-)
-
-WATCHER_SLEEP_INTERVAL = 1
-LOG_LEVEL_INFO = "INFO"
-
-
-class Architecture(StrEnum):
- X86_64 = "x86_64"
- AARCH64 = "aarch64"
- ARM64 = "arm64"
- AMD64 = "amd64"
-
-
-BINARY_NAME_TEMPLATE = "code-graph-rag-{system}-{machine}"
-BINARY_FILE_PERMISSION = 0o755
-DIST_DIR = "dist"
-BYTES_PER_MB_FLOAT = 1024 * 1024
-
-PYPROJECT_PATH = "pyproject.toml"
-TREESITTER_EXTRA_KEY = "treesitter-full"
-TREESITTER_PKG_PREFIX = "tree-sitter-"
-
-# (H) PyInstaller CLI constants
-PYINSTALLER_CMD = "pyinstaller"
-PYINSTALLER_ARG_NAME = "--name"
-PYINSTALLER_ARG_ONEFILE = "--onefile"
-PYINSTALLER_ARG_NOCONFIRM = "--noconfirm"
-PYINSTALLER_ARG_CLEAN = "--clean"
-PYINSTALLER_ARG_COLLECT_ALL = "--collect-all"
-PYINSTALLER_ARG_COLLECT_DATA = "--collect-data"
-PYINSTALLER_ARG_HIDDEN_IMPORT = "--hidden-import"
-PYINSTALLER_ENTRY_POINT = "main.py"
-
-# (H) TOML parsing constants
-TOML_KEY_PROJECT = "project"
-TOML_KEY_OPTIONAL_DEPS = "optional-dependencies"
-
-# (H) Version string parsing
-VERSION_SPLIT_GTE = ">="
-VERSION_SPLIT_EQ = "=="
-VERSION_SPLIT_LT = "<"
-CHAR_HYPHEN = "-"
-CHAR_UNDERSCORE = "_"
-
-PYINSTALLER_PACKAGES: list["PyInstallerPackage"] = [
- PyInstallerPackage(
- name="pydantic_ai",
- collect_all=True,
- collect_data=True,
- hidden_import="pydantic_ai_slim",
- ),
- PyInstallerPackage(name="rich", collect_all=True),
- PyInstallerPackage(name="typer", collect_all=True),
- PyInstallerPackage(name="loguru", collect_all=True),
- PyInstallerPackage(name="toml", collect_all=True),
- PyInstallerPackage(name="protobuf", collect_all=True),
-]
-
-ALLOWED_COMMENT_MARKERS = frozenset(
- {"(H)", "type:", "noqa", "pyright", "ty:", "@@protoc", "nosec"}
-)
-QUOTE_CHARS = frozenset({'"', "'"})
-TRIPLE_QUOTES = ('"""', "'''")
-COMMENT_CHAR = "#"
-ESCAPE_CHAR = "\\"
-CHAR_SEMICOLON = ";"
-CHAR_COMMA = ","
-CHAR_COLON = ":"
-CHAR_ANGLE_OPEN = "<"
-CHAR_ANGLE_CLOSE = ">"
-CHAR_PAREN_OPEN = "("
-CHAR_PAREN_CLOSE = ")"
-CHAR_UNDERSCORE = "_"
-CHAR_SPACE = " "
-SEPARATOR_COMMA_SPACE = ", "
-PUNCTUATION_TYPES = (CHAR_PAREN_OPEN, CHAR_PAREN_CLOSE, CHAR_COMMA)
-
-REGEX_METHOD_CHAIN_SUFFIX = r"\)\.[^)]*$"
-REGEX_FINAL_METHOD_CAPTURE = r"\.([^.()]+)$"
-
-DEFAULT_NAME = "Unknown"
-TEXT_UNKNOWN = "unknown"
-
-MODULE_TORCH = "torch"
-MODULE_TRANSFORMERS = "transformers"
-MODULE_QDRANT_CLIENT = "qdrant_client"
-
-SEMANTIC_DEPENDENCIES = (MODULE_QDRANT_CLIENT, MODULE_TORCH, MODULE_TRANSFORMERS)
-ML_DEPENDENCIES = (MODULE_TORCH, MODULE_TRANSFORMERS)
-
-
-class UniXcoderMode(StrEnum):
- ENCODER_ONLY = ""
- DECODER_ONLY = ""
- ENCODER_DECODER = ""
-
-
-UNIXCODER_MASK_TOKEN = ""
-UNIXCODER_BUFFER_BIAS = "bias"
-UNIXCODER_MAX_CONTEXT = 1024
-
-REL_TYPE_CALLS = "CALLS"
-
-NODE_UNIQUE_CONSTRAINTS: dict[str, str] = {
- label.value: key.value for label, key in _NODE_LABEL_UNIQUE_KEYS.items()
-}
-
-# (H) Cypher response cleaning
-CYPHER_PREFIX = "cypher"
-CYPHER_SEMICOLON = ";"
-CYPHER_BACKTICK = "`"
-CYPHER_MATCH_KEYWORD = "MATCH"
-
-# (H) Tool success messages
-MSG_SURGICAL_SUCCESS = "Successfully applied surgical code replacement in: {path}"
-MSG_SURGICAL_FAILED = (
- "Failed to apply surgical replacement in {path}. "
- "Target code not found or patches failed."
-)
-
-# (H) Grep suggestion
-GREP_SUGGESTION = " Use 'rg' instead of 'grep' for text searching."
-
-# (H) Shell command constants
-SHELL_CMD_GREP = "grep"
-SHELL_CMD_GIT = "git"
-SHELL_CMD_RM = "rm"
-SHELL_RM_RF_FLAG = "-rf"
-SHELL_RETURN_CODE_ERROR = -1
-SHELL_PIPE_OPERATORS = ("|", "&&", "||", ";")
-SHELL_SUBSHELL_PATTERNS = ("$(", "`")
-SHELL_REDIRECT_OPERATORS = frozenset({">", ">>", "<", "<<"})
-
-# (H) Dangerous commands - absolutely blocked
-SHELL_DANGEROUS_COMMANDS = frozenset(
- {
- "dd",
- "mkfs",
- "mkfs.ext4",
- "mkfs.ext3",
- "mkfs.xfs",
- "mkfs.btrfs",
- "mkfs.vfat",
- "fdisk",
- "parted",
- "shred",
- "wipefs",
- "mkswap",
- "swapon",
- "swapoff",
- "mount",
- "umount",
- "insmod",
- "rmmod",
- "modprobe",
- "shutdown",
- "reboot",
- "halt",
- "poweroff",
- "init",
- "telinit",
- "systemctl",
- "service",
- "chroot",
- "nohup",
- "disown",
- "crontab",
- "at",
- "batch",
- }
-)
-
-# (H) Dangerous rm flags
-SHELL_RM_DANGEROUS_FLAGS = frozenset({"-rf", "-fr"})
-SHELL_RM_FORCE_FLAG = "-f"
-
-# (H) System directories to protect from rm -rf
-SHELL_SYSTEM_DIRECTORIES = frozenset(
- {
- "bin",
- "boot",
- "dev",
- "etc",
- "home",
- "lib",
- "lib64",
- "media",
- "mnt",
- "opt",
- "proc",
- "root",
- "run",
- "sbin",
- "srv",
- "sys",
- "tmp",
- "usr",
- "var",
- }
-)
-
-# (H) Dangerous patterns for full pipeline (cross-segment patterns with pipes/operators)
-SHELL_DANGEROUS_PATTERNS_PIPELINE = (
- (r"(wget|curl)\s+.*\|\s*(sh|bash|zsh|ksh)", "remote script execution"),
- (r"(wget|curl)\s+.*>\s*.*\.sh\s*&&", "download and execute script"),
- (r"base64\s+-d.*\|", "base64 decode pipe execution"),
-)
-
-# (H) Build system directory regex pattern dynamically
-_SYSTEM_DIRS_PATTERN = "|".join(SHELL_SYSTEM_DIRECTORIES)
-
-# (H) Dangerous patterns for individual segments (per-command patterns)
-SHELL_DANGEROUS_PATTERNS_SEGMENT = (
- (r"rm\s+.*-[rf]+\s+/($|\s)", "rm with root path"),
- (rf"rm\s+.*-[rf]+\s+/({_SYSTEM_DIRS_PATTERN})($|/|\s)", "rm with system directory"),
- (r"rm\s+.*-[rf]+\s+~($|\s)", "rm with home directory"),
- (r"rm\s+.*-[rf]+\s+\*", "rm with wildcard"),
- (r"rm\s+.*-[rf]+\s+\.\.", "rm with parent directory"),
- (r"dd\s+.*of=/dev/", "dd writing to device"),
- (r">\s*/dev/sd[a-z]", "redirect to disk device"),
- (r">\s*/dev/nvme", "redirect to nvme device"),
- (r">\s*/dev/null.*<", "null device manipulation"),
- (r"chmod\s+.*-R\s+777\s+/", "recursive 777 on root"),
- (r"chmod\s+.*777\s+/($|\s)", "777 on root"),
- (r"chown\s+.*-R\s+.*\s+/($|\s)", "recursive chown on root"),
- (r":\(\)\s*\{.*:\s*\|", "fork bomb pattern"),
- (r"mv\s+.*\s+/dev/null", "move to /dev/null"),
- (r"ln\s+-[sf]+\s+/dev/null", "symlink to /dev/null"),
- (r"cat\s+.*/dev/zero", "cat /dev/zero"),
- (r"cat\s+.*/dev/random", "cat /dev/random"),
- (r">\s*/etc/passwd", "overwrite passwd"),
- (r">\s*/etc/shadow", "overwrite shadow"),
- (r">\s*/etc/sudoers", "overwrite sudoers"),
- (r"echo\s+.*>\s*/etc/", "write to /etc"),
- (
- r"python.*-c.*(import\s+os|__import__\s*\(\s*['\"]os['\"]\s*\))",
- "python os import in command",
- ),
- (r"perl\s+-e", "perl one-liner"),
- (r"ruby\s+-e", "ruby one-liner"),
- (r"nc\s+-[el]", "netcat listener"),
- (r"ncat\s+-[el]", "ncat listener"),
- (r"/dev/tcp/", "bash tcp device"),
- (r"eval\s+", "eval command"),
- (r"exec\s+[0-9]+<>", "exec file descriptor manipulation"),
- (r"awk\s+.*system\s*\(", "awk system() call"),
- (r"awk\s+.*getline\s*[<|]", "awk getline file/pipe execution"),
- (r"sed\s+.*s(.).*?\1.*?\1[gip]*e[gip]*", "sed execute flag"),
- (r"xargs\s+.*(rm|chmod|chown|mv|dd|mkfs)", "xargs with destructive command"),
- (r"xargs\s+-I.*sh", "xargs shell execution"),
- (r"xargs\s+.*bash", "xargs bash execution"),
-)
-
-# (H) Query tool messages
-QUERY_NOT_AVAILABLE = "N/A"
-DICT_KEY_RESULTS = "results"
-QUERY_SUMMARY_SUCCESS = "Successfully retrieved {count} item(s) from the graph."
-QUERY_SUMMARY_TRANSLATION_FAILED = (
- "I couldn't translate your request into a database query. Error: {error}"
-)
-QUERY_SUMMARY_DB_ERROR = "There was an error querying the database: {error}"
-QUERY_RESULTS_PANEL_TITLE = "[bold blue]Cypher Query Results[/bold blue]"
-
-# (H) File editor constants
-TMP_EXTENSION = ".tmp"
-
-# (H) Semantic search constants
-MSG_SEMANTIC_NO_RESULTS = (
- "No semantic matches found for query: '{query}'. This could mean:\n"
- "1. No functions match this description\n"
- "2. Semantic search dependencies are not installed\n"
- "3. No embeddings have been generated yet"
-)
-MSG_SEMANTIC_SOURCE_UNAVAILABLE = (
- "Could not retrieve source code for node ID {id}. "
- "The node may not exist or source file may be unavailable."
-)
-MSG_SEMANTIC_SOURCE_FORMAT = "Source code for node ID {id}:\n\n```\n{code}\n```"
-MSG_SEMANTIC_RESULT_HEADER = "Found {count} semantic matches for '{query}':\n\n"
-MSG_SEMANTIC_RESULT_FOOTER = "\n\nUse the qualified names above with other tools to get more details or source code."
-SEMANTIC_BATCH_SIZE = 100
-SEMANTIC_TYPE_UNKNOWN = "Unknown"
-
-# (H) Document analyzer constants
-MSG_DOC_NO_CANDIDATES = "No valid text found in response candidates."
-MSG_DOC_NO_CONTENT = "No text content received from the API."
-MIME_TYPE_DEFAULT = "application/octet-stream"
-DOC_PROMPT_PREFIX = (
- "Based on the document provided, please answer the following question: {question}"
-)
-
-# (H) Call processor constants
-MOD_RS = "mod.rs"
-SEPARATOR_DOUBLE_COLON = "::"
-SEPARATOR_COLON = ":"
-SEPARATOR_PROTOTYPE = ".prototype."
-RUST_CRATE_PREFIX = "crate::"
-BUILTIN_PREFIX = "builtin"
-IIFE_FUNC_PREFIX = "iife_func_"
-IIFE_ARROW_PREFIX = "iife_arrow_"
-OPERATOR_PREFIX = "operator"
-KEYWORD_SUPER = "super"
-KEYWORD_SELF = "self"
-KEYWORD_CONSTRUCTOR = "constructor"
-
-# (H) JavaScript built-in types
-JS_BUILTIN_TYPES: frozenset[str] = frozenset(
- {
- "Array",
- "Object",
- "String",
- "Number",
- "Date",
- "RegExp",
- "Function",
- "Map",
- "Set",
- "Promise",
- "Error",
- "Boolean",
- }
-)
-
-# (H) JavaScript built-in function patterns
-JS_BUILTIN_PATTERNS: frozenset[str] = frozenset(
- {
- "Object.create",
- "Object.keys",
- "Object.values",
- "Object.entries",
- "Object.assign",
- "Object.freeze",
- "Object.seal",
- "Object.defineProperty",
- "Object.getPrototypeOf",
- "Object.setPrototypeOf",
- "Array.from",
- "Array.of",
- "Array.isArray",
- "parseInt",
- "parseFloat",
- "isNaN",
- "isFinite",
- "encodeURIComponent",
- "decodeURIComponent",
- "setTimeout",
- "clearTimeout",
- "setInterval",
- "clearInterval",
- "console.log",
- "console.error",
- "console.warn",
- "console.info",
- "console.debug",
- "JSON.parse",
- "JSON.stringify",
- "Math.random",
- "Math.floor",
- "Math.ceil",
- "Math.round",
- "Math.abs",
- "Math.max",
- "Math.min",
- "Date.now",
- "Date.parse",
- }
-)
-
-JS_METHOD_BIND = "bind"
-JS_METHOD_CALL = "call"
-JS_METHOD_APPLY = "apply"
-JS_SUFFIX_BIND = ".bind"
-JS_SUFFIX_CALL = ".call"
-JS_SUFFIX_APPLY = ".apply"
-JS_FUNCTION_PROTOTYPE_SUFFIXES: dict[str, str] = {
- JS_SUFFIX_BIND: JS_METHOD_BIND,
- JS_SUFFIX_CALL: JS_METHOD_CALL,
- JS_SUFFIX_APPLY: JS_METHOD_APPLY,
-}
-
-# (H) C++ operator mappings
-CPP_OPERATORS: dict[str, str] = {
- "operator_plus": "builtin.cpp.operator_plus",
- "operator_minus": "builtin.cpp.operator_minus",
- "operator_multiply": "builtin.cpp.operator_multiply",
- "operator_divide": "builtin.cpp.operator_divide",
- "operator_modulo": "builtin.cpp.operator_modulo",
- "operator_equal": "builtin.cpp.operator_equal",
- "operator_not_equal": "builtin.cpp.operator_not_equal",
- "operator_less": "builtin.cpp.operator_less",
- "operator_greater": "builtin.cpp.operator_greater",
- "operator_less_equal": "builtin.cpp.operator_less_equal",
- "operator_greater_equal": "builtin.cpp.operator_greater_equal",
- "operator_assign": "builtin.cpp.operator_assign",
- "operator_plus_assign": "builtin.cpp.operator_plus_assign",
- "operator_minus_assign": "builtin.cpp.operator_minus_assign",
- "operator_multiply_assign": "builtin.cpp.operator_multiply_assign",
- "operator_divide_assign": "builtin.cpp.operator_divide_assign",
- "operator_modulo_assign": "builtin.cpp.operator_modulo_assign",
- "operator_increment": "builtin.cpp.operator_increment",
- "operator_decrement": "builtin.cpp.operator_decrement",
- "operator_left_shift": "builtin.cpp.operator_left_shift",
- "operator_right_shift": "builtin.cpp.operator_right_shift",
- "operator_bitwise_and": "builtin.cpp.operator_bitwise_and",
- "operator_bitwise_or": "builtin.cpp.operator_bitwise_or",
- "operator_bitwise_xor": "builtin.cpp.operator_bitwise_xor",
- "operator_bitwise_not": "builtin.cpp.operator_bitwise_not",
- "operator_logical_and": "builtin.cpp.operator_logical_and",
- "operator_logical_or": "builtin.cpp.operator_logical_or",
- "operator_logical_not": "builtin.cpp.operator_logical_not",
- "operator_subscript": "builtin.cpp.operator_subscript",
- "operator_call": "builtin.cpp.operator_call",
-}
-
-# (H) Language CLI paths and patterns
-LANG_GRAMMARS_DIR = "grammars"
-LANG_CONFIG_FILE = "codebase_rag/language_spec.py"
-LANG_TREE_SITTER_JSON = "tree-sitter.json"
-LANG_NODE_TYPES_JSON = "node-types.json"
-LANG_SRC_DIR = "src"
-LANG_GIT_MODULES_PATH = ".git/modules/{path}"
-LANG_DEFAULT_GRAMMAR_URL = "https://github.com/tree-sitter/tree-sitter-{name}"
-LANG_TREE_SITTER_URL_MARKER = "github.com/tree-sitter/tree-sitter"
-
-# (H) Language CLI default node types
-LANG_DEFAULT_FUNCTION_NODES = ("function_definition", "method_definition")
-LANG_DEFAULT_CLASS_NODES = ("class_declaration",)
-LANG_DEFAULT_MODULE_NODES = ("compilation_unit",)
-LANG_DEFAULT_CALL_NODES = ("invocation_expression",)
-LANG_FALLBACK_METHOD_NODE = "method_declaration"
-
-# (H) Language CLI node type detection keywords
-LANG_FUNCTION_KEYWORDS = frozenset(
- {
- "function",
- "method",
- "constructor",
- "destructor",
- "lambda",
- "arrow_function",
- "anonymous_function",
- "closure",
- }
-)
-LANG_CLASS_KEYWORDS = frozenset(
- {
- "class",
- "interface",
- "struct",
- "enum",
- "trait",
- "object",
- "type",
- "impl",
- "union",
- }
-)
-LANG_CALL_KEYWORDS = frozenset({"call", "invoke", "invocation"})
-LANG_MODULE_KEYWORDS = frozenset(
- {"program", "source_file", "compilation_unit", "module", "chunk"}
-)
-LANG_EXCLUSION_KEYWORDS = frozenset({"access", "call"})
-
-# (H) Language CLI messages
-LANG_MSG_USING_DEFAULT_URL = "Using default tree-sitter URL: {url}"
-LANG_MSG_CUSTOM_URL_WARNING = (
- "WARNING: You are adding a grammar from a custom URL. "
- "This may execute code from the repository. Only proceed if you trust the source."
-)
-LANG_MSG_ADDING_SUBMODULE = "Adding submodule from {url}..."
-LANG_MSG_SUBMODULE_SUCCESS = "Successfully added submodule at {path}"
-LANG_MSG_SUBMODULE_EXISTS = (
- "Submodule already exists at {path}. Forcing re-installation..."
-)
-LANG_MSG_REMOVING_ENTRY = " -> Removing existing submodule entry..."
-LANG_MSG_READDING_SUBMODULE = " -> Re-adding submodule..."
-LANG_MSG_REINSTALL_SUCCESS = "Successfully re-installed submodule at {path}"
-LANG_MSG_AUTO_DETECTED_LANG = "Auto-detected language: {name}"
-LANG_MSG_USING_LANG_NAME = "Using language name: {name}"
-LANG_MSG_AUTO_DETECTED_EXT = "Auto-detected file extensions: {extensions}"
-LANG_MSG_FOUND_NODE_TYPES = "Found {count} total node types in grammar"
-LANG_MSG_SEMANTIC_CATEGORIES = "Tree-sitter semantic categories:"
-LANG_MSG_CATEGORY_FORMAT = " {category}: {subtypes} ({count} total)"
-LANG_MSG_MAPPED_CATEGORIES = "\nMapped to our categories:"
-LANG_MSG_FUNCTIONS = "Functions: {nodes}"
-LANG_MSG_CLASSES = "Classes: {nodes}"
-LANG_MSG_MODULES = "Modules: {nodes}"
-LANG_MSG_CALLS = "Calls: {nodes}"
-LANG_MSG_LANG_ADDED = "\nLanguage '{name}' has been added to the configuration!"
-LANG_MSG_UPDATED_CONFIG = "Updated {path}"
-LANG_MSG_REVIEW_PROMPT = "Please review the detected node types:"
-LANG_MSG_REVIEW_HINT = " The auto-detection is good but may need manual adjustments."
-LANG_MSG_EDIT_HINT = " Edit the configuration in: {path}"
-LANG_MSG_COMMON_ISSUES = "Look for these common issues:"
-LANG_MSG_ISSUE_MISCLASSIFIED = (
- " - Remove misclassified types (e.g., table_constructor in functions)"
-)
-LANG_MSG_ISSUE_MISSING = " - Add missing types that should be included"
-LANG_MSG_ISSUE_CLASS_TYPES = (
- " - Verify class_node_types includes all relevant class-like constructs"
-)
-LANG_MSG_ISSUE_CALL_TYPES = (
- " - Check call_node_types covers all function call patterns"
-)
-LANG_MSG_LIST_HINT = (
- "You can run 'cgr language list-languages' to see the current config."
-)
-LANG_MSG_LANG_NOT_FOUND = "Language '{name}' not found."
-LANG_MSG_AVAILABLE_LANGS = "Available languages: {langs}"
-LANG_MSG_REMOVED_FROM_CONFIG = "Removed language '{name}' from configuration file."
-LANG_MSG_REMOVING_SUBMODULE = "Removing git submodule '{path}'..."
-LANG_MSG_CLEANED_MODULES = "Cleaned up git modules directory: {path}"
-LANG_MSG_SUBMODULE_REMOVED = "Successfully removed submodule '{path}'"
-LANG_MSG_NO_SUBMODULE = "No submodule found at '{path}'"
-LANG_MSG_KEEPING_SUBMODULE = "Keeping submodule (--keep-submodule flag used)"
-LANG_MSG_LANG_REMOVED = "Language '{name}' has been removed successfully!"
-LANG_MSG_NO_MODULES_DIR = "No grammars modules directory found."
-LANG_MSG_NO_GITMODULES = "No .gitmodules file found."
-LANG_MSG_NO_ORPHANS = "No orphaned modules found!"
-LANG_MSG_FOUND_ORPHANS = "Found {count} orphaned module(s): {modules}"
-LANG_MSG_REMOVED_ORPHAN = "Removed orphaned module: {module}"
-LANG_MSG_CLEANUP_COMPLETE = "Cleanup complete!"
-LANG_MSG_CLEANUP_CANCELLED = "Cleanup cancelled."
-
-# (H) Language CLI error messages
-LANG_ERR_MISSING_ARGS = "Error: Either language_name or --grammar-url must be provided"
-LANG_ERR_REINSTALL_FAILED = "Failed to reinstall submodule: {error}"
-LANG_ERR_MANUAL_REMOVE_HINT = "You may need to remove it manually and try again:"
-LANG_ERR_REPO_NOT_FOUND = "Error: Repository not found at {url}"
-LANG_ERR_CUSTOM_URL_HINT = "Try using a custom URL with: --grammar-url "
-LANG_ERR_GIT = "Git error: {error}"
-LANG_ERR_NODE_TYPES_WARNING = (
- "Warning: node-types.json not found in any expected location for {name}"
-)
-LANG_ERR_TREE_SITTER_JSON_WARNING = "Warning: tree-sitter.json not found in {path}"
-LANG_ERR_NO_GRAMMARS_WARNING = "Warning: No grammars found in tree-sitter.json"
-LANG_ERR_PARSE_NODE_TYPES = "Error parsing node-types.json: {error}"
-LANG_ERR_UPDATE_CONFIG = "Error updating config file: {error}"
-LANG_ERR_CONFIG_NOT_FOUND = "Could not find LANGUAGE_SPECS dictionary end"
-LANG_ERR_REMOVE_CONFIG = "Failed to update config file: {error}"
-LANG_ERR_REMOVE_SUBMODULE = "Failed to remove submodule: {error}"
-
-# (H) Language CLI prompts
-LANG_PROMPT_LANGUAGE_NAME = "Language name (e.g., 'c-sharp', 'python')"
-LANG_PROMPT_COMMON_NAME = "What is the common name for this language?"
-LANG_PROMPT_EXTENSIONS = (
- "What file extensions should be associated with this language? (comma-separated)"
-)
-LANG_PROMPT_FUNCTIONS = "Select nodes representing FUNCTIONS (comma-separated)"
-LANG_PROMPT_CLASSES = "Select nodes representing CLASSES (comma-separated)"
-LANG_PROMPT_MODULES = "Select nodes representing MODULES (comma-separated)"
-LANG_PROMPT_CALLS = "Select nodes representing FUNCTION CALLS (comma-separated)"
-LANG_PROMPT_CONTINUE = "Do you want to continue?"
-LANG_PROMPT_REMOVE_ORPHANS = "Do you want to remove these orphaned modules?"
-
-# (H) Language CLI fallback manual add message
-LANG_FALLBACK_MANUAL_ADD = (
- "FALLBACK: Please manually add the following entry to "
- "'LANGUAGE_SPECS' in 'codebase_rag/language_spec.py':"
-)
-
-# (H) Language CLI table configuration
-LANG_TABLE_TITLE = "Configured Languages"
-LANG_TABLE_COL_LANGUAGE = "Language"
-LANG_TABLE_COL_EXTENSIONS = "Extensions"
-LANG_TABLE_COL_FUNCTION_TYPES = "Function Types"
-LANG_TABLE_COL_CLASS_TYPES = "Class Types"
-LANG_TABLE_COL_CALL_TYPES = "Call Types"
-LANG_TABLE_PLACEHOLDER = "—"
-
-LANG_MSG_AVAILABLE_NODES = "Available nodes for mapping:"
-LANG_ELLIPSIS = "..."
-LANG_GIT_SUFFIX = ".git"
-LANG_GITMODULES_FILE = ".gitmodules"
-LANG_CALL_KEYWORD_EXCLUDE = "call"
-
-# (H) Git submodule regex
-LANG_GITMODULES_REGEX = r"path = (grammars/tree-sitter-[^\\n]+)"
-
-
-class CppNodeType(StrEnum):
- TRANSLATION_UNIT = "translation_unit"
- NAMESPACE_DEFINITION = "namespace_definition"
- NAMESPACE_IDENTIFIER = "namespace_identifier"
- IDENTIFIER = "identifier"
- EXPORT = "export"
- EXPORT_KEYWORD = "export_keyword"
- PRIMITIVE_TYPE = "primitive_type"
- DECLARATION = "declaration"
- FUNCTION_DEFINITION = "function_definition"
- TEMPLATE_DECLARATION = "template_declaration"
- CLASS_SPECIFIER = "class_specifier"
- FUNCTION_DECLARATOR = "function_declarator"
- POINTER_DECLARATOR = "pointer_declarator"
- REFERENCE_DECLARATOR = "reference_declarator"
- FIELD_DECLARATION = "field_declaration"
- FIELD_IDENTIFIER = "field_identifier"
- QUALIFIED_IDENTIFIER = "qualified_identifier"
- OPERATOR_NAME = "operator_name"
- DESTRUCTOR_NAME = "destructor_name"
- CONSTRUCTOR_OR_DESTRUCTOR_DEFINITION = "constructor_or_destructor_definition"
- CONSTRUCTOR_OR_DESTRUCTOR_DECLARATION = "constructor_or_destructor_declaration"
- INLINE_METHOD_DEFINITION = "inline_method_definition"
- OPERATOR_CAST_DEFINITION = "operator_cast_definition"
-
-
-CPP_MODULE_EXTENSIONS = (".ixx", ".cppm", ".ccm", ".mxx")
-CPP_MODULE_PATH_MARKERS = frozenset({"interfaces", "modules"})
-
-# (H) C++ module declaration prefixes
-CPP_EXPORT_MODULE_PREFIX = "export module "
-CPP_MODULE_PREFIX = "module "
-CPP_MODULE_PRIVATE_PREFIX = "module ;"
-CPP_IMPL_SUFFIX = "_impl"
-
-# (H) C++ module type values
-CPP_MODULE_TYPE_INTERFACE = "interface"
-CPP_MODULE_TYPE_IMPLEMENTATION = "implementation"
-
-# (H) C++ export prefixes for class detection
-CPP_EXPORT_CLASS_PREFIX = "export class "
-CPP_EXPORT_STRUCT_PREFIX = "export struct "
-CPP_EXPORT_UNION_PREFIX = "export union "
-CPP_EXPORT_TEMPLATE_PREFIX = "export template"
-CPP_EXPORT_PREFIXES = (
- CPP_EXPORT_CLASS_PREFIX,
- CPP_EXPORT_STRUCT_PREFIX,
- CPP_EXPORT_UNION_PREFIX,
- CPP_EXPORT_TEMPLATE_PREFIX,
-)
-
-# (H) C++ keywords for class detection
-CPP_KEYWORD_CLASS = "class"
-CPP_KEYWORD_STRUCT = "struct"
-CPP_EXPORTED_CLASS_KEYWORDS = frozenset({CPP_KEYWORD_CLASS, CPP_KEYWORD_STRUCT})
-
-CPP_FALLBACK_OPERATOR = "operator_unknown"
-CPP_FALLBACK_DESTRUCTOR = "~destructor"
-CPP_OPERATOR_TEXT_PREFIX = "operator"
-CPP_DESTRUCTOR_PREFIX = "~"
-
-CPP_OPERATOR_SYMBOL_MAP: dict[str, str] = {
- "+": "operator_plus",
- "-": "operator_minus",
- "*": "operator_multiply",
- "/": "operator_divide",
- "%": "operator_modulo",
- "=": "operator_assign",
- "==": "operator_equal",
- "!=": "operator_not_equal",
- "<": "operator_less",
- ">": "operator_greater",
- "<=": "operator_less_equal",
- ">=": "operator_greater_equal",
- "&&": "operator_logical_and",
- "||": "operator_logical_or",
- "&": "operator_bitwise_and",
- "|": "operator_bitwise_or",
- "^": "operator_bitwise_xor",
- "~": "operator_bitwise_not",
- "!": "operator_not",
- "<<": "operator_left_shift",
- ">>": "operator_right_shift",
- "++": "operator_increment",
- "--": "operator_decrement",
- "+=": "operator_plus_assign",
- "-=": "operator_minus_assign",
- "*=": "operator_multiply_assign",
- "/=": "operator_divide_assign",
- "%=": "operator_modulo_assign",
- "&=": "operator_and_assign",
- "|=": "operator_or_assign",
- "^=": "operator_xor_assign",
- "<<=": "operator_left_shift_assign",
- ">>=": "operator_right_shift_assign",
- "[]": "operator_subscript",
- "()": "operator_call",
-}
-
-# (H) Dependency parser TOML/JSON keys
-DEP_KEY_TOOL = "tool"
-DEP_KEY_POETRY = "poetry"
-DEP_KEY_DEPENDENCIES = "dependencies"
-DEP_KEY_DEV_DEPENDENCIES = "dev-dependencies"
-DEP_KEY_PROJECT = "project"
-DEP_KEY_OPTIONAL_DEPS = "optional-dependencies"
-DEP_KEY_DEV_DEPS_JSON = "devDependencies"
-DEP_KEY_PEER_DEPS = "peerDependencies"
-DEP_KEY_REQUIRE = "require"
-DEP_KEY_REQUIRE_DEV = "require-dev"
-DEP_KEY_VERSION = "version"
-DEP_KEY_GROUP = "group"
-
-# (H) Dependency parser XML attributes
-DEP_ATTR_INCLUDE = "Include"
-DEP_ATTR_VERSION = "Version"
-DEP_XML_PACKAGE_REF = "PackageReference"
-
-# (H) Dependency parser language exclusions
-DEP_EXCLUDE_PYTHON = "python"
-DEP_EXCLUDE_PHP = "php"
-
-# (H) Dependency file names (lowercase)
-DEP_FILE_PYPROJECT = "pyproject.toml"
-DEP_FILE_REQUIREMENTS = "requirements.txt"
-DEP_FILE_PACKAGE_JSON = "package.json"
-DEP_FILE_CARGO = "cargo.toml"
-DEP_FILE_GOMOD = "go.mod"
-DEP_FILE_GEMFILE = "gemfile"
-DEP_FILE_COMPOSER = "composer.json"
-
-# (H) Go.mod parsing patterns
-GOMOD_REQUIRE_BLOCK_START = "require ("
-GOMOD_BLOCK_END = ")"
-GOMOD_REQUIRE_LINE_PREFIX = "require "
-GOMOD_COMMENT_PREFIX = "//"
-
-# (H) Gemfile parsing patterns
-GEMFILE_GEM_PREFIX = "gem "
-
-# (H) Import processor cache config
-IMPORT_CACHE_TTL = 3600
-IMPORT_CACHE_DIR = ".cache/codebase_rag"
-IMPORT_CACHE_FILE = "stdlib_cache.json"
-IMPORT_CACHE_KEY = "cache"
-IMPORT_TIMESTAMPS_KEY = "timestamps"
-
-# (H) Tree-sitter Python import node types
-TS_IMPORT_STATEMENT = "import_statement"
-TS_IMPORT_FROM_STATEMENT = "import_from_statement"
-TS_DOTTED_NAME = "dotted_name"
-TS_ALIASED_IMPORT = "aliased_import"
-TS_RELATIVE_IMPORT = "relative_import"
-TS_IMPORT_PREFIX = "import_prefix"
-TS_WILDCARD_IMPORT = "wildcard_import"
-
-# (H) Tree-sitter JS/TS import node types
-TS_STRING = "string"
-TS_IMPORT_CLAUSE = "import_clause"
-TS_LEXICAL_DECLARATION = "lexical_declaration"
-TS_EXPORT_STATEMENT = "export_statement"
-TS_NAMED_IMPORTS = "named_imports"
-TS_IMPORT_SPECIFIER = "import_specifier"
-TS_NAMESPACE_IMPORT = "namespace_import"
-TS_IDENTIFIER = "identifier"
-TS_VARIABLE_DECLARATOR = "variable_declarator"
-TS_CALL_EXPRESSION = "call_expression"
-TS_EXPORT_CLAUSE = "export_clause"
-TS_EXPORT_SPECIFIER = "export_specifier"
-
-# (H) Tree-sitter Java import node types
-TS_IMPORT_DECLARATION = "import_declaration"
-TS_STATIC = "static"
-TS_SCOPED_IDENTIFIER = "scoped_identifier"
-TS_ASTERISK = "asterisk"
-
-# (H) Tree-sitter Rust import node types
-TS_USE_DECLARATION = "use_declaration"
-
-# (H) Tree-sitter Go import node types
-TS_IMPORT_SPEC = "import_spec"
-TS_IMPORT_SPEC_LIST = "import_spec_list"
-TS_PACKAGE_IDENTIFIER = "package_identifier"
-TS_INTERPRETED_STRING_LITERAL = "interpreted_string_literal"
-
-# (H) Tree-sitter C++ import node types
-TS_PREPROC_INCLUDE = "preproc_include"
-TS_TEMPLATE_FUNCTION = "template_function"
-TS_DECLARATION = "declaration"
-TS_STRING_LITERAL = "string_literal"
-TS_SYSTEM_LIB_STRING = "system_lib_string"
-TS_TEMPLATE_ARGUMENT_LIST = "template_argument_list"
-TS_TYPE_DESCRIPTOR = "type_descriptor"
-TS_TYPE_IDENTIFIER = "type_identifier"
-LUA_STRING_TYPES = (TS_STRING, TS_STRING_LITERAL)
-
-# (H) Tree-sitter Lua node types
-TS_DOT_INDEX_EXPRESSION = "dot_index_expression"
-TS_LUA_VARIABLE_DECLARATION = "variable_declaration"
-TS_LUA_ASSIGNMENT_STATEMENT = "assignment_statement"
-TS_LUA_VARIABLE_LIST = "variable_list"
-TS_LUA_EXPRESSION_LIST = "expression_list"
-TS_LUA_FUNCTION_CALL = "function_call"
-TS_LUA_METHOD_INDEX_EXPRESSION = "method_index_expression"
-TS_LUA_IDENTIFIER = "identifier"
-TS_LUA_LOCAL_STATEMENT = "local_statement"
-LUA_STATEMENT_SUFFIX = "statement"
-LUA_DEFAULT_VAR_TYPES = (TS_LUA_IDENTIFIER,)
-
-# (H) Lua method separator
-LUA_METHOD_SEPARATOR = ":"
-
-# (H) Fallback display value
-STR_NONE = "None"
-
-# (H) Tree-sitter JS/TS utility node types
-TS_RETURN_STATEMENT = "return_statement"
-TS_RETURN = "return"
-TS_NEW_EXPRESSION = "new_expression"
-
-# (H) Tree-sitter class/module node types for class_ingest
-TS_MODULE_DECLARATION = "module_declaration"
-TS_IMPL_ITEM = "impl_item"
-TS_INTERFACE_DECLARATION = "interface_declaration"
-TS_ENUM_DECLARATION = "enum_declaration"
-TS_ENUM_SPECIFIER = "enum_specifier"
-TS_ENUM_CLASS_SPECIFIER = "enum_class_specifier"
-TS_TYPE_ALIAS_DECLARATION = "type_alias_declaration"
-TS_STRUCT_SPECIFIER = "struct_specifier"
-TS_UNION_SPECIFIER = "union_specifier"
-TS_CLASS_DECLARATION = "class_declaration"
-TS_NAMESPACE_DEFINITION = "namespace_definition"
-TS_ABSTRACT_CLASS_DECLARATION = "abstract_class_declaration"
-TS_INTERNAL_MODULE = "internal_module"
-
-# (H) Tree-sitter Go node types
-TS_GO_TYPE_DECLARATION = "type_declaration"
-TS_GO_SOURCE_FILE = "source_file"
-TS_GO_FUNCTION_DECLARATION = "function_declaration"
-TS_GO_METHOD_DECLARATION = "method_declaration"
-TS_GO_CALL_EXPRESSION = "call_expression"
-TS_GO_IMPORT_DECLARATION = "import_declaration"
-
-# (H) Tree-sitter Scala node types
-TS_SCALA_CLASS_DEFINITION = "class_definition"
-TS_SCALA_OBJECT_DEFINITION = "object_definition"
-TS_SCALA_TRAIT_DEFINITION = "trait_definition"
-TS_SCALA_COMPILATION_UNIT = "compilation_unit"
-TS_SCALA_FUNCTION_DEFINITION = "function_definition"
-TS_SCALA_FUNCTION_DECLARATION = "function_declaration"
-TS_SCALA_CALL_EXPRESSION = "call_expression"
-TS_SCALA_GENERIC_FUNCTION = "generic_function"
-TS_SCALA_FIELD_EXPRESSION = "field_expression"
-TS_SCALA_INFIX_EXPRESSION = "infix_expression"
-TS_SCALA_IMPORT_DECLARATION = "import_declaration"
-
-# (H) Tree-sitter C# node types
-TS_CS_STRUCT_DECLARATION = "struct_declaration"
-TS_CS_COMPILATION_UNIT = "compilation_unit"
-TS_CS_DESTRUCTOR_DECLARATION = "destructor_declaration"
-TS_CS_LOCAL_FUNCTION_STATEMENT = "local_function_statement"
-TS_CS_FUNCTION_POINTER_TYPE = "function_pointer_type"
-TS_CS_ANONYMOUS_METHOD_EXPRESSION = "anonymous_method_expression"
-TS_CS_LAMBDA_EXPRESSION = "lambda_expression"
-TS_CS_INVOCATION_EXPRESSION = "invocation_expression"
-
-# (H) Tree-sitter PHP node types
-TS_PHP_TRAIT_DECLARATION = "trait_declaration"
-TS_PHP_FUNCTION_STATIC_DECLARATION = "function_static_declaration"
-TS_PHP_ANONYMOUS_FUNCTION = "anonymous_function"
-TS_PHP_ARROW_FUNCTION = "arrow_function"
-TS_PHP_MEMBER_CALL_EXPRESSION = "member_call_expression"
-TS_PHP_SCOPED_CALL_EXPRESSION = "scoped_call_expression"
-TS_PHP_FUNCTION_CALL_EXPRESSION = "function_call_expression"
-TS_PHP_NULLSAFE_MEMBER_CALL_EXPRESSION = "nullsafe_member_call_expression"
-
-# (H) Tree-sitter Lua node types for language_spec
-TS_LUA_CHUNK = "chunk"
-TS_LUA_FUNCTION_DECLARATION = "function_declaration"
-TS_LUA_FUNCTION_DEFINITION = "function_definition"
-TS_LUA_FUNCTION_CALL = "function_call"
-
-# (H) Tree-sitter C++ node types for language_spec
-TS_CPP_FUNCTION_DEFINITION = "function_definition"
-TS_CPP_DECLARATION = "declaration"
-TS_CPP_FIELD_DECLARATION = "field_declaration"
-TS_CPP_TEMPLATE_DECLARATION = "template_declaration"
-TS_CPP_LAMBDA_EXPRESSION = "lambda_expression"
-TS_CPP_TRANSLATION_UNIT = "translation_unit"
-TS_CPP_LINKAGE_SPECIFICATION = "linkage_specification"
-TS_CPP_CALL_EXPRESSION = "call_expression"
-TS_CPP_FIELD_EXPRESSION = "field_expression"
-TS_CPP_SUBSCRIPT_EXPRESSION = "subscript_expression"
-TS_CPP_NEW_EXPRESSION = "new_expression"
-TS_CPP_DELETE_EXPRESSION = "delete_expression"
-TS_CPP_BINARY_EXPRESSION = "binary_expression"
-TS_CPP_UNARY_EXPRESSION = "unary_expression"
-TS_CPP_UPDATE_EXPRESSION = "update_expression"
-TS_CPP_FUNCTION_DECLARATOR = "function_declarator"
-
-# (H) Tree-sitter Java node types for language_spec
-TS_JAVA_METHOD_INVOCATION = "method_invocation"
-TS_JAVA_ANNOTATION_TYPE_DECLARATION = "annotation_type_declaration"
-
-TS_BASE_CLASS_CLAUSE = "base_class_clause"
-TS_TEMPLATE_TYPE = "template_type"
-TS_ACCESS_SPECIFIER = "access_specifier"
-TS_VIRTUAL = "virtual"
-TS_TYPE_LIST = "type_list"
-TS_CLASS_HERITAGE = "class_heritage"
-TS_EXTENDS_CLAUSE = "extends_clause"
-TS_MEMBER_EXPRESSION = "member_expression"
-TS_EXTENDS = "extends"
-TS_ARGUMENTS = "arguments"
-TS_EXTENDS_TYPE_CLAUSE = "extends_type_clause"
-TS_METHOD_DEFINITION = "method_definition"
-TS_DECORATOR = "decorator"
-TS_ERROR = "ERROR"
-TS_EXPRESSION_STATEMENT = "expression_statement"
-TS_STATEMENT_BLOCK = "statement_block"
-TS_PARENTHESIZED_EXPRESSION = "parenthesized_expression"
-TS_ATTRIBUTE = "attribute"
-
-FIELD_OPERATOR = "operator"
-
-# (H) Derived node type tuples for class ingestion
-CPP_CLASS_TYPES = (CppNodeType.CLASS_SPECIFIER, TS_STRUCT_SPECIFIER)
-CPP_COMPOUND_TYPES = (*CPP_CLASS_TYPES, TS_UNION_SPECIFIER, TS_ENUM_SPECIFIER)
-JS_TS_PARENT_REF_TYPES = (TS_IDENTIFIER, TS_MEMBER_EXPRESSION)
-
-# (H) Import processor function names
-IMPORT_REQUIRE = "require"
-IMPORT_PCALL = "pcall"
-IMPORT_IMPORT = "import"
-
-# (H) Lua stdlib module names
-LUA_STDLIB_MODULES = frozenset(
- {
- "string",
- "math",
- "table",
- "os",
- "io",
- "debug",
- "package",
- "coroutine",
- "utf8",
- "bit32",
- }
-)
-
-# (H) Entity type names
-ENTITY_CLASS = "Class"
-ENTITY_FUNCTION = "Function"
-ENTITY_METHOD = "Method"
-
-# (H) Anonymous function name prefixes
-PREFIX_LAMBDA = "lambda_"
-PREFIX_ANONYMOUS = "anonymous_"
-PREFIX_IIFE = "iife_"
-PREFIX_IIFE_DIRECT = "iife_direct_"
-PREFIX_ARROW = "arrow"
-PREFIX_FUNC = "func"
-
-# (H) C++ stdlib namespace and type inference prefixes
-CPP_STD_NAMESPACE = "std"
-CPP_PREFIX_IS = "is_"
-CPP_PREFIX_HAS = "has_"
-
-# (H) JSON keys for stdlib introspection subprocess responses
-JSON_KEY_HAS_ENTITY = "hasEntity"
-JSON_KEY_ENTITY_TYPE = "entityType"
-
-# (H) C++ stdlib entity names for heuristic detection
-CPP_STDLIB_ENTITIES = frozenset(
- {
- "vector",
- "string",
- "map",
- "set",
- "list",
- "deque",
- "unique_ptr",
- "shared_ptr",
- "weak_ptr",
- "thread",
- "mutex",
- "condition_variable",
- "future",
- "promise",
- "sort",
- "find",
- "copy",
- "transform",
- "accumulate",
- }
-)
-
-# (H) Java common class names for heuristic detection
-JAVA_STDLIB_CLASSES = frozenset(
- {
- "String",
- "Object",
- "Integer",
- "Double",
- "Boolean",
- "ArrayList",
- "HashMap",
- "HashSet",
- "LinkedList",
- "File",
- "URL",
- "Pattern",
- "LocalDateTime",
- "BigDecimal",
- }
-)
-
-# (H) Import processor misc
-IMPORT_DEFAULT_SUFFIX = ".default"
-IMPORT_STD_PREFIX = "std."
-CPP_STD_PREFIX = "std"
-IMPORT_MODULE_LABEL = "Module"
-IMPORT_QUALIFIED_NAME = "qualified_name"
-IMPORT_RELATIONSHIP = "IMPORTS"
-
-# (H) Java type inference constants
-JAVA_LANG_PREFIX = "java.lang."
-JAVA_ARRAY_SUFFIX = "[]"
-JAVA_SUFFIX_EXCEPTION = "Exception"
-JAVA_SUFFIX_ERROR = "Error"
-JAVA_SUFFIX_INTERFACE = "Interface"
-JAVA_SUFFIX_BUILDER = "Builder"
-JAVA_PRIMITIVE_TYPES = frozenset(
- {
- "int",
- "long",
- "double",
- "float",
- "boolean",
- "char",
- "byte",
- "short",
- }
-)
-JAVA_WRAPPER_TYPES = frozenset(
- {
- "String",
- "Object",
- "Integer",
- "Long",
- "Double",
- "Boolean",
- }
-)
-
-# (H) Java tree-sitter node types
-TS_FORMAL_PARAMETER = "formal_parameter"
-TS_SPREAD_PARAMETER = "spread_parameter"
-TS_LOCAL_VARIABLE_DECLARATION = "local_variable_declaration"
-TS_FIELD_DECLARATION = "field_declaration"
-TS_ASSIGNMENT_EXPRESSION = "assignment_expression"
-TS_OBJECT_CREATION_EXPRESSION = "object_creation_expression"
-TS_METHOD_INVOCATION = "method_invocation"
-TS_FIELD_ACCESS = "field_access"
-TS_INTEGER_LITERAL = "integer_literal"
-TS_DECIMAL_FLOATING_POINT_LITERAL = "decimal_floating_point_literal"
-TS_ARRAY_CREATION_EXPRESSION = "array_creation_expression"
-TS_METHOD_DECLARATION = "method_declaration"
-TS_ENHANCED_FOR_STATEMENT = "enhanced_for_statement"
-TS_RECORD_DECLARATION = "record_declaration"
-TS_TRUE = "true"
-TS_FALSE = "false"
-
-# (H) Tree-sitter field names for child_by_field_name
-TS_FIELD_NAME = "name"
-TS_FIELD_TYPE = "type"
-TS_FIELD_SUPERCLASS = "superclass"
-TS_FIELD_INTERFACES = "interfaces"
-TS_FIELD_TYPE_PARAMETERS = "type_parameters"
-TS_FIELD_PARAMETERS = "parameters"
-TS_FIELD_DECLARATOR = "declarator"
-TS_FIELD_OBJECT = "object"
-TS_FIELD_ARGUMENTS = "arguments"
-TS_FIELD_FUNCTION = "function"
-TS_FIELD_BODY = "body"
-TS_FIELD_LEFT = "left"
-TS_FIELD_RIGHT = "right"
-
-QUERY_CAPTURE_CLASS = "class"
-QUERY_CAPTURE_FUNCTION = "function"
-QUERY_KEY_CLASSES = "classes"
-QUERY_KEY_FUNCTIONS = "functions"
-
-# (H) Java type inference keywords
-JAVA_KEYWORD_THIS = "this"
-JAVA_KEYWORD_SUPER = "super"
-
-# (H) Java array type suffix
-JAVA_ARRAY_SUFFIX = "[]"
-
-# (H) Java heuristic patterns
-JAVA_GETTER_PATTERN = "get"
-JAVA_NAME_PATTERN = "name"
-JAVA_ID_PATTERN = "id"
-JAVA_SIZE_PATTERN = "size"
-JAVA_LENGTH_PATTERN = "length"
-JAVA_CREATE_PATTERN = "create"
-JAVA_NEW_PATTERN = "new"
-JAVA_IS_PATTERN = "is"
-JAVA_HAS_PATTERN = "has"
-JAVA_USER_PATTERN = "user"
-JAVA_ORDER_PATTERN = "order"
-
-# (H) Java entity type names
-ENTITY_CONSTRUCTOR = "Constructor"
-
-# (H) Java callable entity types for method resolution
-JAVA_CALLABLE_ENTITY_TYPES = frozenset({ENTITY_METHOD, ENTITY_CONSTRUCTOR})
-
-# (H) Java primitive type names
-JAVA_TYPE_STRING = "String"
-JAVA_TYPE_INT = "int"
-JAVA_TYPE_DOUBLE = "double"
-JAVA_TYPE_BOOLEAN = "boolean"
-JAVA_TYPE_LONG = "java.lang.Long"
-JAVA_TYPE_STRING_FQN = "java.lang.String"
-JAVA_TYPE_OBJECT = "Object"
-
-# (H) Java heuristic return type names
-JAVA_HEURISTIC_USER = "User"
-JAVA_HEURISTIC_ORDER = "Order"
-
-# (H) Java tree-sitter node types for java_utils
-TS_PACKAGE_DECLARATION = "package_declaration"
-TS_ANNOTATION_TYPE_DECLARATION = "annotation_type_declaration"
-TS_CONSTRUCTOR_DECLARATION = "constructor_declaration"
-TS_ANNOTATION = "annotation"
-TS_MARKER_ANNOTATION = "marker_annotation"
-TS_GENERIC_TYPE = "generic_type"
-TS_TYPE_PARAMETER = "type_parameter"
-TS_MODIFIERS = "modifiers"
-TS_VOID_TYPE = "void_type"
-TS_PROGRAM = "program"
-TS_THIS = "this"
-TS_SUPER = "super"
-
-# (H) Java modifier node types
-JAVA_MODIFIER_PUBLIC = "public"
-JAVA_MODIFIER_PRIVATE = "private"
-JAVA_MODIFIER_PROTECTED = "protected"
-JAVA_MODIFIER_STATIC = "static"
-JAVA_MODIFIER_FINAL = "final"
-JAVA_MODIFIER_ABSTRACT = "abstract"
-JAVA_MODIFIER_SYNCHRONIZED = "synchronized"
-JAVA_MODIFIER_TRANSIENT = "transient"
-JAVA_MODIFIER_VOLATILE = "volatile"
-
-JAVA_CLASS_MODIFIERS = frozenset(
- {
- JAVA_MODIFIER_PUBLIC,
- JAVA_MODIFIER_PRIVATE,
- JAVA_MODIFIER_PROTECTED,
- JAVA_MODIFIER_STATIC,
- JAVA_MODIFIER_FINAL,
- JAVA_MODIFIER_ABSTRACT,
- }
-)
-
-JAVA_METHOD_MODIFIERS = frozenset(
- {
- JAVA_MODIFIER_PUBLIC,
- JAVA_MODIFIER_PRIVATE,
- JAVA_MODIFIER_PROTECTED,
- JAVA_MODIFIER_STATIC,
- JAVA_MODIFIER_FINAL,
- JAVA_MODIFIER_ABSTRACT,
- JAVA_MODIFIER_SYNCHRONIZED,
- }
-)
-
-JAVA_FIELD_MODIFIERS = frozenset(
- {
- JAVA_MODIFIER_PUBLIC,
- JAVA_MODIFIER_PRIVATE,
- JAVA_MODIFIER_PROTECTED,
- JAVA_MODIFIER_STATIC,
- JAVA_MODIFIER_FINAL,
- JAVA_MODIFIER_TRANSIENT,
- JAVA_MODIFIER_VOLATILE,
- }
-)
-
-# (H) Java visibility values
-JAVA_VISIBILITY_PUBLIC = "public"
-JAVA_VISIBILITY_PROTECTED = "protected"
-JAVA_VISIBILITY_PRIVATE = "private"
-JAVA_VISIBILITY_PACKAGE = "package"
-
-# (H) Java class type suffixes and names
-JAVA_DECLARATION_SUFFIX = "_declaration"
-JAVA_TYPE_METHOD = "method"
-JAVA_TYPE_CONSTRUCTOR = "constructor"
-
-# (H) Java class node types for matching
-JAVA_CLASS_NODE_TYPES = frozenset(
- {
- TS_CLASS_DECLARATION,
- TS_INTERFACE_DECLARATION,
- TS_ENUM_DECLARATION,
- TS_ANNOTATION_TYPE_DECLARATION,
- TS_RECORD_DECLARATION,
- }
-)
-
-# (H) Java method node types
-JAVA_METHOD_NODE_TYPES = frozenset(
- {
- TS_METHOD_DECLARATION,
- TS_CONSTRUCTOR_DECLARATION,
- }
-)
-
-# (H) Java main method constants
-JAVA_MAIN_METHOD_NAME = "main"
-JAVA_MAIN_PARAM_ARRAY = "String[]"
-JAVA_MAIN_PARAM_VARARGS = "String..."
-JAVA_MAIN_PARAM_TYPE = "String"
-
-# (H) Java path parsing constants
-JAVA_PATH_JAVA = "java"
-JAVA_PATH_KOTLIN = "kotlin"
-JAVA_PATH_SCALA = "scala"
-JAVA_PATH_SRC = "src"
-JAVA_PATH_MAIN = "main"
-JAVA_PATH_TEST = "test"
-
-JAVA_JVM_LANGUAGES = frozenset(
- {
- JAVA_PATH_JAVA,
- JAVA_PATH_KOTLIN,
- JAVA_PATH_SCALA,
- }
-)
-
-JAVA_SRC_FOLDERS = frozenset(
- {
- JAVA_PATH_MAIN,
- JAVA_PATH_TEST,
- }
-)
-
-# (H) Delimiter tokens for argument parsing
-DELIMITER_TOKENS = frozenset({"(", ")", ","})
-
-# (H) Python tree-sitter node types for type inference
-TS_PY_IDENTIFIER = "identifier"
-TS_PY_TYPED_PARAMETER = "typed_parameter"
-TS_PY_TYPED_DEFAULT_PARAMETER = "typed_default_parameter"
-TS_PY_ATTRIBUTE = "attribute"
-TS_PY_CALL = "call"
-TS_PY_LIST = "list"
-TS_PY_LIST_COMPREHENSION = "list_comprehension"
-TS_PY_FOR_STATEMENT = "for_statement"
-TS_PY_FOR_IN_CLAUSE = "for_in_clause"
-TS_PY_ASSIGNMENT = "assignment"
-TS_PY_CLASS_DEFINITION = "class_definition"
-TS_PY_BLOCK = "block"
-TS_PY_FUNCTION_DEFINITION = "function_definition"
-TS_PY_RETURN_STATEMENT = "return_statement"
-TS_PY_RETURN = "return"
-TS_PY_KEYWORD = "keyword"
-TS_PY_MODULE = "module"
-TS_PY_IMPORT_STATEMENT = "import_statement"
-TS_PY_IMPORT_FROM_STATEMENT = "import_from_statement"
-TS_PY_WITH_STATEMENT = "with_statement"
-TS_PY_EXPRESSION_STATEMENT = "expression_statement"
-TS_PY_STRING = "string"
-TS_PY_DECORATED_DEFINITION = "decorated_definition"
-TS_PY_DECORATOR = "decorator"
-
-# (H) Python keyword identifiers
-PY_KEYWORD_SELF = "self"
-PY_KEYWORD_CLS = "cls"
-PY_METHOD_INIT = "__init__"
-
-# (H) Python attribute prefixes
-PY_SELF_PREFIX = "self."
-
-# (H) Python type inference patterns
-PY_VAR_PATTERN_ALL = "all_"
-PY_VAR_SUFFIX_PLURAL = "s"
-PY_CLASS_REPOSITORY = "Repository"
-PY_MODELS_BASE_PATH = ".models.base."
-PY_METHOD_CREATE = "create"
-
-# (H) Type inference scoring
-PY_SCORE_EXACT_MATCH = 100
-PY_SCORE_SUFFIX_MATCH = 90
-PY_SCORE_CONTAINS_BASE = 80
-
-# (H) Type inference defaults
-TYPE_INFERENCE_LIST = "list"
-TYPE_INFERENCE_BASE_MODEL = "BaseModel"
-
-# (H) Type inference guard attribute
-ATTR_TYPE_INFERENCE_IN_PROGRESS = "_type_inference_in_progress"
-
-# (H) JS/TS ingest node types
-TS_PAIR = "pair"
-TS_OBJECT = "object"
-TS_FUNCTION_EXPRESSION = "function_expression"
-TS_ARROW_FUNCTION = "arrow_function"
-TS_MODULE = "module"
-TS_CLASS_BODY = "class_body"
-TS_STATIC = "static"
-TS_PROPERTY_IDENTIFIER = "property_identifier"
-TS_VARIABLE_DECLARATOR = "variable_declarator"
-
-# (H) JS prototype property keywords
-JS_PROTOTYPE_KEYWORD = "prototype"
-JS_OBJECT_NAME = "Object"
-JS_CREATE_METHOD = "create"
-
-# (H) JS/TS ingest query capture names
-CAPTURE_CHILD_CLASS = "child_class"
-CAPTURE_PARENT_CLASS = "parent_class"
-CAPTURE_CONSTRUCTOR_NAME = "constructor_name"
-CAPTURE_PROTOTYPE_KEYWORD = "prototype_keyword"
-CAPTURE_METHOD_NAME = "method_name"
-CAPTURE_METHOD_FUNCTION = "method_function"
-CAPTURE_MEMBER_EXPR = "member_expr"
-CAPTURE_FUNCTION_EXPR = "function_expr"
-CAPTURE_ARROW_FUNCTION = "arrow_function"
-
-# (H) JS prototype inheritance query
-JS_PROTOTYPE_INHERITANCE_QUERY = """
-(assignment_expression
- left: (member_expression
- object: (identifier) @child_class
- property: (property_identifier) @prototype (#eq? @prototype "prototype"))
- right: (call_expression
- function: (member_expression
- object: (identifier) @object_name (#eq? @object_name "Object")
- property: (property_identifier) @create_method (#eq? @create_method "create"))
- arguments: (arguments
- (member_expression
- object: (identifier) @parent_class
- property: (property_identifier) @parent_prototype (#eq? @parent_prototype "prototype")))))
-"""
-
-# (H) JS prototype method assignment query
-JS_PROTOTYPE_METHOD_QUERY = """
-(assignment_expression
- left: (member_expression
- object: (member_expression
- object: (identifier) @constructor_name
- property: (property_identifier) @prototype_keyword (#eq? @prototype_keyword "prototype"))
- property: (property_identifier) @method_name)
- right: (function_expression) @method_function)
-"""
-
-# (H) JS object method query
-JS_OBJECT_METHOD_QUERY = """
-(pair
- key: (property_identifier) @method_name
- value: (function_expression) @method_function)
-"""
-
-# (H) JS method definition query
-JS_METHOD_DEF_QUERY = """
-(object
- (method_definition
- name: (property_identifier) @method_name) @method_function)
-"""
-
-# (H) JS object arrow function query
-JS_OBJECT_ARROW_QUERY = """
-(object
- (pair
- (property_identifier) @method_name
- (arrow_function) @arrow_function))
-"""
-
-# (H) JS assignment arrow function query
-JS_ASSIGNMENT_ARROW_QUERY = """
-(assignment_expression
- (member_expression) @member_expr
- (arrow_function) @arrow_function)
-"""
-
-# (H) JS assignment function expression query
-JS_ASSIGNMENT_FUNCTION_QUERY = """
-(assignment_expression
- (member_expression) @member_expr
- (function_expression) @function_expr)
-"""
-
-# (H) JS/TS module system node types
-TS_OBJECT_PATTERN = "object_pattern"
-TS_SHORTHAND_PROPERTY_IDENTIFIER_PATTERN = "shorthand_property_identifier_pattern"
-TS_PAIR_PATTERN = "pair_pattern"
-TS_FUNCTION_DECLARATION = "function_declaration"
-TS_GENERATOR_FUNCTION_DECLARATION = "generator_function_declaration"
-
-# (H) Tree-sitter field names for module system
-FIELD_FUNCTION = "function"
-FIELD_KEY = "key"
-
-# (H) JS/TS module system keywords
-JS_REQUIRE_KEYWORD = "require"
-JS_EXPORTS_KEYWORD = "exports"
-JS_MODULE_KEYWORD = "module"
-
-# (H) JS/TS export type descriptions
-JS_EXPORT_TYPE_COMMONJS = "CommonJS Export"
-JS_EXPORT_TYPE_COMMONJS_MODULE = "CommonJS Module Export"
-JS_EXPORT_TYPE_ES6_FUNCTION = "ES6 Export Function"
-JS_EXPORT_TYPE_ES6_FUNCTION_DECL = "ES6 Export Function Declaration"
-
-# (H) JS/TS CommonJS destructure query
-JS_COMMONJS_DESTRUCTURE_QUERY = """
-(lexical_declaration
- (variable_declarator
- name: (object_pattern)
- value: (call_expression
- function: (identifier) @func (#eq? @func "require")
- )
- ) @variable_declarator
-)
-"""
-
-# (H) JS/TS CommonJS exports function query
-JS_COMMONJS_EXPORTS_FUNCTION_QUERY = """
-(assignment_expression
- left: (member_expression
- object: (identifier) @exports_obj
- property: (property_identifier) @export_name)
- right: [(function_expression) (arrow_function)] @export_function)
-"""
-
-# (H) JS/TS CommonJS module.exports query
-JS_COMMONJS_MODULE_EXPORTS_QUERY = """
-(assignment_expression
- left: (member_expression
- object: (member_expression
- object: (identifier) @module_obj
- property: (property_identifier) @exports_prop)
- property: (property_identifier) @export_name)
- right: [(function_expression) (arrow_function)] @export_function)
-"""
-
-# (H) JS/TS ES6 export const query
-JS_ES6_EXPORT_CONST_QUERY = """
-(export_statement
- (lexical_declaration
- (variable_declarator
- name: (identifier) @export_name
- value: [(function_expression) (arrow_function)] @export_function)))
-"""
-
-# (H) JS/TS ES6 export function query
-JS_ES6_EXPORT_FUNCTION_QUERY = """
-(export_statement
- [(function_declaration) (generator_function_declaration)] @export_function)
-"""
-
-# (H) Query capture names for module system
-CAPTURE_FUNC = "func"
-CAPTURE_VARIABLE_DECLARATOR = "variable_declarator"
-CAPTURE_EXPORTS_OBJ = "exports_obj"
-CAPTURE_MODULE_OBJ = "module_obj"
-CAPTURE_EXPORTS_PROP = "exports_prop"
-CAPTURE_EXPORT_NAME = "export_name"
-CAPTURE_EXPORT_FUNCTION = "export_function"
-
-# (H) Tree-sitter Rust node types
-TS_RS_SCOPED_TYPE_IDENTIFIER = "scoped_type_identifier"
-TS_RS_USE_AS_CLAUSE = "use_as_clause"
-TS_RS_USE_WILDCARD = "use_wildcard"
-TS_RS_USE_LIST = "use_list"
-TS_RS_SCOPED_USE_LIST = "scoped_use_list"
-TS_RS_SOURCE_FILE = "source_file"
-TS_RS_MOD_ITEM = "mod_item"
-TS_RS_CRATE = "crate"
-TS_RS_KEYWORD_AS = "as"
-TS_RS_STRUCT_ITEM = "struct_item"
-TS_RS_ENUM_ITEM = "enum_item"
-TS_RS_TRAIT_ITEM = "trait_item"
-TS_RS_TYPE_ITEM = "type_item"
-TS_RS_FUNCTION_ITEM = "function_item"
-TS_RS_IMPL_ITEM = "impl_item"
-TS_RS_FUNCTION_SIGNATURE_ITEM = "function_signature_item"
-TS_RS_CLOSURE_EXPRESSION = "closure_expression"
-TS_RS_UNION_ITEM = "union_item"
-TS_RS_USE_DECLARATION = "use_declaration"
-TS_RS_EXTERN_CRATE_DECLARATION = "extern_crate_declaration"
-TS_RS_CALL_EXPRESSION = "call_expression"
-TS_RS_MACRO_INVOCATION = "macro_invocation"
-TS_RS_ATTRIBUTE_ITEM = "attribute_item"
-TS_RS_INNER_ATTRIBUTE_ITEM = "inner_attribute_item"
-
-# (H) Rust identifier tuples
-RS_IDENTIFIER_TYPES = (TS_IDENTIFIER, TS_TYPE_IDENTIFIER)
-RS_SCOPED_TYPES = (TS_SCOPED_IDENTIFIER, TS_RS_SCOPED_TYPE_IDENTIFIER)
-RS_PATH_KEYWORDS = (TS_RS_CRATE, KEYWORD_SUPER, KEYWORD_SELF)
-
-# (H) Delimiter tokens for Rust use lists
-RS_USE_LIST_DELIMITERS = frozenset({"{", "}", ","})
-
-# (H) Rust encoding
-RS_ENCODING_UTF8 = "utf8"
-
-# (H) Rust wildcard prefix
-RS_WILDCARD_PREFIX = "*"
-
-# (H) Rust field names
-RS_FIELD_ARGUMENT = "argument"
-
-
-# (H) MCP tool names
-class MCPToolName(StrEnum):
- LIST_PROJECTS = "list_projects"
- DELETE_PROJECT = "delete_project"
- WIPE_DATABASE = "wipe_database"
- INDEX_REPOSITORY = "index_repository"
- QUERY_CODE_GRAPH = "query_code_graph"
- GET_CODE_SNIPPET = "get_code_snippet"
- SURGICAL_REPLACE_CODE = "surgical_replace_code"
- READ_FILE = "read_file"
- WRITE_FILE = "write_file"
- LIST_DIRECTORY = "list_directory"
-
-
-# (H) MCP environment variables
-class MCPEnvVar(StrEnum):
- TARGET_REPO_PATH = "TARGET_REPO_PATH"
- CLAUDE_PROJECT_ROOT = "CLAUDE_PROJECT_ROOT"
- PWD = "PWD"
-
-
-# (H) MCP schema types
-class MCPSchemaType(StrEnum):
- OBJECT = "object"
- STRING = "string"
- INTEGER = "integer"
- BOOLEAN = "boolean"
-
-
-# (H) MCP schema fields
-class MCPSchemaField(StrEnum):
- TYPE = "type"
- PROPERTIES = "properties"
- REQUIRED = "required"
- DESCRIPTION = "description"
- DEFAULT = "default"
-
-
-# (H) MCP parameter names
-class MCPParamName(StrEnum):
- PROJECT_NAME = "project_name"
- CONFIRM = "confirm"
- NATURAL_LANGUAGE_QUERY = "natural_language_query"
- QUALIFIED_NAME = "qualified_name"
- FILE_PATH = "file_path"
- TARGET_CODE = "target_code"
- REPLACEMENT_CODE = "replacement_code"
- OFFSET = "offset"
- LIMIT = "limit"
- CONTENT = "content"
- DIRECTORY_PATH = "directory_path"
-
-
-# (H) MCP server constants
-MCP_SERVER_NAME = "code-graph-rag"
-MCP_CONTENT_TYPE_TEXT = "text"
-MCP_DEFAULT_DIRECTORY = "."
-MCP_JSON_INDENT = 2
-MCP_LOG_LEVEL_INFO = "INFO"
-MCP_LOG_FORMAT = "{time:YYYY-MM-DD HH:mm:ss} | {level: <8} | {message}"
-MCP_PAGINATION_HEADER = "# Lines {start}-{end} of {total}\n"
-
-# (H) MCP response messages
-MCP_INDEX_SUCCESS = "Successfully indexed repository at {path}. Knowledge graph has been updated (previous data cleared)."
-MCP_INDEX_SUCCESS_PROJECT = "Successfully indexed repository at {path}. Project '{project_name}' has been updated."
-MCP_INDEX_ERROR = "Error indexing repository: {error}"
-MCP_WRITE_SUCCESS = "Successfully wrote file: {path}"
-MCP_UNKNOWN_TOOL_ERROR = "Unknown tool: {name}"
-MCP_TOOL_EXEC_ERROR = "Error executing tool '{name}': {error}"
-MCP_PROJECT_DELETED = "Successfully deleted project '{project_name}'."
-MCP_WIPE_CANCELLED = "Database wipe cancelled. Set confirm=true to proceed."
-MCP_WIPE_SUCCESS = "Database completely wiped. All projects have been removed."
-MCP_WIPE_ERROR = "Error wiping database: {error}"
-
-# (H) MCP dict keys and values
-MCP_KEY_RESULTS = "results"
-MCP_KEY_ERROR = "error"
-MCP_KEY_FOUND = "found"
-MCP_KEY_ERROR_MESSAGE = "error_message"
-MCP_KEY_QUERY_USED = "query_used"
-MCP_KEY_SUMMARY = "summary"
-MCP_NOT_AVAILABLE = "N/A"
-MCP_TOOL_NAME_QUERY = "query"
-
-# (H) TS-specific node types
-TS_FUNCTION_SIGNATURE = "function_signature"
-
-# (H) FQN node type tuples for Python
-FQN_PY_SCOPE_TYPES = (
- TS_PY_CLASS_DEFINITION,
- TS_PY_MODULE,
- TS_PY_FUNCTION_DEFINITION,
-)
-FQN_PY_FUNCTION_TYPES = (TS_PY_FUNCTION_DEFINITION,)
-
-# (H) FQN node type tuples for JS
-FQN_JS_SCOPE_TYPES = (
- TS_CLASS_DECLARATION,
- TS_PROGRAM,
- TS_FUNCTION_DECLARATION,
- TS_FUNCTION_EXPRESSION,
- TS_ARROW_FUNCTION,
- TS_METHOD_DEFINITION,
-)
-FQN_JS_FUNCTION_TYPES = (
- TS_FUNCTION_DECLARATION,
- TS_METHOD_DEFINITION,
- TS_ARROW_FUNCTION,
- TS_FUNCTION_EXPRESSION,
-)
-
-# (H) FQN node type tuples for TS
-FQN_TS_SCOPE_TYPES = (
- TS_CLASS_DECLARATION,
- TS_INTERFACE_DECLARATION,
- TS_NAMESPACE_DEFINITION,
- TS_PROGRAM,
- TS_FUNCTION_DECLARATION,
- TS_FUNCTION_EXPRESSION,
- TS_ARROW_FUNCTION,
- TS_METHOD_DEFINITION,
-)
-FQN_TS_FUNCTION_TYPES = (
- TS_FUNCTION_DECLARATION,
- TS_METHOD_DEFINITION,
- TS_ARROW_FUNCTION,
- TS_FUNCTION_EXPRESSION,
- TS_FUNCTION_SIGNATURE,
-)
-
-# (H) FQN node type tuples for Rust
-FQN_RS_SCOPE_TYPES = (
- TS_RS_STRUCT_ITEM,
- TS_RS_ENUM_ITEM,
- TS_RS_TRAIT_ITEM,
- TS_RS_IMPL_ITEM,
- TS_RS_MOD_ITEM,
- TS_RS_SOURCE_FILE,
-)
-FQN_RS_FUNCTION_TYPES = (
- TS_RS_FUNCTION_ITEM,
- TS_RS_FUNCTION_SIGNATURE_ITEM,
- TS_RS_CLOSURE_EXPRESSION,
-)
-
-# (H) FQN node type tuples for Java
-FQN_JAVA_SCOPE_TYPES = (
- TS_CLASS_DECLARATION,
- TS_INTERFACE_DECLARATION,
- TS_ENUM_DECLARATION,
- TS_PROGRAM,
-)
-FQN_JAVA_FUNCTION_TYPES = (
- TS_METHOD_DECLARATION,
- TS_CONSTRUCTOR_DECLARATION,
-)
-
-# (H) FQN node type tuples for C++
-FQN_CPP_SCOPE_TYPES = (
- CppNodeType.CLASS_SPECIFIER,
- TS_STRUCT_SPECIFIER,
- TS_NAMESPACE_DEFINITION,
- TS_CPP_TRANSLATION_UNIT,
-)
-FQN_CPP_FUNCTION_TYPES = (
- TS_CPP_FUNCTION_DEFINITION,
- TS_CPP_DECLARATION,
- TS_CPP_FIELD_DECLARATION,
- TS_CPP_TEMPLATE_DECLARATION,
- TS_CPP_LAMBDA_EXPRESSION,
-)
-
-# (H) FQN node type tuples for Lua
-FQN_LUA_SCOPE_TYPES = (TS_LUA_CHUNK,)
-FQN_LUA_FUNCTION_TYPES = (
- TS_LUA_FUNCTION_DECLARATION,
- TS_LUA_FUNCTION_DEFINITION,
-)
-
-# (H) FQN node type tuples for Go
-FQN_GO_SCOPE_TYPES = (
- TS_GO_TYPE_DECLARATION,
- TS_GO_SOURCE_FILE,
-)
-FQN_GO_FUNCTION_TYPES = (
- TS_GO_FUNCTION_DECLARATION,
- TS_GO_METHOD_DECLARATION,
-)
-
-# (H) FQN node type tuples for Scala
-FQN_SCALA_SCOPE_TYPES = (
- TS_SCALA_CLASS_DEFINITION,
- TS_SCALA_OBJECT_DEFINITION,
- TS_SCALA_TRAIT_DEFINITION,
- TS_SCALA_COMPILATION_UNIT,
-)
-FQN_SCALA_FUNCTION_TYPES = (
- TS_SCALA_FUNCTION_DEFINITION,
- TS_SCALA_FUNCTION_DECLARATION,
-)
-
-# (H) FQN node type tuples for C#
-FQN_CS_SCOPE_TYPES = (
- TS_CLASS_DECLARATION,
- TS_CS_STRUCT_DECLARATION,
- TS_INTERFACE_DECLARATION,
- TS_CS_COMPILATION_UNIT,
-)
-FQN_CS_FUNCTION_TYPES = (
- TS_CS_DESTRUCTOR_DECLARATION,
- TS_CS_LOCAL_FUNCTION_STATEMENT,
- TS_CS_FUNCTION_POINTER_TYPE,
- TS_CONSTRUCTOR_DECLARATION,
- TS_CS_ANONYMOUS_METHOD_EXPRESSION,
- TS_CS_LAMBDA_EXPRESSION,
- TS_METHOD_DECLARATION,
-)
-
-# (H) FQN node type tuples for PHP
-FQN_PHP_SCOPE_TYPES = (
- TS_CLASS_DECLARATION,
- TS_INTERFACE_DECLARATION,
- TS_PHP_TRAIT_DECLARATION,
- TS_PROGRAM,
-)
-FQN_PHP_FUNCTION_TYPES = (
- TS_PY_FUNCTION_DEFINITION,
- TS_PHP_ANONYMOUS_FUNCTION,
- TS_PHP_ARROW_FUNCTION,
- TS_PHP_FUNCTION_STATIC_DECLARATION,
-)
-
-# (H) LANGUAGE_SPECS node type tuples for Python
-SPEC_PY_FUNCTION_TYPES = (TS_PY_FUNCTION_DEFINITION,)
-SPEC_PY_CLASS_TYPES = (TS_PY_CLASS_DEFINITION,)
-SPEC_PY_MODULE_TYPES = (TS_PY_MODULE,)
-SPEC_PY_CALL_TYPES = (TS_PY_CALL, TS_PY_WITH_STATEMENT)
-SPEC_PY_IMPORT_TYPES = (TS_PY_IMPORT_STATEMENT,)
-SPEC_PY_IMPORT_FROM_TYPES = (TS_PY_IMPORT_FROM_STATEMENT,)
-SPEC_PY_PACKAGE_INDICATORS = (PKG_INIT_PY,)
-
-# (H) LANGUAGE_SPECS node type tuples for JS/TS
-SPEC_JS_MODULE_TYPES = (TS_PROGRAM,)
-SPEC_JS_CALL_TYPES = (TS_CALL_EXPRESSION,)
-
-# (H) Derived node types for _js_get_name
-JS_NAME_NODE_TYPES = (
- TS_FUNCTION_DECLARATION,
- TS_CLASS_DECLARATION,
- TS_METHOD_DEFINITION,
-)
-
-# (H) Derived node types for _rust_get_name
-RS_TYPE_NODE_TYPES = (
- TS_RS_STRUCT_ITEM,
- TS_RS_ENUM_ITEM,
- TS_RS_TRAIT_ITEM,
- TS_RS_TYPE_ITEM,
-)
-RS_IDENT_NODE_TYPES = (TS_RS_FUNCTION_ITEM, TS_RS_MOD_ITEM)
-
-# (H) Derived node types for _cpp_get_name
-CPP_NAME_NODE_TYPES = (
- CppNodeType.CLASS_SPECIFIER,
- TS_STRUCT_SPECIFIER,
- TS_ENUM_SPECIFIER,
-)
-
-# (H) LANGUAGE_SPECS node type tuples for Rust
-SPEC_RS_FUNCTION_TYPES = (
- TS_RS_FUNCTION_ITEM,
- TS_RS_FUNCTION_SIGNATURE_ITEM,
- TS_RS_CLOSURE_EXPRESSION,
-)
-SPEC_RS_CLASS_TYPES = (
- TS_RS_STRUCT_ITEM,
- TS_RS_ENUM_ITEM,
- TS_RS_UNION_ITEM,
- TS_RS_TRAIT_ITEM,
- TS_RS_IMPL_ITEM,
- TS_RS_TYPE_ITEM,
-)
-SPEC_RS_MODULE_TYPES = (TS_RS_SOURCE_FILE, TS_RS_MOD_ITEM)
-SPEC_RS_CALL_TYPES = (TS_RS_CALL_EXPRESSION, TS_RS_MACRO_INVOCATION)
-SPEC_RS_IMPORT_TYPES = (TS_RS_USE_DECLARATION, TS_RS_EXTERN_CRATE_DECLARATION)
-SPEC_RS_IMPORT_FROM_TYPES = (TS_RS_USE_DECLARATION,)
-SPEC_RS_PACKAGE_INDICATORS = (PKG_CARGO_TOML,)
-
-# (H) LANGUAGE_SPECS node type tuples for Go
-SPEC_GO_FUNCTION_TYPES = (TS_GO_FUNCTION_DECLARATION, TS_GO_METHOD_DECLARATION)
-SPEC_GO_CLASS_TYPES = (TS_GO_TYPE_DECLARATION,)
-SPEC_GO_MODULE_TYPES = (TS_GO_SOURCE_FILE,)
-SPEC_GO_CALL_TYPES = (TS_GO_CALL_EXPRESSION,)
-SPEC_GO_IMPORT_TYPES = (TS_GO_IMPORT_DECLARATION,)
-
-# (H) LANGUAGE_SPECS node type tuples for Scala
-SPEC_SCALA_FUNCTION_TYPES = (
- TS_SCALA_FUNCTION_DEFINITION,
- TS_SCALA_FUNCTION_DECLARATION,
-)
-SPEC_SCALA_CLASS_TYPES = (
- TS_SCALA_CLASS_DEFINITION,
- TS_SCALA_OBJECT_DEFINITION,
- TS_SCALA_TRAIT_DEFINITION,
-)
-SPEC_SCALA_MODULE_TYPES = (TS_SCALA_COMPILATION_UNIT,)
-SPEC_SCALA_CALL_TYPES = (
- TS_SCALA_CALL_EXPRESSION,
- TS_SCALA_GENERIC_FUNCTION,
- TS_SCALA_FIELD_EXPRESSION,
- TS_SCALA_INFIX_EXPRESSION,
-)
-SPEC_SCALA_IMPORT_TYPES = (TS_SCALA_IMPORT_DECLARATION,)
-
-# (H) LANGUAGE_SPECS node type tuples for Java
-SPEC_JAVA_FUNCTION_TYPES = (TS_METHOD_DECLARATION, TS_CONSTRUCTOR_DECLARATION)
-SPEC_JAVA_CLASS_TYPES = (
- TS_CLASS_DECLARATION,
- TS_INTERFACE_DECLARATION,
- TS_ENUM_DECLARATION,
- TS_JAVA_ANNOTATION_TYPE_DECLARATION,
- TS_RECORD_DECLARATION,
-)
-SPEC_JAVA_MODULE_TYPES = (TS_PROGRAM,)
-SPEC_JAVA_CALL_TYPES = (TS_JAVA_METHOD_INVOCATION,)
-SPEC_JAVA_IMPORT_TYPES = (TS_IMPORT_DECLARATION,)
-
-# (H) LANGUAGE_SPECS node type tuples for C++
-SPEC_CPP_FUNCTION_TYPES = (
- TS_CPP_FUNCTION_DEFINITION,
- TS_CPP_DECLARATION,
- TS_CPP_FIELD_DECLARATION,
- TS_CPP_TEMPLATE_DECLARATION,
- TS_CPP_LAMBDA_EXPRESSION,
-)
-SPEC_CPP_CLASS_TYPES = (
- CppNodeType.CLASS_SPECIFIER,
- TS_STRUCT_SPECIFIER,
- TS_UNION_SPECIFIER,
- TS_ENUM_SPECIFIER,
-)
-SPEC_CPP_MODULE_TYPES = (
- TS_CPP_TRANSLATION_UNIT,
- TS_NAMESPACE_DEFINITION,
- TS_CPP_LINKAGE_SPECIFICATION,
- TS_CPP_DECLARATION,
-)
-SPEC_CPP_CALL_TYPES = (
- TS_CPP_CALL_EXPRESSION,
- TS_CPP_FIELD_EXPRESSION,
- TS_CPP_SUBSCRIPT_EXPRESSION,
- TS_CPP_NEW_EXPRESSION,
- TS_CPP_DELETE_EXPRESSION,
- TS_CPP_BINARY_EXPRESSION,
- TS_CPP_UNARY_EXPRESSION,
- TS_CPP_UPDATE_EXPRESSION,
-)
-SPEC_CPP_PACKAGE_INDICATORS = (
- PKG_CMAKE_LISTS,
- PKG_MAKEFILE,
- PKG_VCXPROJ_GLOB,
- PKG_CONANFILE,
-)
-
-# (H) LANGUAGE_SPECS node type tuples for C#
-SPEC_CS_FUNCTION_TYPES = (
- TS_CS_DESTRUCTOR_DECLARATION,
- TS_CS_LOCAL_FUNCTION_STATEMENT,
- TS_CS_FUNCTION_POINTER_TYPE,
- TS_CONSTRUCTOR_DECLARATION,
- TS_CS_ANONYMOUS_METHOD_EXPRESSION,
- TS_CS_LAMBDA_EXPRESSION,
- TS_METHOD_DECLARATION,
-)
-SPEC_CS_CLASS_TYPES = (
- TS_CLASS_DECLARATION,
- TS_CS_STRUCT_DECLARATION,
- TS_ENUM_DECLARATION,
- TS_INTERFACE_DECLARATION,
-)
-SPEC_CS_MODULE_TYPES = (TS_CS_COMPILATION_UNIT,)
-SPEC_CS_CALL_TYPES = (TS_CS_INVOCATION_EXPRESSION,)
-
-# (H) LANGUAGE_SPECS node type tuples for PHP
-SPEC_PHP_FUNCTION_TYPES = (
- TS_PHP_FUNCTION_STATIC_DECLARATION,
- TS_PHP_ANONYMOUS_FUNCTION,
- TS_PY_FUNCTION_DEFINITION,
- TS_PHP_ARROW_FUNCTION,
-)
-SPEC_PHP_CLASS_TYPES = (
- TS_PHP_TRAIT_DECLARATION,
- TS_ENUM_DECLARATION,
- TS_INTERFACE_DECLARATION,
- TS_CLASS_DECLARATION,
-)
-SPEC_PHP_MODULE_TYPES = (TS_PROGRAM,)
-SPEC_PHP_CALL_TYPES = (
- TS_PHP_MEMBER_CALL_EXPRESSION,
- TS_PHP_SCOPED_CALL_EXPRESSION,
- TS_PHP_FUNCTION_CALL_EXPRESSION,
- TS_PHP_NULLSAFE_MEMBER_CALL_EXPRESSION,
-)
-
-# (H) LANGUAGE_SPECS node type tuples for Lua
-SPEC_LUA_FUNCTION_TYPES = (TS_LUA_FUNCTION_DECLARATION, TS_LUA_FUNCTION_DEFINITION)
-SPEC_LUA_CLASS_TYPES: tuple[str, ...] = ()
-SPEC_LUA_MODULE_TYPES = (TS_LUA_CHUNK,)
-SPEC_LUA_CALL_TYPES = (TS_LUA_FUNCTION_CALL,)
-SPEC_LUA_IMPORT_TYPES = (TS_LUA_FUNCTION_CALL,)
-
-HEALTH_CHECK_DOCKER_RUNNING = "Docker daemon is running"
-HEALTH_CHECK_DOCKER_NOT_RUNNING = "Docker daemon is not running"
-HEALTH_CHECK_DOCKER_RUNNING_MSG = "Running (version {version})"
-HEALTH_CHECK_DOCKER_NOT_RESPONDING_MSG = "Not responding"
-HEALTH_CHECK_DOCKER_NOT_INSTALLED_MSG = "Not installed"
-HEALTH_CHECK_DOCKER_NOT_IN_PATH = "docker command not found in PATH"
-HEALTH_CHECK_DOCKER_TIMEOUT_MSG = "Check timed out"
-HEALTH_CHECK_DOCKER_TIMEOUT_ERROR = (
- "The 'docker info' command took more than 5 seconds to respond."
-)
-HEALTH_CHECK_DOCKER_FAILED_MSG = "Check failed"
-HEALTH_CHECK_DOCKER_EXIT_CODE = "Non-zero exit code"
-
-HEALTH_CHECK_MEMGRAPH_SUCCESSFUL = "Memgraph connection successful"
-HEALTH_CHECK_MEMGRAPH_FAILED = "Memgraph connection failed"
-HEALTH_CHECK_MEMGRAPH_CONNECTED_MSG = "Connected and responsive at {host}:{port}"
-HEALTH_CHECK_MEMGRAPH_CONNECTION_FAILED_MSG = "Connection or query failed"
-HEALTH_CHECK_MEMGRAPH_UNEXPECTED_FAILURE_MSG = "Unexpected failure"
-HEALTH_CHECK_MEMGRAPH_ERROR = "Memgraph error: {error}"
-HEALTH_CHECK_MEMGRAPH_QUERY = "RETURN 1 AS test;"
-
-HEALTH_CHECK_API_KEY_SET = "{display_name} API key is set"
-HEALTH_CHECK_API_KEY_NOT_SET = "{display_name} API key is not set"
-HEALTH_CHECK_API_KEY_CONFIGURED = "Configured"
-HEALTH_CHECK_API_KEY_NOT_CONFIGURED = "Not set"
-HEALTH_CHECK_API_KEY_MISSING_MSG = (
- "Set the {env_name} environment variable or configure it in your settings."
-)
-
-HEALTH_CHECK_TOOL_INSTALLED = "{tool_name} is installed"
-HEALTH_CHECK_TOOL_NOT_INSTALLED = "{tool_name} is not installed"
-HEALTH_CHECK_TOOL_INSTALLED_MSG = "Installed ({path})"
-HEALTH_CHECK_TOOL_NOT_IN_PATH_MSG = "'{cmd}' not found in PATH"
-HEALTH_CHECK_TOOL_TIMEOUT_MSG = "Check timed out"
-HEALTH_CHECK_TOOL_TIMEOUT_ERROR = (
- "The command to find '{cmd}' took more than 4 seconds to respond."
-)
-HEALTH_CHECK_TOOL_FAILED_MSG = "Check failed"
-
-HEALTH_CHECK_TOOLS = [
- ("GEMINI_API_KEY", "Gemini"),
- ("OPENAI_API_KEY", "OpenAI"),
- ("ORCHESTRATOR_API_KEY", "Orchestrator"),
- ("CYPHER_API_KEY", "Cypher"),
-]
-
-HEALTH_CHECK_EXTERNAL_TOOLS = [
- ("ripgrep", "rg"),
- ("cmake", "cmake"),
-]
-
-SHELL_CMD_WHERE = "where"
-SHELL_CMD_WHICH = "which"
diff --git a/codebase_rag/constants/__init__.py b/codebase_rag/constants/__init__.py
new file mode 100644
index 000000000..dadbc205a
--- /dev/null
+++ b/codebase_rag/constants/__init__.py
@@ -0,0 +1,30 @@
+# Application constants, re-exported from themed submodules.
+
+from .ast_cpp import * # noqa: F403
+from .ast_csharp import * # noqa: F403
+from .ast_dart import * # noqa: F403
+from .ast_go import * # noqa: F403
+from .ast_java import * # noqa: F403
+from .ast_js import * # noqa: F403
+from .ast_lua import * # noqa: F403
+from .ast_nodes import * # noqa: F403
+from .ast_php import * # noqa: F403
+from .ast_python import * # noqa: F403
+from .ast_rust import * # noqa: F403
+from .ast_scala import * # noqa: F403
+from .build import * # noqa: F403
+from .cli import * # noqa: F403
+from .core import * # noqa: F403
+from .deadcode_roots import * # noqa: F403
+from .deps import * # noqa: F403
+from .fqn_specs import * # noqa: F403
+from .graph import * # noqa: F403
+from .graph import _NODE_LABEL_UNIQUE_KEYS as _NODE_LABEL_UNIQUE_KEYS
+from .health import * # noqa: F403
+from .lang_tooling import * # noqa: F403
+from .languages import * # noqa: F403
+from .mcp import * # noqa: F403
+from .providers import * # noqa: F403
+from .security import * # noqa: F403
+from .stdlib_types import * # noqa: F403
+from .structural import * # noqa: F403
diff --git a/codebase_rag/constants/ast_cpp.py b/codebase_rag/constants/ast_cpp.py
new file mode 100644
index 000000000..ad65d4406
--- /dev/null
+++ b/codebase_rag/constants/ast_cpp.py
@@ -0,0 +1,269 @@
+# C/C++ tree-sitter node types, module markers, and operator maps.
+
+from enum import StrEnum
+
+from .ast_nodes import TS_ENUM_SPECIFIER, TS_STRUCT_SPECIFIER, TS_UNION_SPECIFIER
+
+
+class CppNodeType(StrEnum):
+ TRANSLATION_UNIT = "translation_unit"
+ NAMESPACE_DEFINITION = "namespace_definition"
+ NAMESPACE_IDENTIFIER = "namespace_identifier"
+ IDENTIFIER = "identifier"
+ EXPORT = "export"
+ EXPORT_KEYWORD = "export_keyword"
+ PRIMITIVE_TYPE = "primitive_type"
+ DECLARATION = "declaration"
+ FUNCTION_DEFINITION = "function_definition"
+ TEMPLATE_DECLARATION = "template_declaration"
+ CLASS_SPECIFIER = "class_specifier"
+ FUNCTION_DECLARATOR = "function_declarator"
+ POINTER_DECLARATOR = "pointer_declarator"
+ REFERENCE_DECLARATOR = "reference_declarator"
+ # An attribute MACRO before a definition (`JSON_HEDLEY_NON_NULL(3)
+ # bool sax_parse(...)`) parses as a parenthesized_declarator wrapping
+ # an ERROR plus the real function_declarator; the name walk descends it.
+ PARENTHESIZED_DECLARATOR = "parenthesized_declarator"
+ FIELD_DECLARATION = "field_declaration"
+ FIELD_IDENTIFIER = "field_identifier"
+ FIELD_INITIALIZER_LIST = "field_initializer_list"
+ FIELD_INITIALIZER = "field_initializer"
+ TEMPLATE_METHOD = "template_method"
+ QUALIFIED_IDENTIFIER = "qualified_identifier"
+ OPERATOR_NAME = "operator_name"
+ DESTRUCTOR_NAME = "destructor_name"
+ CONSTRUCTOR_OR_DESTRUCTOR_DEFINITION = "constructor_or_destructor_definition"
+ CONSTRUCTOR_OR_DESTRUCTOR_DECLARATION = "constructor_or_destructor_declaration"
+ INLINE_METHOD_DEFINITION = "inline_method_definition"
+ OPERATOR_CAST_DEFINITION = "operator_cast_definition"
+ TYPE_IDENTIFIER = "type_identifier"
+ PARAMETER_LIST = "parameter_list"
+ PARAMETER_DECLARATION = "parameter_declaration"
+ OPTIONAL_PARAMETER_DECLARATION = "optional_parameter_declaration"
+ INIT_DECLARATOR = "init_declarator"
+ TEMPLATE_TYPE = "template_type"
+ FIELD_EXPRESSION = "field_expression"
+ COMPOUND_STATEMENT = "compound_statement"
+ THIS = "this"
+ TYPE_DEFINITION = "type_definition"
+ ALIAS_DECLARATION = "alias_declaration"
+ TYPE_DESCRIPTOR = "type_descriptor"
+
+
+CPP_MODULE_EXTENSIONS = (".ixx", ".cppm", ".ccm", ".mxx")
+CPP_MODULE_PATH_MARKERS = frozenset({"interfaces", "modules"})
+
+# C++ module declaration prefixes
+CPP_EXPORT_MODULE_PREFIX = "export module "
+CPP_MODULE_PREFIX = "module "
+CPP_MODULE_PRIVATE_PREFIX = "module ;"
+CPP_IMPL_SUFFIX = "_impl"
+
+# C++ module type values
+CPP_MODULE_TYPE_INTERFACE = "interface"
+CPP_MODULE_TYPE_IMPLEMENTATION = "implementation"
+
+# C++ export prefixes for class detection
+CPP_EXPORT_CLASS_PREFIX = "export class "
+CPP_EXPORT_STRUCT_PREFIX = "export struct "
+CPP_EXPORT_UNION_PREFIX = "export union "
+CPP_EXPORT_TEMPLATE_PREFIX = "export template"
+CPP_EXPORT_PREFIXES = (
+ CPP_EXPORT_CLASS_PREFIX,
+ CPP_EXPORT_STRUCT_PREFIX,
+ CPP_EXPORT_UNION_PREFIX,
+ CPP_EXPORT_TEMPLATE_PREFIX,
+)
+
+# C++ keywords for class detection
+CPP_KEYWORD_CLASS = "class"
+CPP_KEYWORD_STRUCT = "struct"
+# `static` storage on a declaration: internal linkage, TU-local symbol.
+CPP_KEYWORD_STATIC = "static"
+TS_CPP_STORAGE_CLASS_SPECIFIER = "storage_class_specifier"
+CPP_EXPORTED_CLASS_KEYWORDS = frozenset({CPP_KEYWORD_CLASS, CPP_KEYWORD_STRUCT})
+
+# A C/C++ class/struct/union tag with no body is a forward declaration
+# (`class Widget;`); making it its own node collides with the real
+# definition's qn and fragments the class into same-named nodes.
+CPP_TYPE_SPECIFIER_NODE_TYPES = frozenset(
+ {"class_specifier", "struct_specifier", "union_specifier"}
+)
+
+CPP_FALLBACK_OPERATOR = "operator_unknown"
+CPP_FALLBACK_DESTRUCTOR = "~destructor"
+CPP_OPERATOR_TEXT_PREFIX = "operator"
+CPP_DESTRUCTOR_PREFIX = "~"
+
+CPP_OPERATOR_SYMBOL_MAP: dict[str, str] = {
+ "+": "operator_plus",
+ "-": "operator_minus",
+ "*": "operator_multiply",
+ "/": "operator_divide",
+ "%": "operator_modulo",
+ "=": "operator_assign",
+ "==": "operator_equal",
+ "!=": "operator_not_equal",
+ "<": "operator_less",
+ ">": "operator_greater",
+ "<=": "operator_less_equal",
+ ">=": "operator_greater_equal",
+ "&&": "operator_logical_and",
+ "||": "operator_logical_or",
+ "&": "operator_bitwise_and",
+ "|": "operator_bitwise_or",
+ "^": "operator_bitwise_xor",
+ "~": "operator_bitwise_not",
+ "!": "operator_not",
+ "<<": "operator_left_shift",
+ ">>": "operator_right_shift",
+ "++": "operator_increment",
+ "--": "operator_decrement",
+ "+=": "operator_plus_assign",
+ "-=": "operator_minus_assign",
+ "*=": "operator_multiply_assign",
+ "/=": "operator_divide_assign",
+ "%=": "operator_modulo_assign",
+ "&=": "operator_and_assign",
+ "|=": "operator_or_assign",
+ "^=": "operator_xor_assign",
+ "<<=": "operator_left_shift_assign",
+ ">>=": "operator_right_shift_assign",
+ "[]": "operator_subscript",
+ "()": "operator_call",
+}
+
+# Tree-sitter C++ node types for language_spec
+TS_CPP_FUNCTION_DEFINITION = "function_definition"
+TS_CPP_DECLARATION = "declaration"
+TS_CPP_FIELD_DECLARATION = "field_declaration"
+TS_CPP_TEMPLATE_DECLARATION = "template_declaration"
+TS_CPP_TEMPLATE_PARAMETER_LIST = "template_parameter_list"
+# The template TYPE-parameter declaration node types. A value/non-type param
+# (`parameter_declaration`, e.g. `int N` / `MyEnum E`) and a template-template param
+# are deliberately excluded: their type name is a concrete type, not a stand-in that
+# a call receiver could be instantiated as, so it must not enter the template-param set.
+CPP_TYPE_PARAMETER_DECL_TYPES = frozenset(
+ {
+ "type_parameter_declaration",
+ "optional_type_parameter_declaration",
+ "variadic_type_parameter_declaration",
+ }
+)
+TS_CPP_LAMBDA_EXPRESSION = "lambda_expression"
+TS_CPP_TRANSLATION_UNIT = "translation_unit"
+TS_CPP_LINKAGE_SPECIFICATION = "linkage_specification"
+TS_CPP_CALL_EXPRESSION = "call_expression"
+TS_CPP_FIELD_EXPRESSION = "field_expression"
+TS_CPP_SUBSCRIPT_EXPRESSION = "subscript_expression"
+TS_CPP_NEW_EXPRESSION = "new_expression"
+TS_CPP_DELETE_EXPRESSION = "delete_expression"
+TS_CPP_BINARY_EXPRESSION = "binary_expression"
+TS_CPP_UNARY_EXPRESSION = "unary_expression"
+TS_CPP_UPDATE_EXPRESSION = "update_expression"
+TS_CPP_FUNCTION_DECLARATOR = "function_declarator"
+# Substring shared by C++ declarator node types (pointer_declarator,
+# reference_declarator, ...), used to unwrap a parameter declarator down
+# to its bound identifier.
+CPP_DECLARATOR_SUFFIX = "declarator"
+
+FIELD_OPERATOR = "operator"
+FIELD_MACRO = "macro"
+
+# C++ I/O direct-sink walk node types (issue #714). call_expression keeps a
+# `function` field so call_name works unchanged. The stdout write path is the
+# stream-insertion operator `std::cout << x` -- a `binary_expression` with a `<<`
+# operator whose left-spine base is cout/cerr (no call node), handled via
+# stream_sink_type like Rust's macro sinks. A string_literal wraps string_content;
+# `compound_statement` is the block scope; `declaration` holds init_declarator
+# locals whose bound name is nested under the `declarator` field.
+TS_CPP_STRING_LITERAL = "string_literal"
+TS_CPP_STRING_CONTENT = "string_content"
+TS_CPP_COMPOUND_STATEMENT = "compound_statement"
+# `if (int x = 1)` / `switch (int q = f())`: the condition declaration nests
+# inside this wrapper, one level below the statement node.
+TS_CPP_CONDITION_CLAUSE = "condition_clause"
+# A lambda's parameter list hangs off this declarator (no name to declare).
+TS_CPP_ABSTRACT_FUNCTION_DECLARATOR = "abstract_function_declarator"
+TS_CPP_DECLARATION = "declaration"
+TS_CPP_INIT_DECLARATOR = "init_declarator"
+TS_CPP_PARAMETER_DECLARATION = "parameter_declaration"
+TS_CPP_IDENTIFIER = "identifier"
+TS_CPP_QUALIFIED_IDENTIFIER = "qualified_identifier"
+# `Reader(...)` as a call target: the callee wraps name + template args.
+TS_CPP_TEMPLATE_FUNCTION = "template_function"
+# `return {args};` -- a braced construction of the declared return type.
+TS_CPP_INITIALIZER_LIST = "initializer_list"
+# Stream-insertion operator; a `binary_expression` using it whose left-spine base
+# is std::cout / std::cerr writes STDOUT.
+CPP_OP_LEFT_SHIFT = "<<"
+# Stream-extraction operator; on a bound fstream handle (`in >> word`) it is a
+# READ of that handle's resource (issue #714).
+CPP_OP_RIGHT_SHIFT = ">>"
+TS_CPP_FOR_RANGE_LOOP = "for_range_loop"
+# Switch family: cases may fall through; a default arm is a
+# case_statement without a `value` field.
+TS_CPP_SWITCH_STATEMENT = "switch_statement"
+TS_CPP_CASE_STATEMENT = "case_statement"
+# field_expression = `obj.field` (argument/field); subscript_expression =
+# `arr[i]` (argument/indices). Inert for C++ I/O, wired for shape correctness.
+CPP_FIELD_ARGUMENT = "argument"
+CPP_FIELD_FIELD = "field"
+CPP_FIELD_INDICES = "indices"
+
+# Derived node type tuples for class ingestion
+CPP_CLASS_TYPES = (CppNodeType.CLASS_SPECIFIER, TS_STRUCT_SPECIFIER)
+CPP_COMPOUND_TYPES = (*CPP_CLASS_TYPES, TS_UNION_SPECIFIER, TS_ENUM_SPECIFIER)
+# Node types that open their own variable scope; local-variable inference must
+# not descend, or a name in a lambda / nested function / local class body gets
+# attributed to the enclosing function's scope.
+CPP_NESTED_SCOPE_NODE_TYPES = frozenset(
+ (
+ TS_CPP_FUNCTION_DEFINITION,
+ TS_CPP_LAMBDA_EXPRESSION,
+ *CPP_COMPOUND_TYPES,
+ )
+)
+
+# Preprocessor conditional directive heads, matched at line start (C allows
+# whitespace around '#'). Drives the whole-file-ERROR parse recovery: a
+# conditional branch whose brace count does not balance (nlohmann's
+# `#ifdef __cpp_lib_byteswap ... else { #endif`) breaks tree-sitter, which
+# keeps every branch's tokens.
+CPP_PREPROC_CONDITIONAL_PATTERN = (
+ rb"^\s*#\s*(if|ifdef|ifndef|elif|elifdef|elifndef|else|endif)\b"
+)
+CPP_PREPROC_OPEN_DIRECTIVES = frozenset({b"if", b"ifdef", b"ifndef"})
+CPP_PREPROC_SPLIT_DIRECTIVES = frozenset({b"elif", b"elifdef", b"elifndef", b"else"})
+
+# Reserved keywords that error recovery can leave in declarator position
+# (nlohmann: a macro access-label followed by `const decltype(MACRO_)`
+# members parses as a function declaration NAMED decltype). None can ever
+# name a real C/C++ function or method, so extraction rejects them.
+CPP_RESERVED_DEF_NAMES = frozenset(
+ {
+ "decltype",
+ "sizeof",
+ "alignof",
+ "alignas",
+ "typeid",
+ "static_assert",
+ "noexcept",
+ "typename",
+ "template",
+ "requires",
+ "if",
+ "for",
+ "while",
+ "switch",
+ "return",
+ "catch",
+ }
+)
+
+# Lambda capture list: `[a]` holds identifiers, `[=]`/`[&]` a default
+# capture pulling every enclosing local into scope.
+TS_CPP_LAMBDA_CAPTURE_SPECIFIER = "lambda_capture_specifier"
+TS_CPP_LAMBDA_DEFAULT_CAPTURE = "lambda_default_capture"
+# `[name = expr]` init-capture: binds its leading identifier in the lambda.
+TS_CPP_LAMBDA_CAPTURE_INITIALIZER = "lambda_capture_initializer"
diff --git a/codebase_rag/constants/ast_csharp.py b/codebase_rag/constants/ast_csharp.py
new file mode 100644
index 000000000..d5c5baf86
--- /dev/null
+++ b/codebase_rag/constants/ast_csharp.py
@@ -0,0 +1,314 @@
+# C# tree-sitter node types and field names (tree-sitter-c-sharp).
+
+# Compilation unit is the file root, and a FQN scope so a file-scoped namespace
+# (a SIBLING of its declarations, not their ancestor) folds into every type's qn
+# via _csharp_get_name.
+TS_CSHARP_COMPILATION_UNIT = "compilation_unit"
+
+# Namespace forms: block `namespace N { ... }` nests declarations under a
+# declaration_list child (ordinary ancestor scope); file-scoped
+# `namespace N;` does not (handled via the compilation_unit shim).
+TS_CSHARP_NAMESPACE_DECLARATION = "namespace_declaration"
+TS_CSHARP_FILE_SCOPED_NAMESPACE_DECLARATION = "file_scoped_namespace_declaration"
+# A type/member body; a block namespace also nests its top-level types under
+# one. Used to tell a top-level type (default `internal`) from a nested type
+# or member (default `private`) for export detection.
+TS_CSHARP_DECLARATION_LIST = "declaration_list"
+TS_CSHARP_EXPLICIT_INTERFACE_SPECIFIER = "explicit_interface_specifier"
+
+# Type declarations -> Class nodes.
+TS_CSHARP_CLASS_DECLARATION = "class_declaration"
+TS_CSHARP_STRUCT_DECLARATION = "struct_declaration"
+# `record`, `record struct`, and `record class` all parse as
+# record_declaration (the struct/class kind is a keyword child), so there is
+# no separate record_struct_declaration node in tree-sitter-c-sharp 0.23.5.
+TS_CSHARP_RECORD_DECLARATION = "record_declaration"
+TS_CSHARP_INTERFACE_DECLARATION = "interface_declaration"
+TS_CSHARP_ENUM_DECLARATION = "enum_declaration"
+
+# A conditional-compilation block wrapping a declaration's attributes
+# (`#if SYMBOL [Attr] #endif`) parses as this node, which sits as the leading
+# child of the declaration -- so the declaration's start_point is the `#if`
+# directive line. The real first token (Roslyn's span start) is the
+# attribute_list nested inside it.
+TS_CSHARP_PREPROC_IF_IN_ATTR_LIST = "preproc_if_in_attribute_list"
+TS_CSHARP_ATTRIBUTE_LIST = "attribute_list"
+
+# Member declarations -> Function/Method nodes.
+TS_CSHARP_METHOD_DECLARATION = "method_declaration"
+TS_CSHARP_CONSTRUCTOR_DECLARATION = "constructor_declaration"
+TS_CSHARP_DESTRUCTOR_DECLARATION = "destructor_declaration"
+TS_CSHARP_LOCAL_FUNCTION_STATEMENT = "local_function_statement"
+TS_CSHARP_OPERATOR_DECLARATION = "operator_declaration"
+TS_CSHARP_CONVERSION_OPERATOR_DECLARATION = "conversion_operator_declaration"
+TS_CSHARP_PROPERTY_DECLARATION = "property_declaration"
+
+# The scopes a local function can be declared in (and be call-visible from):
+# a method/constructor body or an enclosing local function. Pins each local
+# function to its HOST so bare-name resolution honors C# scoping (a local fn in
+# one overload's body is not in scope in a sibling overload).
+CSHARP_LOCAL_FN_HOST_TYPES = frozenset(
+ {
+ TS_CSHARP_METHOD_DECLARATION,
+ TS_CSHARP_CONSTRUCTOR_DECLARATION,
+ TS_CSHARP_LOCAL_FUNCTION_STATEMENT,
+ }
+)
+
+# Members whose registered leaf name is synthesized (no usable `name` field,
+# or one that collides), routed through csharp.utils.synthesize_method_name.
+CSHARP_SYNTHESIZED_NAME_TYPES = frozenset(
+ {
+ TS_CSHARP_OPERATOR_DECLARATION,
+ TS_CSHARP_CONVERSION_OPERATOR_DECLARATION,
+ TS_CSHARP_DESTRUCTOR_DECLARATION,
+ }
+)
+
+# Declaration node types that are grammatically ONLY ever a type member -- C#
+# has no top-level method/constructor/operator/property (a real top-level
+# function is a local_function_statement, deliberately excluded). When a `#if`
+# split truncates a class_declaration node early, tree-sitter detaches the
+# following members into the namespace's declaration_list with no class
+# ancestor, so the generic is_method_node ancestor walk returns False and they
+# would be mislabelled module Functions. This set drives their recovery as
+# Methods (function_ingest), by grammar invariant.
+CSHARP_MEMBER_ONLY_TYPES = frozenset(
+ {
+ TS_CSHARP_METHOD_DECLARATION,
+ TS_CSHARP_CONSTRUCTOR_DECLARATION,
+ TS_CSHARP_DESTRUCTOR_DECLARATION,
+ TS_CSHARP_OPERATOR_DECLARATION,
+ TS_CSHARP_CONVERSION_OPERATOR_DECLARATION,
+ TS_CSHARP_PROPERTY_DECLARATION,
+ }
+)
+
+# Base spec: `class C : Base, IShape` / `enum E : byte`. A single base_list
+# lumps the base class and interfaces together (unlike Java's separate
+# clauses), so the split is heuristic. base_list is unique to C# among the
+# grammars, so its presence identifies a C# type node without a language arg.
+TS_CSHARP_BASE_LIST = "base_list"
+# A base type may be a bare `identifier`, a `generic_name` (`List` ->
+# strip type args to the identifier), a `qualified_name` (`System.Exception`),
+# a record positional base `primary_constructor_base_type` (`Animal(Name)`),
+# or a `predefined_type` (an enum's underlying integral type -> not a base).
+TS_CSHARP_GENERIC_NAME = "generic_name"
+TS_CSHARP_CAST_EXPRESSION = "cast_expression"
+TS_CSHARP_POSTFIX_UNARY_EXPRESSION = "postfix_unary_expression"
+TS_CSHARP_IMPLICIT_PARAMETER = "implicit_parameter"
+TS_CSHARP_FIELD_TYPE_PARAMETERS = "type_parameters"
+TS_CSHARP_TYPE_PARAMETER_LIST = "type_parameter_list"
+# The `base` receiver keyword parses as a bare "base" node in this
+# grammar version (not "base_expression").
+TS_CSHARP_BASE_EXPRESSION = "base"
+
+# object's virtual/universal members: a call to one of these on an UNTYPED
+# receiver resolves to System.Object (or a BCL override), so binding it by
+# bare name to whatever first-party override happens to exist fabricates
+# an edge (exposed by the semantic calls oracle on Polly's
+# hide-object-members regions).
+CSHARP_OBJECT_VIRTUALS = frozenset(
+ {
+ "Equals",
+ "GetHashCode",
+ "GetType",
+ "ToString",
+ "ReferenceEquals",
+ "MemberwiseClone",
+ }
+)
+TS_CSHARP_LAMBDA_EXPRESSION = "lambda_expression"
+TS_CSHARP_BLOCK = "block"
+TS_CSHARP_PRIMARY_CONSTRUCTOR_BASE_TYPE = "primary_constructor_base_type"
+TS_CSHARP_PREDEFINED_TYPE = "predefined_type"
+
+# A `modifier` child wraps a single keyword (`public`, `override`, `new`,
+# `virtual`). Override detection reads it to tell a real override
+# (`override`) from an explicit `new` hide (which must not become OVERRIDES).
+TS_CSHARP_MODIFIER = "modifier"
+TS_CSHARP_MODIFIER_OVERRIDE = "override"
+# A type split across files carries `partial` on every part; parts with the
+# same namespace-qualified name are one logical type, unified for member and
+# base resolution (see csharp_partial_groups).
+TS_CSHARP_MODIFIER_PARTIAL = "partial"
+
+# Visibility modifiers that make a type/member external API surface (seed
+# dead-code roots). `protected internal` is two separate modifier children.
+TS_CSHARP_MODIFIER_PUBLIC = "public"
+TS_CSHARP_MODIFIER_INTERNAL = "internal"
+TS_CSHARP_MODIFIER_PROTECTED = "protected"
+
+# Parameter shapes for the method-qn signature. Each `parameter` exposes a
+# `type` field; a `params object[]` tail is an unwrapped `array_type` child
+# of the parameter_list (grammar quirk), captured directly.
+TS_CSHARP_PARAMETER = "parameter"
+TS_CSHARP_ARRAY_TYPE = "array_type"
+TS_CSHARP_PARAMETER_LIST = "parameter_list"
+
+# Local/field declarations for type inference. A local is a variable_declaration
+# (type field + variable_declarator[s]); `var` makes the type field an
+# implicit_type, inferred from the initializer. A field_declaration wraps a
+# variable_declaration; a property_declaration exposes `type` and `name` directly.
+TS_CSHARP_VARIABLE_DECLARATION = "variable_declaration"
+TS_CSHARP_VARIABLE_DECLARATOR = "variable_declarator"
+TS_CSHARP_IMPLICIT_TYPE = "implicit_type"
+TS_CSHARP_FIELD_DECLARATION = "field_declaration"
+
+# Call forms. A member call `recv.Method(...)` is an invocation_expression
+# whose `function` field is a member_access_expression (`expression` receiver
+# + `name` method); `this` is its own node type; args are `argument` nodes.
+TS_CSHARP_INVOCATION_EXPRESSION = "invocation_expression"
+TS_CSHARP_OBJECT_CREATION_EXPRESSION = "object_creation_expression"
+# C# 9 target-typed `new()`: a distinct node with NO `type` field; the
+# constructed type comes from the enclosing declaration (issue #773).
+TS_CSHARP_IMPLICIT_OBJECT_CREATION_EXPRESSION = "implicit_object_creation_expression"
+TS_CSHARP_MEMBER_ACCESS_EXPRESSION = "member_access_expression"
+# A conditional call `recv?.Method(...)`: the invocation's `function` field
+# is a conditional_access_expression whose member_binding_expression child
+# carries the method name.
+TS_CSHARP_CONDITIONAL_ACCESS_EXPRESSION = "conditional_access_expression"
+TS_CSHARP_MEMBER_BINDING_EXPRESSION = "member_binding_expression"
+TS_CSHARP_FIELD_EXPRESSION = "expression"
+TS_CSHARP_THIS = "this"
+TS_CSHARP_ARGUMENT = "argument"
+# implicit_object_creation_expression (`new(...)`) exposes NO field names;
+# its argument_list is an unfielded named child, so argument parsing needs
+# a typed fallback where the `arguments` field lookup returns nothing.
+TS_CSHARP_ARGUMENT_LIST = "argument_list"
+
+# String literals: `"x"` is a string_literal wrapping a string_literal_content
+# token. Used by the I/O walk to read a sink's static path/key argument.
+TS_CSHARP_STRING_LITERAL = "string_literal"
+TS_CSHARP_STRING_LITERAL_CONTENT = "string_literal_content"
+# `map[k]` subscript; inert in the I/O walk (C# env access is a call, not a
+# member read) but wired into the descriptor for shape correctness.
+TS_CSHARP_ELEMENT_ACCESS_EXPRESSION = "element_access_expression"
+
+# Nested scopes that own their own locals; the variable-type walk stops at
+# these so a lambda/local-function local cannot leak into (or shadow) the
+# enclosing method's type map.
+TS_CSHARP_NESTED_SCOPE_TYPES = (
+ "lambda_expression",
+ "anonymous_method_expression",
+ "local_function_statement",
+)
+
+# Import form: `using System;`, `using X = Y;`, `global using System.Linq;`.
+TS_CSHARP_USING_DIRECTIVE = "using_directive"
+
+# The name node inside a using directive: a dotted `qualified_name` or a bare
+# `identifier` (both the imported path and, in the alias form, the alias).
+TS_CSHARP_QUALIFIED_NAME = "qualified_name"
+TS_CSHARP_IDENTIFIER = "identifier"
+
+# Expression body `=> expr` on methods, properties, and accessors.
+TS_CSHARP_ARROW_EXPRESSION_CLAUSE = "arrow_expression_clause"
+# `public T this[int i] { ... }`: return-typed via `type` like a property.
+TS_CSHARP_INDEXER_DECLARATION = "indexer_declaration"
+# Some grammar versions wrap a declarator's `= value` initializer in an
+# equals_value_clause node (the pinned grammar hangs the value directly off
+# the declarator); the target-typed-new walk skips it either way, mirroring
+# the initializer search in csharp/type_inference.py.
+TS_CSHARP_EQUALS_VALUE_CLAUSE = "equals_value_clause"
+
+# Field names used with child_by_field_name.
+TS_CSHARP_FIELD_NAME = "name"
+TS_CSHARP_FIELD_OPERATOR = "operator"
+TS_CSHARP_FIELD_TYPE = "type"
+# method_declaration/local_function_statement expose the return type via
+# `returns` (there is no `type` field on them).
+TS_CSHARP_FIELD_RETURNS = "returns"
+
+# Operator/conversion-operator declarations expose no `name` field; a stable
+# synthetic name is built from these prefixes so the node still gets a qn.
+TS_CSHARP_OPERATOR_NAME_PREFIX = "operator_"
+TS_CSHARP_DESTRUCTOR_NAME_PREFIX = "~"
+
+# C# reserved keywords can never be identifiers, so a member/local-function
+# whose `name` field is one is a parse-recovery artifact -- e.g. a `#if`
+# directive splitting an if/else chain mid-method makes tree-sitter recover
+# the trailing `else if (...)` as a local_function_statement named `if`. Drop
+# those instead of emitting bogus Function nodes. Contextual keywords (`record`,
+# `async`, `var`, ...) ARE valid identifiers, so only the reserved set is here.
+CSHARP_RESERVED_KEYWORDS = frozenset(
+ {
+ "abstract",
+ "as",
+ "base",
+ "bool",
+ "break",
+ "byte",
+ "case",
+ "catch",
+ "char",
+ "checked",
+ "class",
+ "const",
+ "continue",
+ "decimal",
+ "default",
+ "delegate",
+ "do",
+ "double",
+ "else",
+ "enum",
+ "event",
+ "explicit",
+ "extern",
+ "false",
+ "finally",
+ "fixed",
+ "float",
+ "for",
+ "foreach",
+ "goto",
+ "if",
+ "implicit",
+ "in",
+ "int",
+ "interface",
+ "internal",
+ "is",
+ "lock",
+ "long",
+ "namespace",
+ "new",
+ "null",
+ "object",
+ "operator",
+ "out",
+ "override",
+ "params",
+ "private",
+ "protected",
+ "public",
+ "readonly",
+ "ref",
+ "return",
+ "sbyte",
+ "sealed",
+ "short",
+ "sizeof",
+ "stackalloc",
+ "static",
+ "string",
+ "struct",
+ "switch",
+ "this",
+ "throw",
+ "true",
+ "try",
+ "typeof",
+ "uint",
+ "ulong",
+ "unchecked",
+ "unsafe",
+ "ushort",
+ "using",
+ "virtual",
+ "void",
+ "volatile",
+ "while",
+ }
+)
diff --git a/codebase_rag/constants/ast_dart.py b/codebase_rag/constants/ast_dart.py
new file mode 100644
index 000000000..327467793
--- /dev/null
+++ b/codebase_rag/constants/ast_dart.py
@@ -0,0 +1,174 @@
+# Dart tree-sitter node types.
+# The tree-sitter-dart grammar splits a function/method into a `*_signature`
+# node and a sibling `function_body` (no single node spans both), and has no
+# call-expression node (calls are an identifier plus a following
+# `argument_part`/`selector`). cgr captures the signature nodes and derives
+# spans from the sibling body.
+
+# Call-site nodes: the grammar has no call-expression node; an invocation is
+# whatever selector chain precedes a `selector` holding an `argument_part`
+# (`f(x)` = identifier + selector(argument_part); `a.b()` = identifier +
+# selector(.b) + selector(argument_part)). A cascade (`obj..m()`) holds its
+# argument_part directly inside the `cascade_section`.
+TS_DART_SELECTOR = "selector"
+TS_DART_ARGUMENT_PART = "argument_part"
+# Inside an argument_part: `arguments` holds `argument` wrappers for
+# positional values and `named_argument` (label + expression) for named ones.
+TS_DART_ARGUMENTS = "arguments"
+TS_DART_ARGUMENT = "argument"
+TS_DART_NAMED_ARGUMENT = "named_argument"
+TS_DART_LABEL = "label"
+# First-class value containers in argument position: `handlers: [a, b]` and
+# `{...}` literals hold tear-offs one level down. Dart's ternary shares
+# Python's `conditional_expression` node TYPE but orders operands
+# [condition, consequence, alternative], not [body, condition, alternative].
+TS_DART_LIST_LITERAL = "list_literal"
+TS_DART_SET_OR_MAP_LITERAL = "set_or_map_literal"
+# Inside a set_or_map_literal: a MAP entry is a `pair` whose `value` field
+# holds the stored expression; `type_arguments` (`{...}`) carry
+# types, never values.
+TS_DART_PAIR = "pair"
+TS_DART_TYPE_ARGUMENTS = "type_arguments"
+TS_DART_CASCADE_SECTION = "cascade_section"
+TS_DART_CASCADE_SELECTOR = "cascade_selector"
+TS_DART_UNCONDITIONAL_ASSIGNABLE_SELECTOR = "unconditional_assignable_selector"
+TS_DART_CONDITIONAL_ASSIGNABLE_SELECTOR = "conditional_assignable_selector"
+TS_DART_THIS = "this"
+TS_DART_SUPER = "super"
+TS_DART_IDENTIFIER = "identifier"
+
+DART_CALL_QUERY = """
+(selector (argument_part)) @call
+(cascade_section (argument_part)) @call
+"""
+
+# Declaration shapes for receiver typing: a class field is
+# declaration(type_identifier, initialized_identifier_list); a body local is
+# initialized_variable_definition (leading type_identifier declared, or
+# inferred_type plus construction initializer); a parameter is
+# formal_parameter(type_identifier, identifier).
+TS_DART_CLASS_BODY = "class_body"
+TS_DART_BLOCK = "block"
+# Local binders beyond plain declarations: a for-in loop variable is the
+# FIRST identifier of for_loop_parts (the second is the iterable), catch
+# parameters bind every identifier, and a pattern declaration binds the
+# identifiers inside its *_pattern subtree. Their enclosing statements bound
+# the shadow scope.
+TS_DART_FOR_STATEMENT = "for_statement"
+TS_DART_FOR_LOOP_PARTS = "for_loop_parts"
+TS_DART_TRY_STATEMENT = "try_statement"
+TS_DART_CATCH_PARAMETERS = "catch_parameters"
+TS_DART_PATTERN_VARIABLE_DECLARATION = "pattern_variable_declaration"
+DART_PATTERN_NODE_SUFFIX = "_pattern"
+# The grammar's flat operator nodes (additive_expression, if_null_expression,
+# logical_and_expression, ...) share this suffix; a low-precedence operator
+# swallows a trailing ternary as its LAST named child.
+DART_EXPRESSION_NODE_SUFFIX = "_expression"
+TS_DART_FUNCTION_EXPRESSION = "function_expression"
+TS_DART_LOCAL_FUNCTION_DECLARATION = "local_function_declaration"
+
+# Nodes opening their OWN variable scope: the local-type walk must not descend,
+# or a nested function's same-named local conflict-drops the outer binding.
+DART_NESTED_SCOPE_NODE_TYPES = frozenset(
+ {
+ TS_DART_FUNCTION_EXPRESSION,
+ TS_DART_LOCAL_FUNCTION_DECLARATION,
+ }
+)
+TS_DART_INITIALIZED_IDENTIFIER_LIST = "initialized_identifier_list"
+TS_DART_INITIALIZED_IDENTIFIER = "initialized_identifier"
+TS_DART_INITIALIZED_VARIABLE_DEFINITION = "initialized_variable_definition"
+TS_DART_FORMAL_PARAMETER = "formal_parameter"
+
+# Type/class-like declarations (all captured as @class)
+TS_DART_CLASS_DEFINITION = "class_definition"
+TS_DART_MIXIN_DECLARATION = "mixin_declaration"
+TS_DART_ENUM_DECLARATION = "enum_declaration"
+TS_DART_EXTENSION_DECLARATION = "extension_declaration"
+TS_DART_EXTENSION_TYPE_DECLARATION = "extension_type_declaration"
+
+# Dart privacy is lexical: a leading underscore marks a library-private
+# symbol; every other name is public. Export detection walks the enclosing
+# type chain, so a public member of a private type is still unreachable.
+DART_PRIVATE_PREFIX = "_"
+DART_TYPE_DECLARATION_NODE_TYPES = frozenset(
+ {
+ TS_DART_CLASS_DEFINITION,
+ TS_DART_MIXIN_DECLARATION,
+ TS_DART_ENUM_DECLARATION,
+ TS_DART_EXTENSION_DECLARATION,
+ TS_DART_EXTENSION_TYPE_DECLARATION,
+ }
+)
+
+# Function/method signature nodes (all captured as @function)
+TS_DART_FUNCTION_SIGNATURE = "function_signature"
+TS_DART_GETTER_SIGNATURE = "getter_signature"
+TS_DART_SETTER_SIGNATURE = "setter_signature"
+TS_DART_FACTORY_CONSTRUCTOR_SIGNATURE = "factory_constructor_signature"
+TS_DART_CONSTRUCTOR_SIGNATURE = "constructor_signature"
+TS_DART_CONSTANT_CONSTRUCTOR_SIGNATURE = "constant_constructor_signature"
+
+# Wrappers whose sibling `function_body` completes a captured signature's span:
+# `method_signature` wraps class members, `declaration` wraps constructors; a
+# signature under either takes the wrapper's following `function_body` sibling.
+TS_DART_METHOD_SIGNATURE = "method_signature"
+TS_DART_DECLARATION = "declaration"
+TS_DART_FUNCTION_BODY = "function_body"
+
+# `@override`-style metadata: a preceding SIBLING of the (wrapped)
+# signature, not a child, so the highlights walk collects it explicitly.
+TS_DART_ANNOTATION = "annotation"
+# The decorator string as captured on Method nodes; when the enclosing class
+# has an external base, a method carrying it is framework-invoked and roots
+# the dead-code walk.
+DART_OVERRIDE_ANNOTATION = "@override"
+
+# Module and import/directive nodes
+TS_DART_PROGRAM = "program"
+TS_DART_IMPORT_OR_EXPORT = "import_or_export"
+TS_DART_PART_DIRECTIVE = "part_directive"
+TS_DART_PART_OF_DIRECTIVE = "part_of_directive"
+TS_DART_URI = "uri"
+TS_DART_IDENTIFIER_LIST = "dotted_identifier_list"
+
+# Inheritance clause nodes: `extends A`, `with M`, `implements I`, `on T`.
+TS_DART_SUPERCLASS = "superclass"
+TS_DART_MIXINS = "mixins"
+TS_DART_INTERFACES = "interfaces"
+TS_DART_TYPE_IDENTIFIER = "type_identifier"
+
+# `import '...' as name;` alias
+TS_DART_IMPORT_SPECIFICATION = "import_specification"
+DART_IMPORT_ALIAS_KEYWORD = "as"
+
+# URI scheme prefixes distinguishing external (dart:/package:) from
+# first-party (relative path) imports.
+DART_SCHEME_DART = "dart:"
+DART_SCHEME_PACKAGE = "package:"
+DART_QUOTE_CHARS = "'\""
+DART_EXT = ".dart"
+
+# Node types whose captured signature needs the sibling-body span fix.
+DART_SIGNATURE_TYPES = frozenset(
+ {
+ TS_DART_FUNCTION_SIGNATURE,
+ TS_DART_GETTER_SIGNATURE,
+ TS_DART_SETTER_SIGNATURE,
+ TS_DART_FACTORY_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTANT_CONSTRUCTOR_SIGNATURE,
+ }
+)
+DART_SIGNATURE_WRAPPERS = frozenset({TS_DART_METHOD_SIGNATURE, TS_DART_DECLARATION})
+
+# Constructor signatures whose grammar `name` field is the CLASS identifier,
+# not the declared name: `C.named` must take its LAST bare identifier or every
+# named constructor collapses into a duplicate of the default one.
+DART_CONSTRUCTOR_SIGNATURE_TYPES = frozenset(
+ {
+ TS_DART_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTANT_CONSTRUCTOR_SIGNATURE,
+ TS_DART_FACTORY_CONSTRUCTOR_SIGNATURE,
+ }
+)
diff --git a/codebase_rag/constants/ast_go.py b/codebase_rag/constants/ast_go.py
new file mode 100644
index 000000000..4938393af
--- /dev/null
+++ b/codebase_rag/constants/ast_go.py
@@ -0,0 +1,85 @@
+# Go tree-sitter node types.
+
+TS_GO_TYPE_DECLARATION = "type_declaration"
+TS_GO_TYPE_SPEC = "type_spec"
+TS_GO_TYPE_ALIAS = "type_alias"
+TS_GO_STRUCT_TYPE = "struct_type"
+TS_GO_SELECTOR_EXPRESSION = "selector_expression"
+TS_GO_TYPE_ASSERTION_EXPRESSION = "type_assertion_expression"
+# A Go source file ending in `_test.go` is compiled only under `go test`; its
+# module-qn file segment ends with this suffix. It may declare an EXTERNAL test
+# package (`package p_test`) that shares a directory with `package p` but is
+# distinct, so it must be excluded from same-directory fan-out.
+GO_TEST_FILE_SUFFIX = "_test"
+TS_GO_FIELD_DECLARATION_LIST = "field_declaration_list"
+TS_GO_FIELD_DECLARATION = "field_declaration"
+TS_GO_FIELD_IDENTIFIER = "field_identifier"
+TS_GO_INTERFACE_TYPE = "interface_type"
+TS_GO_PARAMETER_DECLARATION = "parameter_declaration"
+TS_GO_FUNC_LITERAL = "func_literal"
+TS_GO_SOURCE_FILE = "source_file"
+TS_GO_FUNCTION_DECLARATION = "function_declaration"
+TS_GO_METHOD_DECLARATION = "method_declaration"
+TS_GO_CALL_EXPRESSION = "call_expression"
+TS_GO_IMPORT_DECLARATION = "import_declaration"
+TS_GO_PARAMETER_LIST = "parameter_list"
+TS_GO_VAR_DECLARATION = "var_declaration"
+TS_GO_VAR_SPEC = "var_spec"
+TS_GO_SHORT_VAR_DECLARATION = "short_var_declaration"
+TS_GO_ASSIGNMENT_STATEMENT = "assignment_statement"
+# I/O detection (issue #714): a function body is a `block`; string arguments are
+# `interpreted_string_literal` (double-quoted) whose text lives in a
+# `interpreted_string_literal_content` child; `index_expression`/operand+field are
+# the selector/subscript node shapes (only used by member-access reads, which Go
+# has none of, so they are inert placeholders here).
+TS_GO_IDENTIFIER = "identifier"
+TS_GO_CONST_SPEC = "const_spec"
+TS_GO_RANGE_CLAUSE = "range_clause"
+# The init;cond;post header of a C-style Go for; its post statement lives in
+# an `update` field.
+TS_GO_FOR_CLAUSE = "for_clause"
+# Switch family: arms are EXCLUSIVE (Go has no implicit fallthrough).
+TS_GO_EXPRESSION_SWITCH_STATEMENT = "expression_switch_statement"
+TS_GO_TYPE_SWITCH_STATEMENT = "type_switch_statement"
+TS_GO_SELECT_STATEMENT = "select_statement"
+TS_GO_EXPRESSION_CASE = "expression_case"
+TS_GO_TYPE_CASE = "type_case"
+TS_GO_COMMUNICATION_CASE = "communication_case"
+TS_GO_DEFAULT_CASE = "default_case"
+# Legal only as an arm's LAST statement; transfers into the next case.
+TS_GO_FALLTHROUGH_STATEMENT = "fallthrough_statement"
+TS_GO_STATEMENT_LIST = "statement_list"
+TS_GO_BLOCK = "block"
+# Go wraps a block's statements in a single `statement_list` node (unlike JS/Java);
+# the source-order I/O walk unwraps it so per-statement shadowing sees the real
+# statement boundaries.
+TS_GO_STATEMENT_LIST = "statement_list"
+TS_GO_INTERPRETED_STRING = "interpreted_string_literal"
+TS_GO_INTERPRETED_STRING_CONTENT = "interpreted_string_literal_content"
+TS_GO_RAW_STRING = "raw_string_literal"
+TS_GO_RAW_STRING_CONTENT = "raw_string_literal_content"
+TS_GO_DOT = "dot"
+TS_GO_INDEX_EXPRESSION = "index_expression"
+TS_GO_FIELD_OPERAND = "operand"
+TS_GO_FIELD_FIELD = "field"
+TS_GO_FIELD_INDEX = "index"
+TS_GO_EXPRESSION_LIST = "expression_list"
+TS_GO_COMPOSITE_LITERAL = "composite_literal"
+TS_GO_LITERAL_VALUE = "literal_value"
+TS_GO_KEYED_ELEMENT = "keyed_element"
+TS_GO_LITERAL_ELEMENT = "literal_element"
+TS_GO_UNARY_EXPRESSION = "unary_expression"
+# `[]byte(s)` / `string(b)`: value-preserving conversion, unwrapped by the
+# lean flow walk so the operand's taint carries through (issue #714).
+TS_GO_TYPE_CONVERSION_EXPRESSION = "type_conversion_expression"
+TS_GO_POINTER_TYPE = "pointer_type"
+# `pkg.Type` in a signature; kept as dotted text so a binding typed to an
+# external package's type stays TYPED (and drops) instead of trie-guessed.
+TS_GO_QUALIFIED_TYPE = "qualified_type"
+# Go composite types a method may return; a chained call lands on the CONTAINER,
+# not its element, so return-type inference must not unwrap these (a `[]Command`
+# return must not resolve `.Run()` to `Command.Run`).
+TS_GO_CONTAINER_TYPES: frozenset[str] = frozenset(
+ {"slice_type", "array_type", "map_type", "channel_type", "function_type"}
+)
+FIELD_OPERAND = "operand"
diff --git a/codebase_rag/constants/ast_java.py b/codebase_rag/constants/ast_java.py
new file mode 100644
index 000000000..07284c115
--- /dev/null
+++ b/codebase_rag/constants/ast_java.py
@@ -0,0 +1,240 @@
+# Java tree-sitter node types, modifiers, and JVM layout constants.
+
+from .ast_nodes import (
+ TS_CLASS_DECLARATION,
+ TS_ENUM_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+)
+from .core import ENTITY_FUNCTION, ENTITY_METHOD
+
+# Tree-sitter Java node types for language_spec
+TS_JAVA_METHOD_INVOCATION = "method_invocation"
+TS_JAVA_ANNOTATION_TYPE_DECLARATION = "annotation_type_declaration"
+
+# Java interface `extends A, B` clause (tree-sitter-java); holds a type_list.
+TS_JAVA_EXTENDS_INTERFACES = "extends_interfaces"
+
+TS_JAVA_CAST_EXPRESSION = "cast_expression"
+
+# Java tree-sitter node types
+TS_FORMAL_PARAMETER = "formal_parameter"
+TS_SPREAD_PARAMETER = "spread_parameter"
+TS_LOCAL_VARIABLE_DECLARATION = "local_variable_declaration"
+TS_FIELD_DECLARATION = "field_declaration"
+TS_ASSIGNMENT_EXPRESSION = "assignment_expression"
+
+TS_OBJECT_CREATION_EXPRESSION = "object_creation_expression"
+TS_METHOD_INVOCATION = "method_invocation"
+TS_FIELD_ACCESS = "field_access"
+TS_INTEGER_LITERAL = "integer_literal"
+TS_DECIMAL_FLOATING_POINT_LITERAL = "decimal_floating_point_literal"
+TS_ARRAY_CREATION_EXPRESSION = "array_creation_expression"
+TS_METHOD_DECLARATION = "method_declaration"
+TS_ENHANCED_FOR_STATEMENT = "enhanced_for_statement"
+# Switch family: colon groups may fall through, arrow rules are exclusive;
+# a default arm is a switch_label with no named children.
+TS_JAVA_SWITCH_EXPRESSION = "switch_expression"
+TS_JAVA_SWITCH_RULE = "switch_rule"
+TS_JAVA_SWITCH_BLOCK_STATEMENT_GROUP = "switch_block_statement_group"
+TS_JAVA_SWITCH_LABEL = "switch_label"
+TS_TRY_WITH_RESOURCES_STATEMENT = "try_with_resources_statement"
+# One declaration inside a try-with-resources header; binds via `name`/`value`
+# fields exactly like a variable_declarator.
+TS_JAVA_RESOURCE = "resource"
+TS_RECORD_DECLARATION = "record_declaration"
+TS_TRUE = "true"
+TS_FALSE = "false"
+
+# Java I/O direct-sink walk node types (issue #714). string_literal wraps a
+# `string_fragment` (shared with JS); `block` is the method-body lexical scope;
+# `lambda_expression` is a nested scope pruned from the enclosing walk. field_access
+# / array_access describe member/subscript access (inert for Java, which has no
+# IO_MEMBER_READS entry -- Java env access is a call, System.getenv, not a member).
+TS_JAVA_STRING_LITERAL = "string_literal"
+TS_JAVA_BLOCK = "block"
+TS_JAVA_LAMBDA_EXPRESSION = "lambda_expression"
+TS_JAVA_ARRAY_ACCESS = "array_access"
+JAVA_FIELD_FIELD = "field"
+JAVA_FIELD_INDEX = "index"
+
+# Tree-sitter field names for child_by_field_name
+TS_FIELD_NAME = "name"
+TS_FIELD_TYPE = "type"
+TS_SCOPED_TYPE_IDENTIFIER = "scoped_type_identifier"
+TS_FIELD_SUPERCLASS = "superclass"
+TS_FIELD_INTERFACES = "interfaces"
+TS_FIELD_TYPE_PARAMETERS = "type_parameters"
+TS_FIELD_PARAMETERS = "parameters"
+TS_FIELD_DECLARATOR = "declarator"
+TS_FIELD_OBJECT = "object"
+TS_FIELD_ARGUMENTS = "arguments"
+TS_FIELD_FUNCTION = "function"
+TS_FIELD_BODY = "body"
+TS_FIELD_LEFT = "left"
+TS_FIELD_RIGHT = "right"
+
+QUERY_CAPTURE_CLASS = "class"
+QUERY_CAPTURE_FUNCTION = "function"
+QUERY_KEY_CLASSES = "classes"
+QUERY_KEY_FUNCTIONS = "functions"
+
+# Java type inference keywords
+JAVA_KEYWORD_THIS = "this"
+JAVA_KEYWORD_SUPER = "super"
+
+# Java heuristic patterns
+JAVA_GETTER_PATTERN = "get"
+JAVA_NAME_PATTERN = "name"
+JAVA_ID_PATTERN = "id"
+JAVA_SIZE_PATTERN = "size"
+JAVA_LENGTH_PATTERN = "length"
+JAVA_CREATE_PATTERN = "create"
+JAVA_NEW_PATTERN = "new"
+JAVA_IS_PATTERN = "is"
+JAVA_HAS_PATTERN = "has"
+JAVA_USER_PATTERN = "user"
+JAVA_ORDER_PATTERN = "order"
+
+# Java entity type names
+ENTITY_CONSTRUCTOR = "Constructor"
+
+# Java callable entity types for method resolution
+# FUNCTION is included so an unqualified call inside a method-body anonymous class
+# can reach the anon's OWN methods (registered as Function nodes under the enclosing
+# scope, e.g. gson's `delegate()` called by the same anon's `read()`); the module
+# scan is a last-resort fallback after precise class/anon-base/enclosing lookups.
+JAVA_CALLABLE_ENTITY_TYPES = frozenset(
+ {ENTITY_METHOD, ENTITY_CONSTRUCTOR, ENTITY_FUNCTION}
+)
+
+# Java primitive type names
+JAVA_TYPE_STRING = "String"
+JAVA_TYPE_INT = "int"
+JAVA_TYPE_DOUBLE = "double"
+JAVA_TYPE_BOOLEAN = "boolean"
+JAVA_TYPE_LONG = "java.lang.Long"
+JAVA_TYPE_STRING_FQN = "java.lang.String"
+JAVA_TYPE_OBJECT = "Object"
+
+# Java heuristic return type names
+JAVA_HEURISTIC_USER = "User"
+JAVA_HEURISTIC_ORDER = "Order"
+
+# Java tree-sitter node types for java_utils
+TS_PACKAGE_DECLARATION = "package_declaration"
+TS_ANNOTATION_TYPE_DECLARATION = "annotation_type_declaration"
+TS_CONSTRUCTOR_DECLARATION = "constructor_declaration"
+TS_ANNOTATION = "annotation"
+TS_MARKER_ANNOTATION = "marker_annotation"
+TS_GENERIC_TYPE = "generic_type"
+TS_TYPE_PARAMETER = "type_parameter"
+TS_MODIFIERS = "modifiers"
+TS_VOID_TYPE = "void_type"
+TS_PROGRAM = "program"
+TS_THIS = "this"
+TS_SUPER = "super"
+
+# Java modifier node types
+JAVA_MODIFIER_PUBLIC = "public"
+JAVA_MODIFIER_PRIVATE = "private"
+JAVA_MODIFIER_PROTECTED = "protected"
+JAVA_MODIFIER_STATIC = "static"
+JAVA_MODIFIER_FINAL = "final"
+JAVA_MODIFIER_ABSTRACT = "abstract"
+JAVA_MODIFIER_SYNCHRONIZED = "synchronized"
+JAVA_MODIFIER_TRANSIENT = "transient"
+JAVA_MODIFIER_VOLATILE = "volatile"
+
+JAVA_CLASS_MODIFIERS = frozenset(
+ {
+ JAVA_MODIFIER_PUBLIC,
+ JAVA_MODIFIER_PRIVATE,
+ JAVA_MODIFIER_PROTECTED,
+ JAVA_MODIFIER_STATIC,
+ JAVA_MODIFIER_FINAL,
+ JAVA_MODIFIER_ABSTRACT,
+ }
+)
+
+JAVA_METHOD_MODIFIERS = frozenset(
+ {
+ JAVA_MODIFIER_PUBLIC,
+ JAVA_MODIFIER_PRIVATE,
+ JAVA_MODIFIER_PROTECTED,
+ JAVA_MODIFIER_STATIC,
+ JAVA_MODIFIER_FINAL,
+ JAVA_MODIFIER_ABSTRACT,
+ JAVA_MODIFIER_SYNCHRONIZED,
+ }
+)
+
+JAVA_FIELD_MODIFIERS = frozenset(
+ {
+ JAVA_MODIFIER_PUBLIC,
+ JAVA_MODIFIER_PRIVATE,
+ JAVA_MODIFIER_PROTECTED,
+ JAVA_MODIFIER_STATIC,
+ JAVA_MODIFIER_FINAL,
+ JAVA_MODIFIER_TRANSIENT,
+ JAVA_MODIFIER_VOLATILE,
+ }
+)
+
+# Java visibility values
+JAVA_VISIBILITY_PUBLIC = "public"
+JAVA_VISIBILITY_PROTECTED = "protected"
+JAVA_VISIBILITY_PRIVATE = "private"
+JAVA_VISIBILITY_PACKAGE = "package"
+
+# Java class type suffixes and names
+JAVA_DECLARATION_SUFFIX = "_declaration"
+JAVA_TYPE_METHOD = "method"
+JAVA_TYPE_CONSTRUCTOR = "constructor"
+
+# Java class node types for matching
+JAVA_CLASS_NODE_TYPES = frozenset(
+ {
+ TS_CLASS_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+ TS_ENUM_DECLARATION,
+ TS_ANNOTATION_TYPE_DECLARATION,
+ TS_RECORD_DECLARATION,
+ }
+)
+
+# Java method node types
+JAVA_METHOD_NODE_TYPES = frozenset(
+ {
+ TS_METHOD_DECLARATION,
+ TS_CONSTRUCTOR_DECLARATION,
+ }
+)
+
+# Java main method constants
+JAVA_MAIN_METHOD_NAME = "main"
+JAVA_MAIN_PARAM_ARRAY = "String[]"
+JAVA_MAIN_PARAM_VARARGS = "String..."
+JAVA_MAIN_PARAM_TYPE = "String"
+
+# Java path parsing constants
+JAVA_PATH_JAVA = "java"
+JAVA_PATH_KOTLIN = "kotlin"
+JAVA_PATH_SCALA = "scala"
+JAVA_PATH_SRC = "src"
+JAVA_PATH_MAIN = "main"
+JAVA_PATH_TEST = "test"
+
+JAVA_JVM_LANGUAGES = frozenset(
+ {
+ JAVA_PATH_JAVA,
+ JAVA_PATH_KOTLIN,
+ JAVA_PATH_SCALA,
+ }
+)
+
+JAVA_SRC_FOLDERS = frozenset(
+ {
+ JAVA_PATH_MAIN,
+ JAVA_PATH_TEST,
+ }
+)
diff --git a/codebase_rag/constants/ast_js.py b/codebase_rag/constants/ast_js.py
new file mode 100644
index 000000000..4d9b10956
--- /dev/null
+++ b/codebase_rag/constants/ast_js.py
@@ -0,0 +1,289 @@
+# JavaScript/TypeScript tree-sitter node types, queries, and captures.
+
+from .ast_nodes import (
+ TS_CALL_EXPRESSION,
+ TS_IDENTIFIER,
+ TS_MEMBER_EXPRESSION,
+ TS_NEW_EXPRESSION,
+)
+
+# Locals query patterns for JS/TS
+JS_LOCALS_PATTERN = """
+; Variable definitions
+(variable_declarator name: (identifier) @local.definition)
+(function_declaration name: (identifier) @local.definition)
+(class_declaration name: (identifier) @local.definition)
+
+; Variable references
+(identifier) @local.reference
+"""
+
+TS_LOCALS_PATTERN = """
+; Variable definitions (TypeScript has multiple declaration types)
+(variable_declarator name: (identifier) @local.definition)
+(lexical_declaration (variable_declarator name: (identifier) @local.definition))
+(variable_declaration (variable_declarator name: (identifier) @local.definition))
+
+; Function definitions
+(function_declaration name: (identifier) @local.definition)
+
+; Class definitions (uses type_identifier for class names)
+(class_declaration name: (type_identifier) @local.definition)
+
+; Variable references
+(identifier) @local.reference
+"""
+
+# Receivers that address the MODULE itself in CommonJS code (`exports.render()`,
+# `module.exports.x()`, prototype-pattern `this`): only these bind a dotted call
+# to a same-module free function; `view.render()` is an instance call.
+JS_MODULE_RECEIVERS = frozenset({"exports", "module", "this"})
+# `this.` receiver prefix of a call name; a prototype-assigned function
+# (`Date.prototype.strftime`) dispatches such calls to a sibling method of
+# the same prototype target before the module-receiver fallback applies.
+JS_THIS_CALL_PREFIX = "this."
+
+JS_TS_PARENT_REF_TYPES = (TS_IDENTIFIER, TS_MEMBER_EXPRESSION)
+# JSX element nodes that carry a component name (javascript and tsx grammars
+# share these); the closing element repeats the name and must not double-emit.
+TS_JSX_SELF_CLOSING_ELEMENT = "jsx_self_closing_element"
+TS_JSX_OPENING_ELEMENT = "jsx_opening_element"
+# The `{...}` wrapper around an expression in a JSX attribute value or child
+# (`onClick={handleLogout}`, `onClick={() => x()}`); its inner expression can
+# hand a function to the element as a prop.
+TS_JSX_EXPRESSION = "jsx_expression"
+
+# TS "cast" wrappers transparent for reference resolution: `x as T`,
+# `x satisfies T`, and non-null `x!`. Their first named child is the wrapped
+# value, so unwrapping reaches the referenced expression
+# (`persistImpl as unknown as Persist`).
+TS_AS_EXPRESSION = "as_expression"
+TS_SATISFIES_EXPRESSION = "satisfies_expression"
+TS_NON_NULL_EXPRESSION = "non_null_expression"
+TS_CAST_WRAPPER_TYPES = frozenset(
+ {TS_AS_EXPRESSION, TS_SATISFIES_EXPRESSION, TS_NON_NULL_EXPRESSION}
+)
+
+# JS/TS ingest node types
+TS_PAIR = "pair"
+TS_OBJECT = "object"
+TS_TEMPLATE_STRING = "template_string"
+TS_TEMPLATE_SUBSTITUTION = "template_substitution"
+TS_ARRAY = "array"
+
+# When a variable_declarator's value is one of these, the variable binds the
+# call/construction RESULT, not a function, so an arrow inside its arguments
+# (`const m = useMutation({fn: () => {}})`) must not inherit the variable's
+# name; arrows nested under a const-bound object still take the object's name.
+JS_CALL_RESULT_VALUE_TYPES = frozenset({TS_CALL_EXPRESSION, TS_NEW_EXPRESSION})
+TS_FUNCTION_EXPRESSION = "function_expression"
+TS_ARROW_FUNCTION = "arrow_function"
+TS_REQUIRED_PARAMETER = "required_parameter"
+TS_OPTIONAL_PARAMETER = "optional_parameter"
+TS_ASSIGNMENT_PATTERN = "assignment_pattern"
+TS_JS_ASSIGNMENT_EXPRESSION = "assignment_expression"
+# `x += v` and friends: reads the old value AND writes the new one.
+TS_JS_AUGMENTED_ASSIGNMENT_EXPRESSION = "augmented_assignment_expression"
+# `x++` / `--x`: also a read-then-write; the operand is the `argument` field.
+TS_JS_UPDATE_EXPRESSION = "update_expression"
+TS_JS_FIELD_ARGUMENT = "argument"
+TS_FIELD_PATTERN = "pattern"
+TS_FIELD_PARAMETER = "parameter"
+TS_MODULE = "module"
+TS_CLASS_BODY = "class_body"
+
+TS_PROPERTY_IDENTIFIER = "property_identifier"
+
+# JS prototype property keywords
+JS_PROTOTYPE_KEYWORD = "prototype"
+JS_OBJECT_NAME = "Object"
+JS_CREATE_METHOD = "create"
+
+# JS/TS ingest query capture names
+CAPTURE_CHILD_CLASS = "child_class"
+CAPTURE_PARENT_CLASS = "parent_class"
+CAPTURE_CONSTRUCTOR_NAME = "constructor_name"
+CAPTURE_PROTOTYPE_KEYWORD = "prototype_keyword"
+CAPTURE_METHOD_NAME = "method_name"
+CAPTURE_METHOD_FUNCTION = "method_function"
+CAPTURE_MEMBER_EXPR = "member_expr"
+CAPTURE_FUNCTION_EXPR = "function_expr"
+CAPTURE_ARROW_FUNCTION = "arrow_function"
+
+# JS prototype inheritance query
+JS_PROTOTYPE_INHERITANCE_QUERY = """
+(assignment_expression
+ left: (member_expression
+ object: (identifier) @child_class
+ property: (property_identifier) @prototype (#eq? @prototype "prototype"))
+ right: (call_expression
+ function: (member_expression
+ object: (identifier) @object_name (#eq? @object_name "Object")
+ property: (property_identifier) @create_method (#eq? @create_method "create"))
+ arguments: (arguments
+ (member_expression
+ object: (identifier) @parent_class
+ property: (property_identifier) @parent_prototype (#eq? @parent_prototype "prototype")))))
+"""
+
+# JS prototype method assignment query
+JS_PROTOTYPE_METHOD_QUERY = """
+(assignment_expression
+ left: (member_expression
+ object: (member_expression
+ object: (identifier) @constructor_name
+ property: (property_identifier) @prototype_keyword (#eq? @prototype_keyword "prototype"))
+ property: (property_identifier) @method_name)
+ right: (function_expression) @method_function)
+"""
+
+# JS object method query
+JS_OBJECT_METHOD_QUERY = """
+(pair
+ key: (property_identifier) @method_name
+ value: (function_expression) @method_function)
+"""
+
+# JS method definition query
+JS_METHOD_DEF_QUERY = """
+(object
+ (method_definition
+ name: (property_identifier) @method_name) @method_function)
+"""
+
+# JS object arrow function query
+JS_OBJECT_ARROW_QUERY = """
+(object
+ (pair
+ (property_identifier) @method_name
+ (arrow_function) @arrow_function))
+"""
+
+# JS assignment arrow function query
+JS_ASSIGNMENT_ARROW_QUERY = """
+(assignment_expression
+ (member_expression) @member_expr
+ (arrow_function) @arrow_function)
+"""
+
+# JS assignment function expression query
+JS_ASSIGNMENT_FUNCTION_QUERY = """
+(assignment_expression
+ (member_expression) @member_expr
+ (function_expression) @function_expr)
+"""
+
+# JS/TS control-flow node types + fields for the path-sensitive taint walk
+# (issue #714 follow-up). Each if/else, loop, and try branch is evaluated against
+# a COPY of the incoming taint state and unioned at the merge, so taint surviving
+# on ANY path survives and a kill counts only when it happens on EVERY path. The
+# values coincide with the Python grammar's but stay JS-scoped per the per-language
+# constants convention.
+TS_JS_IF_STATEMENT = "if_statement"
+# Switch family: cases may fall through into the next case.
+TS_JS_SWITCH_STATEMENT = "switch_statement"
+TS_JS_SWITCH_CASE = "switch_case"
+TS_JS_SWITCH_DEFAULT = "switch_default"
+# `c ? a : b` (shared name with the Java grammar); C++ spells it
+# conditional_expression.
+TS_JS_TERNARY_EXPRESSION = "ternary_expression"
+# Short-circuit operators whose result IS one of the operands, so a
+# bind through them unions both operands' taints.
+JS_SHORT_CIRCUIT_OPERATORS: frozenset[str] = frozenset({"||", "??", "&&"})
+TS_JS_ELSE_CLAUSE = "else_clause"
+TS_JS_WHILE_STATEMENT = "while_statement"
+TS_JS_FOR_STATEMENT = "for_statement"
+TS_JS_FOR_IN_STATEMENT = "for_in_statement"
+TS_JS_TRY_STATEMENT = "try_statement"
+TS_JS_CATCH_CLAUSE = "catch_clause"
+TS_JS_FINALLY_CLAUSE = "finally_clause"
+FIELD_ALTERNATIVE = "alternative"
+FIELD_HANDLER = "handler"
+FIELD_FINALIZER = "finalizer"
+# The C-style `for (init; cond; increment)` update clause, which runs AFTER the
+# body each iteration, so taint the body carries into it reaches the next one.
+FIELD_INCREMENT = "increment"
+
+# JS/TS module system node types
+TS_OBJECT_PATTERN = "object_pattern"
+TS_ARRAY_PATTERN = "array_pattern"
+TS_REST_PATTERN = "rest_pattern"
+TS_SHORTHAND_PROPERTY_IDENTIFIER_PATTERN = "shorthand_property_identifier_pattern"
+TS_PAIR_PATTERN = "pair_pattern"
+# `process.env.X` is a member_expression; `process.env['X']` a subscript, used
+# to detect environment-variable reads (issue #714 process.env follow-up).
+TS_SUBSCRIPT_EXPRESSION = "subscript_expression"
+TS_FIELD_INDEX = "index"
+TS_FUNCTION_DECLARATION = "function_declaration"
+TS_GENERATOR_FUNCTION_DECLARATION = "generator_function_declaration"
+
+# Tree-sitter field names for module system
+FIELD_FUNCTION = "function"
+FIELD_KEY = "key"
+
+# JS/TS module system keywords
+JS_REQUIRE_KEYWORD = "require"
+JS_EXPORTS_KEYWORD = "exports"
+JS_MODULE_KEYWORD = "module"
+
+# JS/TS export type descriptions
+JS_EXPORT_TYPE_COMMONJS = "CommonJS Export"
+JS_EXPORT_TYPE_COMMONJS_MODULE = "CommonJS Module Export"
+JS_EXPORT_TYPE_ES6_FUNCTION = "ES6 Export Function"
+JS_EXPORT_TYPE_ES6_FUNCTION_DECL = "ES6 Export Function Declaration"
+
+# JS/TS CommonJS destructure query
+JS_COMMONJS_DESTRUCTURE_QUERY = """
+(lexical_declaration
+ (variable_declarator
+ name: (object_pattern)
+ value: (call_expression
+ function: (identifier) @func (#eq? @func "require")
+ )
+ ) @variable_declarator
+)
+"""
+
+# JS/TS CommonJS exports function query
+JS_COMMONJS_EXPORTS_FUNCTION_QUERY = """
+(assignment_expression
+ left: (member_expression
+ object: (identifier) @exports_obj
+ property: (property_identifier) @export_name)
+ right: [(function_expression) (arrow_function)] @export_function)
+"""
+
+# JS/TS CommonJS module.exports query
+JS_COMMONJS_MODULE_EXPORTS_QUERY = """
+(assignment_expression
+ left: (member_expression
+ object: (member_expression
+ object: (identifier) @module_obj
+ property: (property_identifier) @exports_prop)
+ property: (property_identifier) @export_name)
+ right: [(function_expression) (arrow_function)] @export_function)
+"""
+
+# JS/TS ES6 export const query
+JS_ES6_EXPORT_CONST_QUERY = """
+(export_statement
+ (lexical_declaration
+ (variable_declarator
+ name: (identifier) @export_name
+ value: [(function_expression) (arrow_function)] @export_function)))
+"""
+
+# JS/TS ES6 export function query
+JS_ES6_EXPORT_FUNCTION_QUERY = """
+(export_statement
+ [(function_declaration) (generator_function_declaration)] @export_function)
+"""
+
+# Query capture names for module system
+CAPTURE_FUNC = "func"
+CAPTURE_VARIABLE_DECLARATOR = "variable_declarator"
+CAPTURE_EXPORTS_OBJ = "exports_obj"
+CAPTURE_MODULE_OBJ = "module_obj"
+CAPTURE_EXPORTS_PROP = "exports_prop"
+CAPTURE_EXPORT_NAME = "export_name"
+CAPTURE_EXPORT_FUNCTION = "export_function"
diff --git a/codebase_rag/constants/ast_lua.py b/codebase_rag/constants/ast_lua.py
new file mode 100644
index 000000000..dd1f2e959
--- /dev/null
+++ b/codebase_rag/constants/ast_lua.py
@@ -0,0 +1,29 @@
+# Lua tree-sitter node types and string forms.
+
+from .ast_nodes import TS_STRING, TS_STRING_LITERAL
+
+LUA_STRING_TYPES = (TS_STRING, TS_STRING_LITERAL)
+
+TS_DOT_INDEX_EXPRESSION = "dot_index_expression"
+TS_LUA_VARIABLE_DECLARATION = "variable_declaration"
+TS_LUA_ASSIGNMENT_STATEMENT = "assignment_statement"
+TS_LUA_VARIABLE_LIST = "variable_list"
+TS_LUA_EXPRESSION_LIST = "expression_list"
+TS_LUA_FUNCTION_CALL = "function_call"
+TS_LUA_METHOD_INDEX_EXPRESSION = "method_index_expression"
+TS_LUA_IDENTIFIER = "identifier"
+TS_LUA_LOCAL_STATEMENT = "local_statement"
+LUA_STATEMENT_SUFFIX = "statement"
+LUA_DEFAULT_VAR_TYPES = (TS_LUA_IDENTIFIER,)
+
+LUA_METHOD_SEPARATOR = ":"
+
+# Tree-sitter Lua node types for language_spec
+TS_LUA_CHUNK = "chunk"
+TS_LUA_FUNCTION_DECLARATION = "function_declaration"
+TS_LUA_FUNCTION_DEFINITION = "function_definition"
+
+# Import processor function names
+IMPORT_REQUIRE = "require"
+IMPORT_PCALL = "pcall"
+IMPORT_IMPORT = "import"
diff --git a/codebase_rag/constants/ast_nodes.py b/codebase_rag/constants/ast_nodes.py
new file mode 100644
index 000000000..e24996a84
--- /dev/null
+++ b/codebase_rag/constants/ast_nodes.py
@@ -0,0 +1,269 @@
+# Shared tree-sitter node types, field names, and query captures.
+
+from .languages import SupportedLanguage
+
+FUNCTION_NODES_BASIC = ("function_declaration", "function_definition")
+FUNCTION_NODES_LAMBDA = (
+ "lambda_expression",
+ "arrow_function",
+ "anonymous_function",
+ "closure_expression",
+)
+FUNCTION_NODES_METHOD = (
+ "method_declaration",
+ "constructor_declaration",
+ "destructor_declaration",
+)
+FUNCTION_NODES_TEMPLATE = (
+ "template_declaration",
+ "function_signature_item",
+ "function_signature",
+)
+FUNCTION_NODES_GENERATOR = ("generator_function_declaration", "function_expression")
+
+CLASS_NODES_BASIC = ("class_declaration", "class_definition")
+CLASS_NODES_STRUCT = ("struct_declaration", "struct_specifier", "struct_item")
+CLASS_NODES_INTERFACE = ("interface_declaration", "trait_declaration", "trait_item")
+CLASS_NODES_ENUM = ("enum_declaration", "enum_item", "enum_specifier")
+CLASS_NODES_TYPE_ALIAS = ("type_alias_declaration", "type_item")
+CLASS_NODES_UNION = ("union_specifier", "union_item")
+
+CALL_NODES_BASIC = ("call_expression", "function_call")
+CALL_NODES_METHOD = (
+ "method_invocation",
+ "member_call_expression",
+ "field_expression",
+)
+CALL_NODES_OPERATOR = ("binary_expression", "unary_expression", "update_expression")
+CALL_NODES_SPECIAL = ("new_expression", "delete_expression", "macro_invocation")
+
+IMPORT_NODES_STANDARD = ("import_declaration", "import_statement")
+IMPORT_NODES_FROM = ("import_from_statement",)
+# variable_declaration: CommonJS `var X = require(...)` (express) binds
+# imports like const/let lexical_declarations.
+IMPORT_NODES_MODULE = (
+ "lexical_declaration",
+ "variable_declaration",
+ "export_statement",
+)
+IMPORT_NODES_INCLUDE = ("preproc_include",)
+
+JS_TS_FUNCTION_NODES = (
+ "function_declaration",
+ "generator_function_declaration",
+ "function_expression",
+ "arrow_function",
+ "method_definition",
+)
+JS_TS_CLASS_NODES = ("class_declaration", "class")
+JS_TS_IMPORT_NODES = (
+ "import_statement",
+ "lexical_declaration",
+ "variable_declaration",
+ "export_statement",
+)
+JS_TS_LANGUAGES = frozenset(
+ {SupportedLanguage.JS, SupportedLanguage.TS, SupportedLanguage.TSX}
+)
+
+CPP_IMPORT_NODES = ("preproc_include", "template_function", "declaration")
+
+# AST field names for name extraction
+NAME_FIELDS = ("identifier", "name", "id")
+
+# Tree-sitter field name constants for child_by_field_name
+FIELD_OBJECT = "object"
+FIELD_PROPERTY = "property"
+FIELD_NAME = "name"
+FIELD_ALIAS = "alias"
+FIELD_MODULE_NAME = "module_name"
+FIELD_ARGUMENTS = "arguments"
+FIELD_BODY = "body"
+FIELD_RETURN_TYPE = "return_type"
+FIELD_CONSTRUCTOR = "constructor"
+FIELD_DECLARATOR = "declarator"
+FIELD_PARAMETERS = "parameters"
+FIELD_RECEIVER = "receiver"
+FIELD_TYPE = "type"
+# The wrapped function/class inside a Python decorated_definition node.
+FIELD_DEFINITION = "definition"
+FIELD_RESULT = "result"
+# Rust impl `trait`/`type` fields and a trait's supertrait `bounds`.
+FIELD_TRAIT = "trait"
+FIELD_BOUNDS = "bounds"
+TS_RS_TRAIT_BOUNDS = "trait_bounds"
+FIELD_VALUE = "value"
+FIELD_LEFT = "left"
+FIELD_RIGHT = "right"
+# A C-style for's post-iteration clause: Java/C++ hold it in an `update`
+# field on the loop node, Go inside its `for_clause`.
+FIELD_UPDATE = "update"
+FIELD_FIELD = "field"
+FIELD_SCOPE = "scope"
+FIELD_SUPERCLASS = "superclass"
+FIELD_SUPERCLASSES = "superclasses"
+FIELD_INTERFACES = "interfaces"
+
+QUERY_FUNCTIONS = "functions"
+QUERY_CLASSES = "classes"
+QUERY_CALLS = "calls"
+QUERY_IMPORTS = "imports"
+QUERY_LOCALS = "locals"
+QUERY_CONFIG = "config"
+QUERY_LANGUAGE = "language"
+QUERY_HIGHLIGHTS = "highlights"
+
+CAPTURE_FUNCTION = "function"
+CAPTURE_CLASS = "class"
+CAPTURE_CALL = "call"
+CAPTURE_IMPORT = "import"
+CAPTURE_IMPORT_FROM = "import_from"
+CAPTURE_KEYWORD_MODIFIER = "keyword.modifier"
+CAPTURE_KEYWORD = "keyword"
+CAPTURE_ATTRIBUTE = "attribute"
+CAPTURE_FUNCTION_DECORATOR = "function.decorator"
+
+EXCLUDED_KEYWORDS = frozenset(
+ {
+ "def",
+ "class",
+ "fn",
+ "struct",
+ "impl",
+ "interface",
+ "enum",
+ "function",
+ "trait",
+ "type",
+ "void",
+ "None",
+ "True",
+ "False",
+ "null",
+ "true",
+ "false",
+ "return",
+ "import",
+ "from",
+ "as",
+ "where",
+ }
+)
+
+TS_IMPORT_STATEMENT = "import_statement"
+TS_IMPORT_FROM_STATEMENT = "import_from_statement"
+TS_DOTTED_NAME = "dotted_name"
+TS_ALIASED_IMPORT = "aliased_import"
+TS_RELATIVE_IMPORT = "relative_import"
+TS_IMPORT_PREFIX = "import_prefix"
+TS_WILDCARD_IMPORT = "wildcard_import"
+
+TS_STRING = "string"
+# JS/TS string literals hold their text in a string_fragment child (like
+# Python's string_content), used for I/O target extraction.
+TS_STRING_FRAGMENT = "string_fragment"
+# Modern Node builtin imports carry a node: scheme (`import fs from 'node:fs'`);
+# stripped when checking whether an imported name is the builtin module.
+NODE_BUILTIN_PREFIX = "node:"
+# `return_statement` node type (shared by Python and JS/TS grammars); used by
+# the language-agnostic flow walk.
+TS_RETURN_STATEMENT = "return_statement"
+# `await fetch(...)` wraps the call in an await_expression; the flow walk
+# unwraps it to see the inner source expression.
+TS_AWAIT_EXPRESSION = "await_expression"
+# tree-sitter parses comments as named children, so the flow walk filters them
+# out before indexing arguments or reading a single sub-expression.
+TS_COMMENT = "comment"
+# `(expr)` wraps its value in a parenthesized_expression; the flow walk unwraps
+# it (like await) to reach the inner source/tainted expression.
+TS_PARENTHESIZED_EXPRESSION = "parenthesized_expression"
+TS_IMPORT_CLAUSE = "import_clause"
+TS_LEXICAL_DECLARATION = "lexical_declaration"
+TS_VARIABLE_DECLARATION = "variable_declaration"
+TS_EXPORT_STATEMENT = "export_statement"
+TS_NAMED_IMPORTS = "named_imports"
+TS_IMPORT_SPECIFIER = "import_specifier"
+TS_NAMESPACE_IMPORT = "namespace_import"
+TS_IDENTIFIER = "identifier"
+TS_VARIABLE_DECLARATOR = "variable_declarator"
+TS_CALL_EXPRESSION = "call_expression"
+TS_EXPORT_CLAUSE = "export_clause"
+TS_EXPORT_SPECIFIER = "export_specifier"
+TS_EXPORT_DEFAULT = "default"
+TS_ACCESSIBILITY_MODIFIER = "accessibility_modifier"
+TS_PRIVATE = "private"
+TS_PRIVATE_PROPERTY_IDENTIFIER = "private_property_identifier"
+
+TS_IMPORT_DECLARATION = "import_declaration"
+TS_STATIC = "static"
+TS_SCOPED_IDENTIFIER = "scoped_identifier"
+TS_ASTERISK = "asterisk"
+
+TS_USE_DECLARATION = "use_declaration"
+
+TS_IMPORT_SPEC = "import_spec"
+TS_IMPORT_SPEC_LIST = "import_spec_list"
+TS_PACKAGE_IDENTIFIER = "package_identifier"
+TS_INTERPRETED_STRING_LITERAL = "interpreted_string_literal"
+
+TS_PREPROC_INCLUDE = "preproc_include"
+TS_TEMPLATE_FUNCTION = "template_function"
+TS_DECLARATION = "declaration"
+TS_STRING_LITERAL = "string_literal"
+TS_SYSTEM_LIB_STRING = "system_lib_string"
+TS_TEMPLATE_ARGUMENT_LIST = "template_argument_list"
+# Plain call/constructor argument list (C++ `in("x.txt")` init_declarator
+# value, Java `new FileWriter("x")` arguments).
+TS_ARGUMENT_LIST = "argument_list"
+# `do { .. } while (cond)`: same node type in the Java and C++ grammars.
+TS_DO_STATEMENT = "do_statement"
+# Shared verbatim by the JS/TS, Java, and C++ grammars.
+TS_BREAK_STATEMENT = "break_statement"
+TS_TYPE_DESCRIPTOR = "type_descriptor"
+TS_TYPE_IDENTIFIER = "type_identifier"
+
+TS_RETURN_STATEMENT = "return_statement"
+TS_RETURN = "return"
+TS_NEW_EXPRESSION = "new_expression"
+
+# Tree-sitter class/module node types for class_ingest
+TS_MODULE_DECLARATION = "module_declaration"
+TS_IMPL_ITEM = "impl_item"
+TS_INTERFACE_DECLARATION = "interface_declaration"
+TS_ENUM_DECLARATION = "enum_declaration"
+TS_ENUM_SPECIFIER = "enum_specifier"
+TS_ENUM_CLASS_SPECIFIER = "enum_class_specifier"
+TS_TYPE_ALIAS_DECLARATION = "type_alias_declaration"
+TS_STRUCT_SPECIFIER = "struct_specifier"
+TS_UNION_SPECIFIER = "union_specifier"
+TS_CLASS_DECLARATION = "class_declaration"
+TS_NAMESPACE_DEFINITION = "namespace_definition"
+TS_ABSTRACT_CLASS_DECLARATION = "abstract_class_declaration"
+TS_INTERNAL_MODULE = "internal_module"
+
+TS_BASE_CLASS_CLAUSE = "base_class_clause"
+TS_TEMPLATE_TYPE = "template_type"
+TS_ACCESS_SPECIFIER = "access_specifier"
+TS_VIRTUAL = "virtual"
+TS_TYPE_LIST = "type_list"
+TS_CLASS_HERITAGE = "class_heritage"
+# TS class `implements I, J` clause (a child of class_heritage).
+TS_IMPLEMENTS_CLAUSE = "implements_clause"
+TS_EXTENDS_CLAUSE = "extends_clause"
+TS_MEMBER_EXPRESSION = "member_expression"
+TS_SELECTOR_EXPRESSION = "selector_expression"
+TS_EXTENDS = "extends"
+TS_ARGUMENTS = "arguments"
+TS_EXTENDS_TYPE_CLAUSE = "extends_type_clause"
+
+TS_METHOD_DEFINITION = "method_definition"
+TS_DECORATOR = "decorator"
+TS_ERROR = "ERROR"
+TS_EXPRESSION_STATEMENT = "expression_statement"
+TS_STATEMENT_BLOCK = "statement_block"
+TS_PARENTHESIZED_EXPRESSION = "parenthesized_expression"
+TS_BINARY_EXPRESSION = "binary_expression"
+
+TS_ATTRIBUTE = "attribute"
+
+TS_FUNCTION_SIGNATURE = "function_signature"
diff --git a/codebase_rag/constants/ast_php.py b/codebase_rag/constants/ast_php.py
new file mode 100644
index 000000000..dce5a421c
--- /dev/null
+++ b/codebase_rag/constants/ast_php.py
@@ -0,0 +1,35 @@
+# PHP tree-sitter node types.
+
+TS_PHP_FUNCTION_DEFINITION = "function_definition"
+TS_PHP_METHOD_DECLARATION = "method_declaration"
+TS_PHP_TRAIT_DECLARATION = "trait_declaration"
+# PHP inheritance clauses: `extends ...` (base_clause, for class AND
+# interface) and `implements ...` (class_interface_clause); each lists `name`
+# nodes for the base types.
+TS_PHP_BASE_CLAUSE = "base_clause"
+TS_PHP_CLASS_INTERFACE_CLAUSE = "class_interface_clause"
+TS_PHP_NAME = "name"
+# PHP fully-qualified base (`\Exception`, `\App\Base`); its trailing `name`
+# child is the simple name cgr resolves against.
+TS_PHP_QUALIFIED_NAME = "qualified_name"
+TS_PHP_FUNCTION_STATIC_DECLARATION = "function_static_declaration"
+TS_PHP_ANONYMOUS_FUNCTION = "anonymous_function"
+TS_PHP_ARROW_FUNCTION = "arrow_function"
+TS_PHP_MEMBER_CALL_EXPRESSION = "member_call_expression"
+TS_PHP_SCOPED_CALL_EXPRESSION = "scoped_call_expression"
+TS_PHP_FUNCTION_CALL_EXPRESSION = "function_call_expression"
+TS_PHP_NULLSAFE_MEMBER_CALL_EXPRESSION = "nullsafe_member_call_expression"
+TS_PHP_OBJECT_CREATION_EXPRESSION = "object_creation_expression"
+TS_PHP_NAMESPACE_DEFINITION = "namespace_definition"
+TS_PHP_NAMESPACE_USE_DECLARATION = "namespace_use_declaration"
+TS_PHP_NAMESPACE_USE_CLAUSE = "namespace_use_clause"
+TS_PHP_FUNCTION = "function"
+TS_PHP_INCLUDE_EXPRESSION = "include_expression"
+TS_PHP_INCLUDE_ONCE_EXPRESSION = "include_once_expression"
+TS_PHP_REQUIRE_EXPRESSION = "require_expression"
+TS_PHP_REQUIRE_ONCE_EXPRESSION = "require_once_expression"
+TS_PHP_ATTRIBUTE_LIST = "attribute_list"
+TS_PHP_ATTRIBUTE = "attribute"
+TS_PHP_ATTRIBUTE_GROUP = "attribute_group"
+TS_PHP_VISIBILITY_MODIFIER = "visibility_modifier"
+TS_PHP_USE_DECLARATION = "use_declaration"
diff --git a/codebase_rag/constants/ast_python.py b/codebase_rag/constants/ast_python.py
new file mode 100644
index 000000000..9d1397412
--- /dev/null
+++ b/codebase_rag/constants/ast_python.py
@@ -0,0 +1,133 @@
+# Python tree-sitter node types and language constants.
+
+# Python tree-sitter node types for type inference
+TS_PY_IDENTIFIER = "identifier"
+TS_PY_TYPED_PARAMETER = "typed_parameter"
+TS_PY_TYPED_DEFAULT_PARAMETER = "typed_default_parameter"
+TS_PY_ATTRIBUTE = "attribute"
+TS_PY_CALL = "call"
+TS_PY_LIST = "list"
+TS_PY_DICTIONARY = "dictionary"
+TS_PY_PAIR = "pair"
+TS_PY_SET = "set"
+TS_PY_TUPLE = "tuple"
+TS_PY_PARENTHESIZED_EXPRESSION = "parenthesized_expression"
+TS_PY_EXPRESSION_LIST = "expression_list"
+TS_PY_LIST_COMPREHENSION = "list_comprehension"
+TS_PY_FOR_STATEMENT = "for_statement"
+TS_PY_FOR_IN_CLAUSE = "for_in_clause"
+TS_PY_ASSIGNMENT = "assignment"
+PY_ASSIGNMENT_QUERY = "(assignment) @assignment"
+PY_RETURN_QUERY = "(return_statement) @return_stmt"
+TS_PY_CLASS_DEFINITION = "class_definition"
+TS_PY_BLOCK = "block"
+TS_PY_FUNCTION_DEFINITION = "function_definition"
+TS_PY_LAMBDA = "lambda"
+TS_PY_RETURN_STATEMENT = "return_statement"
+TS_PY_RETURN = "return"
+TS_PY_KEYWORD = "keyword"
+TS_PY_MODULE = "module"
+TS_PY_IMPORT_STATEMENT = "import_statement"
+TS_PY_IMPORT_FROM_STATEMENT = "import_from_statement"
+TS_PY_WITH_STATEMENT = "with_statement"
+TS_PY_AS_PATTERN = "as_pattern"
+TS_PY_AS_PATTERN_TARGET = "as_pattern_target"
+TS_PY_EXPRESSION_STATEMENT = "expression_statement"
+TS_PY_STRING = "string"
+TS_PY_INTERPOLATION = "interpolation"
+TS_PY_DECORATED_DEFINITION = "decorated_definition"
+TS_PY_DECORATOR = "decorator"
+TS_PY_KEYWORD_ARGUMENT = "keyword_argument"
+TS_PY_DEFAULT_PARAMETER = "default_parameter"
+TS_PY_LIST_SPLAT_PATTERN = "list_splat_pattern"
+TS_PY_DICTIONARY_SPLAT_PATTERN = "dictionary_splat_pattern"
+TS_PY_SUBSCRIPT = "subscript"
+TS_PY_COMPARISON_OPERATOR = "comparison_operator"
+TS_FIELD_OPERATORS = "operators"
+TS_PY_IF_STATEMENT = "if_statement"
+TS_PY_TRY_STATEMENT = "try_statement"
+# Match statement: arms are exclusive; an UNGUARDED `case _` (empty
+# case_pattern) always matches, removing the implicit no-match path.
+TS_PY_MATCH_STATEMENT = "match_statement"
+TS_PY_CASE_CLAUSE = "case_clause"
+TS_PY_CASE_PATTERN = "case_pattern"
+TS_PY_FIELD_GUARD = "guard"
+FIELD_SUBJECT = "subject"
+# A bare name in a case pattern parses as dotted_name with ONE identifier
+# and is a CAPTURE (irrefutable); multi-part dotted names are value
+# patterns that compare.
+TS_PY_DOTTED_NAME = "dotted_name"
+# `a | b` case alternatives; the bare `_` alternative is an ANONYMOUS
+# node, invisible to named_children.
+TS_PY_UNION_PATTERN = "union_pattern"
+TS_PY_WILDCARD_NODE = "_"
+TS_PY_WHILE_STATEMENT = "while_statement"
+TS_PY_ELIF_CLAUSE = "elif_clause"
+TS_PY_ELSE_CLAUSE = "else_clause"
+TS_PY_EXCEPT_CLAUSE = "except_clause"
+TS_PY_FINALLY_CLAUSE = "finally_clause"
+TS_PY_CONDITIONAL_EXPRESSION = "conditional_expression"
+TS_PY_BOOLEAN_OPERATOR = "boolean_operator"
+TS_PY_NOT_OPERATOR = "not_operator"
+TS_FIELD_CONDITION = "condition"
+TS_FIELD_CONSEQUENCE = "consequence"
+TS_FIELD_ARGUMENT = "argument"
+
+# Python operator syntax dispatches to dunder methods at runtime; these names
+# let the call extractor synthesise the implied .__dunder__ call.
+PY_OP_IN = "in"
+PY_BUILTIN_LEN = "len"
+PY_BUILTIN_GETATTR = "getattr"
+TS_PY_STRING_CONTENT = "string_content"
+PY_DUNDER_GETITEM = "__getitem__"
+PY_DUNDER_SETITEM = "__setitem__"
+PY_DUNDER_CONTAINS = "__contains__"
+PY_DUNDER_LEN = "__len__"
+PY_DUNDER_BOOL = "__bool__"
+# Operands with these characters are not simple attribute/name chains (calls,
+# nested subscripts, whitespace), so the operator-dispatch synthesiser skips them.
+PY_OPERAND_REJECT_CHARS = "()[]{}\n\t "
+# Optional annotation handling: X | None names a single concrete class.
+PY_UNION_SEPARATOR = "|"
+PY_NONE = "None"
+# `-> Self` names the enclosing class, not a class called Self.
+PY_ANNOTATION_SELF = "Self"
+
+PY_KEYWORD_SELF = "self"
+PY_KEYWORD_CLS = "cls"
+# Visibility by naming convention: a leading underscore marks a private
+# symbol, while a dunder (__x__) is public API invoked by the runtime.
+PY_NAME_UNDERSCORE = "_"
+PY_NAME_DUNDER = "__"
+# typing.Protocol base name and the conventional XxxProtocol class suffix
+# used to map a Protocol to its concrete implementer.
+PY_PROTOCOL = "Protocol"
+PY_METHOD_INIT = "__init__"
+DECORATOR_AT = "@"
+PROPERTY_DECORATORS: frozenset[str] = frozenset({"property", "cached_property"})
+ABSTRACT_DECORATORS: frozenset[str] = frozenset({"abstractmethod", "abstractproperty"})
+
+# Eager builtins that invoke a callable argument synchronously in the caller's
+# stack frame, so the trace attributes the call to the enclosing function (no
+# Python frame exists for the builtin). Lazy higher-order builtins (map/filter)
+# are excluded: they defer invocation until the result is consumed, elsewhere.
+HIGHER_ORDER_BUILTINS: frozenset[str] = frozenset({"sorted", "min", "max", "reduce"})
+
+PY_SELF_PREFIX = "self."
+PY_CLS_PREFIX = "cls."
+
+PY_VAR_PATTERN_ALL = "all_"
+PY_VAR_SUFFIX_PLURAL = "s"
+PY_CLASS_REPOSITORY = "Repository"
+PY_MODELS_BASE_PATH = ".models.base."
+PY_METHOD_CREATE = "create"
+
+PY_SCORE_EXACT_MATCH = 100
+PY_SCORE_SUFFIX_MATCH = 90
+PY_SCORE_CONTAINS_BASE = 80
+
+TYPE_INFERENCE_LIST = "list"
+TYPE_INFERENCE_BASE_MODEL = "BaseModel"
+
+ATTR_TYPE_INFERENCE_IN_PROGRESS = "_type_inference_in_progress"
+GUARD_INHERITED_METHOD = "_inherited_method_guard"
diff --git a/codebase_rag/constants/ast_rust.py b/codebase_rag/constants/ast_rust.py
new file mode 100644
index 000000000..2ba35c5ab
--- /dev/null
+++ b/codebase_rag/constants/ast_rust.py
@@ -0,0 +1,172 @@
+# Rust tree-sitter node types and resolution constants.
+
+from .ast_java import TS_GENERIC_TYPE
+from .ast_nodes import TS_IDENTIFIER, TS_SCOPED_IDENTIFIER, TS_TYPE_IDENTIFIER
+from .ast_scala import TS_GENERIC_FUNCTION
+from .core import KEYWORD_SELF, KEYWORD_SUPER
+
+TS_RS_SCOPED_TYPE_IDENTIFIER = "scoped_type_identifier"
+TS_RS_PRIMITIVE_TYPE = "primitive_type"
+TS_RS_USE_AS_CLAUSE = "use_as_clause"
+TS_RS_USE_WILDCARD = "use_wildcard"
+TS_RS_USE_LIST = "use_list"
+TS_RS_SCOPED_USE_LIST = "scoped_use_list"
+TS_RS_SOURCE_FILE = "source_file"
+TS_RS_MOD_ITEM = "mod_item"
+TS_RS_CRATE = "crate"
+TS_RS_KEYWORD_AS = "as"
+TS_RS_STRUCT_ITEM = "struct_item"
+TS_RS_ENUM_ITEM = "enum_item"
+TS_RS_TRAIT_ITEM = "trait_item"
+TS_RS_TYPE_ITEM = "type_item"
+TS_RS_FUNCTION_ITEM = "function_item"
+TS_RS_IMPL_ITEM = "impl_item"
+TS_RS_FUNCTION_SIGNATURE_ITEM = "function_signature_item"
+TS_RS_CLOSURE_EXPRESSION = "closure_expression"
+TS_RS_UNION_ITEM = "union_item"
+TS_RS_USE_DECLARATION = "use_declaration"
+TS_RS_EXTERN_CRATE_DECLARATION = "extern_crate_declaration"
+TS_RS_CALL_EXPRESSION = "call_expression"
+TS_RS_MACRO_INVOCATION = "macro_invocation"
+TS_RS_MACRO_DEFINITION = "macro_definition"
+RS_MACRO_EXPORT_ATTR = "macro_export"
+TS_RS_LINE_COMMENT = "line_comment"
+TS_RS_BLOCK_COMMENT = "block_comment"
+RS_COMMENT_TYPES = (TS_RS_LINE_COMMENT, TS_RS_BLOCK_COMMENT)
+TS_RS_ATTRIBUTE_ITEM = "attribute_item"
+TS_RS_INNER_ATTRIBUTE_ITEM = "inner_attribute_item"
+
+# Rust I/O direct-sink walk node types (issue #714). call_expression keeps a
+# `function` field (a scoped_identifier like `std::fs::write`), so call_name works
+# unchanged; `macro_invocation` (`println!`) needs its own handling via macro_type.
+# A string_literal wraps a `string_content`; `block` is the fn-body lexical scope.
+TS_RS_STRING_LITERAL = "string_literal"
+TS_RS_STRING_CONTENT = "string_content"
+TS_RS_BLOCK = "block"
+TS_RS_FIELD_MACRO = "macro"
+# A macro body is a flat `token_tree` of raw tokens, not a parse tree, so a
+# call inside `println!(..)` has no call_expression node.
+TS_RS_TOKEN_TREE = "token_tree"
+TS_RS_TOKEN_SCOPE = "::"
+# `s.field` is a field_expression; `arr[i]` an index_expression. Inert for I/O
+# (Rust env access is a call), wired for correctness and future value sinks.
+TS_RS_INDEX_EXPRESSION = "index_expression"
+RS_FIELD_FIELD = "field"
+RS_FIELD_INDEX = "index"
+
+# Rust node types for local-variable type inference (receiver-dispatch)
+TS_RS_LET_DECLARATION = "let_declaration"
+TS_RS_PARAMETER = "parameter"
+TS_RS_SELF_PARAMETER = "self_parameter"
+TS_RS_STRUCT_EXPRESSION = "struct_expression"
+TS_RS_FIELD_DECLARATION_LIST = "field_declaration_list"
+TS_RS_FIELD_DECLARATION = "field_declaration"
+TS_RS_FIELD_IDENTIFIER = "field_identifier"
+TS_RS_MATCH_EXPRESSION = "match_expression"
+TS_RS_MATCH_ARM = "match_arm"
+TS_RS_IF_EXPRESSION = "if_expression"
+# `&s` / `&mut s`: a borrow is value-preserving, unwrapped by the lean flow
+# walk so the referent's taint carries through (issue #714).
+TS_RS_REFERENCE_EXPRESSION = "reference_expression"
+TS_RS_FOR_EXPRESSION = "for_expression"
+TS_RS_WHILE_EXPRESSION = "while_expression"
+TS_RS_LOOP_EXPRESSION = "loop_expression"
+# A Rust call node whose callee is descended for chain flattening: a plain call
+# or a turbofish generic_function (`f::()`).
+RS_CALL_OR_GENERIC_FN = (TS_RS_CALL_EXPRESSION, TS_GENERIC_FUNCTION)
+TS_RS_TUPLE_STRUCT_PATTERN = "tuple_struct_pattern"
+TS_RS_TYPE_ARGUMENTS = "type_arguments"
+TS_RS_TRY_EXPRESSION = "try_expression"
+TS_RS_FIELD_EXPRESSION = "field_expression"
+# Result-unwrapping method names: `File::open(p)?` / `.unwrap()` / `.expect(..)`
+# all yield the inner handle, so the I/O handle binder unwraps through them.
+RS_RESULT_UNWRAP_METHODS = frozenset({"unwrap", "expect"})
+TS_RS_FIELD_PATH = "path"
+TS_RS_TOKEN_DOT = "."
+# A receiver/chain base that is a plain identifier or the `self` keyword (used
+# both for macro-token receiver reconstruction and value-chain base flattening).
+RS_IDENT_OR_SELF = (TS_IDENTIFIER, KEYWORD_SELF)
+RS_MACRO_RECEIVER_TYPES = RS_IDENT_OR_SELF
+# Rust `Self` return type resolves to the enclosing impl target.
+RS_SELF_TYPE = "Self"
+# Transparent smart pointers that auto-deref to their inner type: a method call
+# on the pointer dispatches to the inner type's method, so strip them from any
+# type name (receiver OR return) to reach the real type.
+RS_DEREF_WRAPPERS = frozenset({"Arc", "Rc", "Box", "Pin"})
+# Guard containers that do NOT deref-coerce: the inner value is only reachable
+# through a lock/borrow guard accessor. Stripped to the inner type ONLY in field
+# extraction (the field is nearly always via a lock chain, e.g.
+# `self.shared.state.lock().unwrap()`); a bare local/param/return guard type is
+# preserved so a direct wrapper call (`m.is_poisoned()`) is not mis-resolved.
+RS_GUARD_WRAPPERS = frozenset({"Mutex", "RwLock", "RefCell", "Cell"})
+# Result/Option: stripped to their inner T only for a RETURN type (the
+# value a `?`/`.unwrap()` yields). NOT stripped for a receiver type, where a
+# method call `opt.map(..)` dispatches to Option itself.
+RS_RESULT_WRAPPERS = frozenset({"Result", "Option"})
+# Full strip set for return types (deref pointers + Result/Option unwrap).
+RS_RETURN_STRIP_WRAPPERS = RS_DEREF_WRAPPERS | RS_RESULT_WRAPPERS
+TS_RS_REFERENCE_TYPE = "reference_type"
+TS_RS_POINTER_TYPE = "pointer_type"
+# Trait-object and impl-Trait wrappers: `dyn Svc` / `impl Svc` / `dyn Svc + Send`.
+# The trait IS the value's static type (a method call dispatches through it), so
+# type walkers descend through these to the trait name, like the Java
+# interface-receiver design.
+TS_RS_DYNAMIC_TYPE = "dynamic_type"
+TS_RS_ABSTRACT_TYPE = "abstract_type"
+TS_RS_BOUNDED_TYPE = "bounded_type"
+# A parenthesized type (`&(dyn Svc + Send)`) parses as tuple_type; only a
+# single-element one is grouping (a real tuple has no single bare type).
+TS_RS_TUPLE_TYPE = "tuple_type"
+# Node types that can stand for a Rust return/field type. Reference/pointer
+# wrappers (`&Frame`, `*const T`) let a generic inner argument (`Result<&Frame>`)
+# and a bare `-> &Frame` descend to the referent; dyn/impl/bounded wrappers let a
+# trait-object type descend to its trait.
+RS_RETURN_TYPE_NODE_TYPES = (
+ TS_TYPE_IDENTIFIER,
+ TS_RS_PRIMITIVE_TYPE,
+ TS_GENERIC_TYPE,
+ TS_RS_SCOPED_TYPE_IDENTIFIER,
+ TS_RS_REFERENCE_TYPE,
+ TS_RS_POINTER_TYPE,
+ TS_RS_DYNAMIC_TYPE,
+ TS_RS_ABSTRACT_TYPE,
+ TS_RS_BOUNDED_TYPE,
+ TS_RS_TUPLE_TYPE,
+)
+# Wrapper-passthrough methods: they return the receiver's own (inner) type, so
+# a call-bound local keeps its type across them (`Type::mk().unwrap().m()`).
+RS_IDENTITY_METHODS = frozenset(
+ {
+ "unwrap",
+ "expect",
+ "clone",
+ "unwrap_or_default",
+ "to_owned",
+ "borrow",
+ "borrow_mut",
+ "as_ref",
+ "as_mut",
+ "as_deref",
+ "as_deref_mut",
+ }
+)
+# Guard accessors: called on a guard container (Mutex/RwLock/RefCell) to obtain a
+# guard that derefs to the inner type. In a receiver chain, one immediately after
+# a guard-wrapped field unwraps the wrapper to its inner type (recorded in
+# class_field_guard_inner), the only sound unwrap point, since guard containers
+# do not deref-coerce.
+RS_GUARD_ACCESSORS = frozenset(
+ {"lock", "read", "write", "try_lock", "borrow", "borrow_mut"}
+)
+
+RS_IDENTIFIER_TYPES = (TS_IDENTIFIER, TS_TYPE_IDENTIFIER)
+RS_SCOPED_TYPES = (TS_SCOPED_IDENTIFIER, TS_RS_SCOPED_TYPE_IDENTIFIER)
+RS_PATH_KEYWORDS = (TS_RS_CRATE, KEYWORD_SUPER, KEYWORD_SELF)
+
+RS_USE_LIST_DELIMITERS = frozenset({"{", "}", ","})
+
+RS_ENCODING_UTF8 = "utf8"
+
+RS_WILDCARD_PREFIX = "*"
+
+RS_FIELD_ARGUMENT = "argument"
diff --git a/codebase_rag/constants/ast_scala.py b/codebase_rag/constants/ast_scala.py
new file mode 100644
index 000000000..9e01b295b
--- /dev/null
+++ b/codebase_rag/constants/ast_scala.py
@@ -0,0 +1,17 @@
+# Scala tree-sitter node types.
+
+TS_SCALA_CLASS_DEFINITION = "class_definition"
+TS_SCALA_OBJECT_DEFINITION = "object_definition"
+TS_SCALA_TRAIT_DEFINITION = "trait_definition"
+TS_SCALA_COMPILATION_UNIT = "compilation_unit"
+TS_SCALA_FUNCTION_DEFINITION = "function_definition"
+TS_SCALA_FUNCTION_DECLARATION = "function_declaration"
+TS_SCALA_CALL_EXPRESSION = "call_expression"
+# Shared tree-sitter node type: a call with explicit type args, e.g. Rust
+# turbofish `f::()` and Scala `f[T]()`. Its `function` field holds the
+# callee (identifier or scoped_identifier).
+TS_GENERIC_FUNCTION = "generic_function"
+TS_SCALA_GENERIC_FUNCTION = TS_GENERIC_FUNCTION
+TS_SCALA_FIELD_EXPRESSION = "field_expression"
+TS_SCALA_INFIX_EXPRESSION = "infix_expression"
+TS_SCALA_IMPORT_DECLARATION = "import_declaration"
diff --git a/codebase_rag/constants/build.py b/codebase_rag/constants/build.py
new file mode 100644
index 000000000..11a21f5e5
--- /dev/null
+++ b/codebase_rag/constants/build.py
@@ -0,0 +1,67 @@
+# PyInstaller binary build constants.
+
+from enum import StrEnum
+from typing import NamedTuple
+
+
+class PyInstallerPackage(NamedTuple):
+ name: str
+ collect_all: bool = False
+ collect_data: bool = False
+ hidden_import: str | None = None
+
+
+class Architecture(StrEnum):
+ X86_64 = "x86_64"
+ AARCH64 = "aarch64"
+ ARM64 = "arm64"
+ AMD64 = "amd64"
+
+
+BINARY_NAME_TEMPLATE = "code-graph-rag-{system}-{machine}"
+BINARY_FILE_PERMISSION = 0o755
+DIST_DIR = "dist"
+BYTES_PER_MB_FLOAT = 1024 * 1024
+
+PYPROJECT_PATH = "pyproject.toml"
+PYPROJECT_KEY_TOOL = "tool"
+PYPROJECT_KEY_SETUPTOOLS = "setuptools"
+PYPROJECT_KEY_PACKAGE_DIR = "package-dir"
+TREESITTER_EXTRA_KEY = "treesitter-full"
+TREESITTER_PKG_PREFIX = "tree-sitter-"
+
+PYINSTALLER_CMD = "pyinstaller"
+PYINSTALLER_ARG_NAME = "--name"
+PYINSTALLER_ARG_ONEFILE = "--onefile"
+PYINSTALLER_ARG_NOCONFIRM = "--noconfirm"
+PYINSTALLER_ARG_CLEAN = "--clean"
+PYINSTALLER_ARG_COLLECT_ALL = "--collect-all"
+PYINSTALLER_ARG_COLLECT_DATA = "--collect-data"
+PYINSTALLER_ARG_HIDDEN_IMPORT = "--hidden-import"
+PYINSTALLER_ARG_EXCLUDE_MODULE = "--exclude-module"
+PYINSTALLER_ARG_COPY_METADATA = "--copy-metadata"
+PYINSTALLER_ENTRY_POINT = "main.py"
+
+PYINSTALLER_EXCLUDED_MODULES = ["logfire"]
+
+TOML_KEY_PROJECT = "project"
+TOML_KEY_OPTIONAL_DEPS = "optional-dependencies"
+
+VERSION_SPLIT_GTE = ">="
+VERSION_SPLIT_EQ = "=="
+VERSION_SPLIT_LT = "<"
+
+PYINSTALLER_PACKAGES: list["PyInstallerPackage"] = [
+ PyInstallerPackage(
+ name="pydantic_ai",
+ collect_all=True,
+ collect_data=True,
+ hidden_import="pydantic_ai_slim",
+ ),
+ PyInstallerPackage(name="rich", collect_all=True),
+ PyInstallerPackage(name="typer", collect_all=True),
+ PyInstallerPackage(name="loguru", collect_all=True),
+ PyInstallerPackage(name="toml", collect_all=True),
+ PyInstallerPackage(name="protobuf", collect_all=True),
+ PyInstallerPackage(name="genai_prices", collect_all=True),
+]
diff --git a/codebase_rag/constants/cli.py b/codebase_rag/constants/cli.py
new file mode 100644
index 000000000..27ed54857
--- /dev/null
+++ b/codebase_rag/constants/cli.py
@@ -0,0 +1,396 @@
+# CLI/TUI messages, styles, prompts, and interactive display constants.
+
+from enum import StrEnum
+
+
+class Color(StrEnum):
+ GREEN = "green"
+ YELLOW = "yellow"
+ CYAN = "cyan"
+ RED = "red"
+ MAGENTA = "magenta"
+ BLUE = "blue"
+
+
+class KeyBinding(StrEnum):
+ CTRL_J = "c-j"
+ CTRL_E = "c-e"
+ ENTER = "enter"
+ CTRL_C = "c-c"
+ SHIFT_TAB = "s-tab"
+
+
+class PermissionMode(StrEnum):
+ NORMAL = "normal"
+ YOLO = "yolo"
+
+
+class StyleModifier(StrEnum):
+ BOLD = "bold"
+ DIM = "dim"
+ NONE = ""
+
+
+class FileAction(StrEnum):
+ READ = "read"
+ EDIT = "edit"
+
+
+HELP_ARG = "help"
+
+CLI_ERR_OUTPUT_REQUIRES_UPDATE = (
+ "Error: --output/-o option requires --update-graph to be specified."
+)
+CLI_ERR_ONLY_JSON = "Error: Currently only JSON format is supported."
+CLI_ERR_JSON_REQUIRES_ASK_AGENT = (
+ "Error: --output-format json requires --ask-agent/-a; "
+ "it only applies to single-query output."
+)
+CLI_ERR_PATH_NOT_EXISTS = "Error: --repo-path does not exist: {path}"
+CLI_ERR_PATH_NOT_DIR = "Error: --repo-path is not a directory: {path}"
+CLI_WARN_NOT_GIT_REPO = "Warning: --repo-path is not a Git repository: {path}"
+CLI_ERR_STARTUP = "Startup Error: {error}"
+CLI_ERR_CONFIG = "Configuration Error: {error}"
+CLI_ERR_INDEXING = "An error occurred during indexing: {error}"
+CLI_ERR_EXPORT_FAILED = "Failed to export graph: {error}"
+CLI_ERR_LOAD_GRAPH = "Failed to load graph: {error}"
+CLI_ERR_MCP_SERVER = "MCP Server Error: {error}"
+
+CLI_MSG_UPDATING_GRAPH = "Updating knowledge graph for: {path}"
+CLI_MSG_SYNCING_GRAPH = "Syncing knowledge graph for: {path} (use --no-sync to skip)"
+CLI_MSG_WORKSPACE_SYNCING = "Syncing workspace '{name}' ({count} repos)..."
+CLI_MSG_WORKSPACE_SYNC_REPO = (
+ "[{idx}/{total}] Syncing {path} as project '{project_name}'"
+)
+CLI_MSG_WORKSPACE_EMPTY = (
+ "Workspace '{name}' has no repos (use cgr workspace add-repo)."
+)
+MSG_SYNCING_KNOWLEDGE_GRAPH = (
+ "[bold cyan]Syncing knowledge graph[/bold cyan] (incremental, --no-sync to skip)"
+)
+MSG_SYNCING_WORKSPACE = (
+ "[bold cyan]Syncing workspace '{name}'[/bold cyan] ({count} repos)"
+)
+CLI_MSG_SYNC_SKIPPED = "Knowledge graph already in sync for '{project}' ({elapsed:.2f}s, no changes detected)."
+CLI_MSG_SYNC_DONE = "Knowledge graph sync done for '{project}' in {elapsed:.2f}s."
+CLI_MSG_CLEANING_DB = "Cleaning database..."
+CLI_MSG_CLEANING_HASH_CACHE = "Removing hash cache: {path}"
+CLI_MSG_CLEAN_DONE = "Clean completed successfully!"
+CLI_MSG_DELETING_PROJECT = "Deleting project '{project_name}' from the graph..."
+CLI_MSG_PROJECT_DELETED = "Project '{project_name}' deleted successfully."
+CLI_ERR_PROJECT_NOT_FOUND = (
+ "Project '{project_name}' not found. Available projects: {projects}"
+)
+CLI_ERR_PROJECT_NAME_REQUIRED = (
+ "Error: --name is required and must be a non-empty project name."
+)
+CLI_ERR_DELETE_PROJECT_FAILED = "Failed to delete project '{project_name}': {error}"
+CLI_MSG_EXPORTING_TO = "Exporting graph to: {path}"
+CLI_MSG_GRAPH_UPDATED = "Graph update completed!"
+CLI_MSG_APP_TERMINATED = "\nApplication terminated by user."
+CLI_MSG_INDEXING_AT = "Indexing codebase at: {path}"
+CLI_MSG_OUTPUT_TO = "Output will be written to: {path}"
+CLI_MSG_INDEXING_DONE = "Indexing process completed successfully!"
+CLI_MSG_CONNECTING_MEMGRAPH = "Connecting to Memgraph to export graph..."
+CLI_MSG_EXPORTING_DATA = "Exporting graph data..."
+CLI_MSG_OPTIMIZATION_TERMINATED = "\nOptimization session terminated by user."
+CLI_MSG_MCP_TERMINATED = "\nMCP server terminated by user."
+PACKAGE_NAME = "code-graph-rag"
+CLI_MSG_VERSION = "{package} version {version}"
+CLI_MSG_HINT_TARGET_REPO = (
+ "\nHint: Make sure TARGET_REPO_PATH environment variable is set."
+)
+CLI_MSG_GRAPH_SUMMARY = "Graph Summary:"
+CLI_MSG_CONNECTING_STATS = "Fetching graph statistics..."
+CLI_STATS_NODE_TITLE = "Node Statistics"
+CLI_STATS_REL_TITLE = "Relationship Statistics"
+CLI_STATS_COL_NODE_TYPE = "Node Type"
+CLI_STATS_COL_REL_TYPE = "Relationship Type"
+CLI_STATS_COL_COUNT = "Count"
+CLI_STATS_TOTAL_NODES = "Total Nodes"
+CLI_STATS_TOTAL_RELS = "Total Relationships"
+CLI_STATS_UNKNOWN = "Unknown"
+CLI_ERR_STATS_FAILED = "Failed to get graph statistics: {error}"
+
+CLI_DEADCODE_CONNECTING = "Scanning for unreachable functions and methods..."
+CLI_DEADCODE_TABLE_TITLE = "Dead Code Candidates ({project_name})"
+CLI_DEADCODE_COL_KIND = "Kind"
+CLI_DEADCODE_COL_QUALIFIED_NAME = "Qualified Name"
+CLI_DEADCODE_COL_LINES = "Lines"
+CLI_DEADCODE_LINE_RANGE = "{start}-{end}"
+CLI_DEADCODE_SUMMARY = "{count} candidate(s) for review."
+CLI_DEADCODE_NONE = "No unreachable functions or methods found."
+CLI_DEADCODE_WRITTEN = "Wrote {count} candidate(s) to {path}"
+CLI_ERR_DEADCODE_FAILED = "Failed to scan for dead code: {error}"
+CLI_ERR_DEADCODE_NO_PROJECTS = (
+ "No projects found in the graph. Index a repository first with 'cgr start'."
+)
+CLI_ERR_DEADCODE_AMBIGUOUS_PROJECT = (
+ "Multiple projects found: {projects}. Specify which one with --project-name/-n."
+)
+CLI_MSG_AUTO_EXCLUDE = (
+ "Auto-excluding common directories (venv, node_modules, .git, etc.). "
+ "Use --interactive-setup to customize."
+)
+
+UI_DIFF_FILE_HEADER = "[bold cyan]File: {path}[/bold cyan]"
+UI_NEW_FILE_HEADER = "[bold cyan]New file: {path}[/bold cyan]"
+UI_SHELL_COMMAND_HEADER = "[bold cyan]Shell command:[/bold cyan]"
+UI_TOOL_APPROVAL = "[bold yellow]⚠️ Tool '{tool_name}' requires approval:[/bold yellow]"
+UI_FEEDBACK_PROMPT = "Feedback (why rejected, or press Enter to skip)"
+UI_OPTIMIZATION_START = (
+ "[bold green]Starting {language} optimization session...[/bold green]"
+)
+UI_OPTIMIZATION_PANEL = (
+ "[bold yellow]The agent will analyze your codebase{document_info} and propose specific optimizations."
+ " You'll be asked to approve each suggestion before implementation."
+ " Type 'exit' or 'quit' to end the session.[/bold yellow]"
+)
+UI_OPTIMIZATION_INIT = "[bold cyan]Initializing optimization session for {language} codebase: {path}[/bold cyan]"
+UI_GRAPH_EXPORT_SUCCESS = (
+ "[bold green]Graph exported successfully to: {path}[/bold green]"
+)
+UI_GRAPH_EXPORT_STATS = "[bold cyan]Export contains {nodes} nodes and {relationships} relationships[/bold cyan]"
+UI_ERR_UNEXPECTED = "[bold red]An unexpected error occurred: {error}[/bold red]"
+UI_ERR_EXPORT_FAILED = "[bold red]Failed to export graph: {error}[/bold red]"
+UI_MODEL_SWITCHED = "[bold green]Model switched to: {model}[/bold green]"
+UI_MODEL_CURRENT = "[bold cyan]Current model: {model}[/bold cyan]"
+UI_MODEL_SWITCH_ERROR = "[bold red]Failed to switch model: {error}[/bold red]"
+UI_MODEL_USAGE = "[bold yellow]Usage: /model (e.g., /model google:gemini-3.1-pro-preview)[/bold yellow]"
+# Per-turn token consumption and USD cost line (issue #80). The cost segment is
+# appended only for proprietary models with a known price.
+UI_TURN_USAGE_TOKENS = "tokens · turn {ti:,}→{to:,} · session {si:,}→{so:,}"
+UI_TURN_USAGE_COST = " · ${tc:.4f} turn · ${sc:.4f} session"
+# When an earlier turn had no known price (e.g. a local model), the running
+# session total understates the true spend, so it is shown as a partial floor.
+UI_TURN_USAGE_COST_PARTIAL = " · ${tc:.4f} turn · ${sc:.4f}+ session (partial)"
+UI_HELP_COMMANDS = """[bold cyan]Available commands:[/bold cyan]
+ /model - Switch to a different model
+ /model - Show current model
+ /help - Show this help
+ exit, quit - Exit the session"""
+UI_TOOL_ARGS_FORMAT = " Arguments: {args}"
+UI_REFERENCE_DOC_INFO = " using the reference document: {reference_document}"
+UI_INPUT_PROMPT_HTML = (
+ "{prompt}{hint}: "
+)
+
+
+class DeadCodeFormat(StrEnum):
+ TABLE = "table"
+ JSON = "json"
+
+
+class QueryFormat(StrEnum):
+ TABLE = "table"
+ JSON = "json"
+
+
+# Image file extensions for chat image handling
+MULTIMODAL_EXTENSIONS = (".png", ".jpg", ".jpeg", ".gif", ".webp", ".pdf")
+MIME_TYPE_PDF = "application/pdf"
+MIME_TYPE_FALLBACK = "application/octet-stream"
+YES_ANSWER = "y"
+YES_ANSWERS = frozenset({"y", "yes", ""})
+NO_ANSWERS = frozenset({"n", "no"})
+SHIFT_TAB_ESCAPE = b"\x1b[Z"
+DIFF_GIT_HEADER = "diff --git "
+MARKDOWN_FENCE = "```"
+MARKDOWN_FENCE_DIFF = "```diff"
+DIFF_CONTINUATION_PREFIXES = (
+ "diff --git ",
+ "index ",
+ "--- ",
+ "+++ ",
+ "@@ ",
+ "+",
+ "-",
+ " ",
+ "\\ ",
+ "new file mode",
+ "deleted file mode",
+ "old mode",
+ "new mode",
+ "rename from ",
+ "rename to ",
+ "similarity index ",
+ "Binary files ",
+)
+
+EXIT_COMMANDS = frozenset({"exit", "quit"})
+
+MODEL_COMMAND_PREFIX = "/model"
+HELP_COMMAND = "/help"
+
+HORIZONTAL_SEPARATOR = "─" * 60
+
+SESSION_LOG_HEADER = "=== CODE-GRAPH RAG SESSION LOG ===\n\n"
+
+LOG_FORMAT = "{time:YYYY-MM-DD HH:mm:ss.SSS} | {message}"
+
+TMP_DIR = ".tmp"
+SESSION_LOG_PREFIX = "session_"
+SESSION_LOG_EXT = ".log"
+
+SESSION_PREFIX_USER = "USER: "
+SESSION_PREFIX_ASSISTANT = "ASSISTANT: "
+
+SESSION_CONTEXT_START = (
+ "\n\n[SESSION CONTEXT - Previous conversation in this session]:\n"
+)
+SESSION_CONTEXT_END = "\n[END SESSION CONTEXT]\n\n"
+
+CONFIRM_ENABLED = "Enabled"
+CONFIRM_DISABLED = "Disabled (YOLO Mode)"
+
+DIFF_LABEL_BEFORE = "before"
+DIFF_LABEL_AFTER = "after"
+DIFF_FALLBACK_PATH = "file"
+
+
+class DiffMarker:
+ ADD = "+"
+ DEL = "-"
+ HUNK = "@"
+ HEADER_ADD = "+++"
+ HEADER_DEL = "---"
+
+
+TABLE_COL_CONFIGURATION = "Configuration"
+TABLE_COL_VALUE = "Value"
+
+TABLE_ROW_TARGET_LANGUAGE = "Target Language"
+TABLE_ROW_ORCHESTRATOR_MODEL = "Orchestrator Model"
+TABLE_ROW_CYPHER_MODEL = "Cypher Model"
+TABLE_ROW_OLLAMA_ENDPOINT = "Ollama Endpoint"
+TABLE_ROW_OLLAMA_ORCHESTRATOR = "Ollama Endpoint (Orchestrator)"
+TABLE_ROW_OLLAMA_CYPHER = "Ollama Endpoint (Cypher)"
+TABLE_ROW_EDIT_CONFIRMATION = "Edit Confirmation"
+TABLE_ROW_TARGET_REPOSITORY = "Target Repository"
+
+MSG_CONNECTED_MEMGRAPH = "Successfully connected to Memgraph."
+MSG_THINKING_CANCELLED = "Thinking cancelled."
+MSG_TIMEOUT_FORMAT = "Operation timed out after {timeout} seconds."
+MSG_TOOL_CALL_CANCELLED = "Tool call cancelled by user."
+MSG_CHAT_INSTRUCTIONS = (
+ "Ask questions about your codebase graph. Type 'exit' or 'quit' to end."
+)
+
+DEFAULT_TABLE_TITLE = "Code-Graph-RAG Initializing..."
+OPTIMIZATION_TABLE_TITLE = "Optimization Session Configuration"
+PROMPT_ASK_QUESTION = "Ask a question"
+PROMPT_YOUR_RESPONSE = "Your response"
+MULTILINE_INPUT_HINT = (
+ "(Press Ctrl+J or Ctrl+E to submit, Enter for new line, Shift+Tab to toggle mode)"
+)
+PERMISSION_MODE_NORMAL_LABEL = "● Normal mode (asks before destructive)"
+PERMISSION_MODE_YOLO_LABEL = "● YOLO mode (auto-approve, allowlist off)"
+PERMISSION_MODE_TOGGLED = "Permission mode: {label}"
+STATUS_BAR_BRANCH_CLEAN_HTML = (
+ ''
+)
+STATUS_BAR_BRANCH_DIRTY_HTML = (
+ ''
+)
+STATUS_BAR_BRANCH_CLEAN_PLAIN = " ⎇ {branch} "
+STATUS_BAR_BRANCH_DIRTY_PLAIN = " ⎇ {branch} ± "
+STATUS_BAR_BRANCH_RICH_TEXT = " ⎇ {branch}{marker} "
+STATUS_BAR_CLEAN_STYLE = "black on green"
+STATUS_BAR_DIRTY_STYLE = "black on yellow"
+STATUS_BAR_DIRTY_MARKER = " ±"
+STATUS_BAR_SPINNER = "dots"
+STATUS_BAR_SEPARATOR_CHAR = "─"
+STATUS_BAR_SEPARATOR_COLOR = "#666666"
+STATUS_BAR_TOKEN_HTML = ' '
+STATUS_BAR_CONFIG_COLOR = "#888888"
+STATUS_BAR_CONFIG_LABEL_COLOR = "#5fafd7"
+STATUS_BAR_CONFIG_SEPARATOR = " │ "
+STATUS_BAR_CONFIG_LABEL_O = "O"
+STATUS_BAR_CONFIG_LABEL_C = "C"
+STATUS_BAR_CONFIG_LABEL_EDIT = "edit"
+STATUS_BAR_CONFIG_LABEL_INSTRUCTIONS = "instructions"
+STATUS_BAR_CONFIG_LABEL_REPO = "repo"
+STATUS_BAR_EDIT_ON = "on"
+STATUS_BAR_EDIT_OFF = "off"
+TOKEN_THRESHOLD_WARNING = 50
+TOKEN_THRESHOLD_CRITICAL = 80
+TOKEN_COLOR_OK = "green"
+TOKEN_COLOR_WARNING = "yellow"
+TOKEN_COLOR_CRITICAL = "red"
+
+INTERACTIVE_TITLE_GROUPED = "Detected Directories (will be excluded unless kept)"
+INTERACTIVE_TITLE_NESTED = "Nested paths in '{pattern}'"
+INTERACTIVE_COL_NUM = "#"
+INTERACTIVE_COL_PATTERN = "Pattern"
+INTERACTIVE_COL_NESTED = "Nested"
+INTERACTIVE_COL_PATH = "Path"
+INTERACTIVE_STYLE_DIM = "dim"
+INTERACTIVE_STATUS_DETECTED = "auto-detected"
+INTERACTIVE_STATUS_CLI = "--exclude"
+INTERACTIVE_STATUS_CGRIGNORE = ".cgrignore"
+INTERACTIVE_NESTED_SINGULAR = "{count} dir"
+INTERACTIVE_NESTED_PLURAL = "{count} dirs"
+INTERACTIVE_INSTRUCTIONS_GROUPED = (
+ "These directories would normally be excluded. "
+ "Options: 'all' (keep all), 'none' (keep none), "
+ "numbers like '1,3' (keep groups), or '1e' to expand group 1"
+)
+INTERACTIVE_INSTRUCTIONS_NESTED = (
+ "Select paths to keep from '{pattern}'. "
+ "Options: 'all', 'none', or numbers like '1,3'"
+)
+INTERACTIVE_PROMPT_KEEP = "Keep"
+INTERACTIVE_KEEP_ALL = "all"
+INTERACTIVE_KEEP_NONE = "none"
+INTERACTIVE_EXPAND_SUFFIX = "e"
+INTERACTIVE_BFS_MAX_DEPTH = 10
+INTERACTIVE_DEFAULT_GROUP = "."
+
+MSG_SURGICAL_SUCCESS = "Successfully applied surgical code replacement in: {path}"
+MSG_SURGICAL_FAILED = (
+ "Failed to apply surgical replacement in {path}. "
+ "Target code not found or patches failed."
+)
+
+GREP_SUGGESTION = " Use 'rg' instead of 'grep' for text searching."
+
+QUERY_NOT_AVAILABLE = "N/A"
+DICT_KEY_RESULTS = "results"
+TIKTOKEN_ENCODING = "cl100k_base"
+QUERY_SUMMARY_SUCCESS = "Successfully retrieved {count} item(s) from the graph."
+QUERY_SUMMARY_TRUNCATED = (
+ "Results truncated: showing {kept} of {total} items (~{tokens} tokens, limit {max_tokens}). "
+ "Refine your query for more specific results."
+)
+QUERY_SUMMARY_TRANSLATION_FAILED = (
+ "I couldn't translate your request into a database query. Error: {error}"
+)
+QUERY_SUMMARY_DB_ERROR = "There was an error querying the database: {error}"
+QUERY_SUMMARY_TIMEOUT = (
+ "Query exceeded the {timeout:.1f}s timeout and was cancelled. "
+ "Avoid unbounded traversals; add depth bounds or use a graph-algorithm procedure."
+)
+QUERY_RESULTS_PANEL_TITLE = "[bold blue]Cypher Query Results[/bold blue]"
+
+MSG_SEMANTIC_NO_RESULTS = (
+ "No semantic matches found for query: '{query}'. This could mean:\n"
+ "1. No functions match this description\n"
+ "2. Semantic search dependencies are not installed\n"
+ "3. No embeddings have been generated yet"
+)
+MSG_SEMANTIC_SOURCE_UNAVAILABLE = (
+ "Could not retrieve source code for node ID {id}. "
+ "The node may not exist or source file may be unavailable."
+)
+MSG_SEMANTIC_SOURCE_FORMAT = "Source code for node ID {id}:\n\n```\n{code}\n```"
+MSG_SEMANTIC_RESULT_HEADER = "Found {count} semantic matches for '{query}':\n\n"
+MSG_SEMANTIC_RESULT_FOOTER = "\n\nUse the qualified names above with other tools to get more details or source code."
+SEMANTIC_BATCH_SIZE = 100
+SEMANTIC_TYPE_UNKNOWN = "Unknown"
+
+MSG_DOC_NO_CANDIDATES = "No valid text found in response candidates."
+MSG_DOC_NO_CONTENT = "No text content received from the API."
+MIME_TYPE_DEFAULT = "application/octet-stream"
+DOC_PROMPT_PREFIX = (
+ "Based on the document provided, please answer the following question: {question}"
+)
diff --git a/codebase_rag/constants/core.py b/codebase_rag/constants/core.py
new file mode 100644
index 000000000..fab0089ea
--- /dev/null
+++ b/codebase_rag/constants/core.py
@@ -0,0 +1,160 @@
+# Cross-cutting kernel constants: separators, chars, paths, misc keys.
+
+from enum import StrEnum
+
+INIT_PY = "__init__.py"
+
+ENCODING_UTF8 = "utf-8"
+
+ARG_TARGET_CODE = "target_code"
+ARG_REPLACEMENT_CODE = "replacement_code"
+ARG_FILE_PATH = "file_path"
+ARG_CONTENT = "content"
+ARG_COMMAND = "command"
+ARG_PATTERN = "pattern"
+ARG_REWRITE = "rewrite"
+ARG_LANGUAGE = "language"
+ARG_DRY_RUN = "dry_run"
+
+SEPARATOR_DOT = "."
+SEPARATOR_SLASH = "/"
+# Disambiguates definitions that share one qualified name (if/else import
+# fallbacks, typing.overload, try/except fallbacks): "@".
+DUP_QN_MARKER = "@"
+
+PATH_CURRENT_DIR = "."
+PATH_PARENT_DIR = ".."
+GLOB_ALL = "*"
+
+TRIE_TYPE_KEY = "__type__"
+TRIE_QN_KEY = "__qn__"
+TRIE_INTERNAL_PREFIX = "__"
+
+BYTES_PER_MB = 1024 * 1024
+
+EMPTY_PARENS = "()"
+DOCSTRING_STRIP_CHARS = "'\" \n"
+
+INLINE_MODULE_PATH_PREFIX = "inline_module_"
+
+# Method name constants for getattr/hasattr
+METHOD_FIND_WITH_PREFIX = "find_with_prefix"
+METHOD_ITEMS = "items"
+
+JSON_INDENT = 2
+
+
+class EventType(StrEnum):
+ MODIFIED = "modified"
+ CREATED = "created"
+ DELETED = "deleted"
+
+
+REALTIME_LOGGER_FORMAT = (
+ "{time:YYYY-MM-DD HH:mm:ss.SSS} | "
+ "{level: <8} | "
+ "{name}:{function}:{line} - "
+ "{message}"
+)
+
+WATCHER_SLEEP_INTERVAL = 1
+LOG_LEVEL_INFO = "INFO"
+LOG_LEVEL_ERROR = "ERROR"
+
+# Debounce settings for realtime watcher
+DEFAULT_DEBOUNCE_SECONDS = 5
+DEFAULT_MAX_WAIT_SECONDS = 30
+
+CHAR_HYPHEN = "-"
+CHAR_UNDERSCORE = "_"
+
+CHAR_SEMICOLON = ";"
+CHAR_COMMA = ","
+CHAR_COLON = ":"
+CHAR_ANGLE_OPEN = "<"
+CHAR_ANGLE_CLOSE = ">"
+CHAR_PAREN_OPEN = "("
+CHAR_PAREN_CLOSE = ")"
+CHAR_QUESTION_MARK = "?"
+
+CHAR_SPACE = " "
+SEPARATOR_COMMA_SPACE = ", "
+PUNCTUATION_TYPES = (CHAR_PAREN_OPEN, CHAR_PAREN_CLOSE, CHAR_COMMA)
+
+REGEX_METHOD_CHAIN_SUFFIX = r"\)\.[^)]*$"
+REGEX_FINAL_METHOD_CAPTURE = r"\.([^.()]+)$"
+
+DEFAULT_NAME = "Unknown"
+TEXT_UNKNOWN = "unknown"
+
+TMP_EXTENSION = ".tmp"
+
+MOD_RS = "mod.rs"
+SEPARATOR_DOUBLE_COLON = "::"
+SEPARATOR_PROTOTYPE = ".prototype."
+RUST_CRATE_PREFIX = "crate::"
+BUILTIN_PREFIX = "builtin"
+IIFE_FUNC_PREFIX = "iife_func_"
+IIFE_ARROW_PREFIX = "iife_arrow_"
+OPERATOR_PREFIX = "operator"
+KEYWORD_SUPER = "super"
+KEYWORD_SELF = "self"
+KEYWORD_CONSTRUCTOR = "constructor"
+
+# Incremental update hash cache
+HASH_CACHE_FILENAME = ".cgr-hash-cache.json"
+DIR_MTIMES_FILENAME = ".cgr-dir-mtimes.json"
+PARSER_FINGERPRINT_FILENAME = ".cgr-parser-fingerprint"
+CGR_STATE_FILENAMES: frozenset[str] = frozenset(
+ {HASH_CACHE_FILENAME, DIR_MTIMES_FILENAME, PARSER_FINGERPRINT_FILENAME}
+)
+
+# Inputs to the parser fingerprint: everything that changes how source files
+# become graph nodes and edges, plus the installed grammar wheels. Paths are
+# relative to the codebase_rag package root.
+PARSER_FINGERPRINT_SOURCE_DIRS: tuple[str, ...] = ("parsers", "constants")
+PARSER_FINGERPRINT_SOURCE_FILES: tuple[str, ...] = (
+ "graph_updater.py",
+ "function_registry.py",
+ "ast_cache.py",
+ "language_spec.py",
+ "parser_loader.py",
+)
+PY_SOURCE_GLOB = "*.py"
+# The bundled Roslyn C# frontend tool is parser code too, though .cs/.csproj
+# rather than Python: an edit changes the semantic edges it produces, so its
+# sources are folded into the parser fingerprint.
+PARSER_FINGERPRINT_TOOL_DIR = "parsers/csharp_frontend/roslyn"
+PARSER_FINGERPRINT_TOOL_GLOBS: tuple[str, ...] = ("*.cs", "*.csproj")
+GRAMMAR_DIST_PREFIX = "tree-sitter"
+GRAMMAR_VERSION_FMT = "{name}=={version}"
+GIT_DIR_NAME = ".git"
+ROOT_DIR_KEY = "."
+JSON_EMPTY_OBJECT = "{}"
+
+STR_NONE = "None"
+
+ENTITY_CLASS = "Class"
+ENTITY_FUNCTION = "Function"
+ENTITY_METHOD = "Method"
+
+PREFIX_LAMBDA = "lambda_"
+PREFIX_ANONYMOUS = "anonymous_"
+PREFIX_IIFE = "iife_"
+PREFIX_IIFE_DIRECT = "iife_direct_"
+PREFIX_ARROW = "arrow"
+PREFIX_FUNC = "func"
+
+# JSON keys for stdlib introspection subprocess responses
+JSON_KEY_HAS_ENTITY = "hasEntity"
+JSON_KEY_ENTITY_TYPE = "entityType"
+
+IMPORT_DEFAULT_SUFFIX = ".default"
+IMPORT_STD_PREFIX = "std."
+CPP_STD_PREFIX = "std"
+IMPORT_MODULE_LABEL = "Module"
+IMPORT_QUALIFIED_NAME = "qualified_name"
+IMPORT_RELATIONSHIP = "IMPORTS"
+
+# Delimiter tokens for argument parsing
+DELIMITER_TOKENS = frozenset({"(", ")", ","})
diff --git a/codebase_rag/constants/deadcode_roots.py b/codebase_rag/constants/deadcode_roots.py
new file mode 100644
index 000000000..d6b0539ca
--- /dev/null
+++ b/codebase_rag/constants/deadcode_roots.py
@@ -0,0 +1,224 @@
+# Dead-code reachability roots: entry points and framework hooks.
+
+# A C++ operator overload / user-defined literal is defined with the reserved
+# `operator` keyword heading the name (`operator==`, `operator[]`, `operator""_json`).
+# It is invoked by operator/literal syntax, not a named call, so it is a dead-code
+# reachability root; the keyword can only head such definitions, so this prefix on a
+# C++ file uniquely identifies them (member or free function).
+CPP_OPERATOR_PREFIX = "operator"
+
+# C/C++ program entries the OS runtime invokes with no visible call site:
+# hosted `main`/Windows `wmain`, GUI `WinMain`/`wWinMain`, and a DLL's
+# `DllMain`. Free functions on C/C++ files only.
+C_CPP_ENTRY_FUNCTION_NAMES: frozenset[str] = frozenset(
+ {"main", "wmain", "WinMain", "wWinMain", "DllMain"}
+)
+
+# Decorators whose presence marks a function/method as an implicit entry point
+# (web routes, task/flow handlers, fixtures, CLI commands, event listeners, and
+# Pydantic validators/serializers the framework invokes by registration).
+DEFAULT_ROOT_DECORATORS: frozenset[str] = frozenset(
+ {
+ "route",
+ "get",
+ "post",
+ "callback",
+ "put",
+ "delete",
+ "patch",
+ "websocket",
+ "task",
+ "flow",
+ "fixture",
+ "command",
+ "cli",
+ "app",
+ "on_event",
+ "listener",
+ "validator",
+ "field_validator",
+ "model_validator",
+ "root_validator",
+ "field_serializer",
+ "model_serializer",
+ "computed_field",
+ "abstractmethod",
+ # Property-family accessors are invoked by ATTRIBUTE syntax -- a bare
+ # read/write like `obj._output_field_or_none` produces no call node,
+ # so no CALLS edge can ever land on them (django WhereNode.
+ # _output_field_or_none, Expression._constructor_signature). The same
+ # invisible-invocation argument as dunders: roots, not dead code.
+ "property",
+ "cached_property",
+ "classproperty",
+ "hybrid_property",
+ "setter",
+ "deleter",
+ }
+)
+
+# Go functions the runtime invokes with no explicit call site: `func init()`
+# runs at package load (any number per package), `func main()` is the program
+# entry. Both are reachability roots (like Python dunders), gated by the .go
+# extension so same-named symbols in other languages are unaffected.
+GO_ROOT_FUNCTION_NAMES: frozenset[str] = frozenset({"init", "main"})
+
+# Rust `fn main()` is the binary entry point, invoked by the runtime with no
+# call site -- a reachability root (gated by .rs).
+RUST_ROOT_FUNCTION_NAMES: frozenset[str] = frozenset({"main"})
+
+# Rust trait-impl methods the language/std dispatches implicitly (Display::fmt
+# via format!, PartialEq::eq via ==, Iterator::next via for, operator traits,
+# Drop::drop, serde, ...), never through an explicit call the graph can see.
+# Rooting them by name (gated by .rs) mirrors the Python-dunder exemption: these
+# names are conventionally reserved for trait impls, so a same-named user method
+# that is genuinely dead is under-reported rather than mis-reported.
+RUST_TRAIT_METHOD_NAMES: frozenset[str] = frozenset(
+ {
+ "fmt",
+ "eq",
+ "ne",
+ "cmp",
+ "partial_cmp",
+ "hash",
+ "next",
+ "next_back",
+ "into_iter",
+ "size_hint",
+ "drop",
+ "clone",
+ "clone_from",
+ "default",
+ "from",
+ "from_str",
+ "try_from",
+ "into",
+ "try_into",
+ "deref",
+ "deref_mut",
+ "as_ref",
+ "as_mut",
+ "borrow",
+ "borrow_mut",
+ "poll",
+ "serialize",
+ "deserialize",
+ "source",
+ "add",
+ "add_assign",
+ "sub",
+ "sub_assign",
+ "mul",
+ "mul_assign",
+ "div",
+ "div_assign",
+ "rem",
+ "rem_assign",
+ "neg",
+ "not",
+ "bitand",
+ "bitand_assign",
+ "bitor",
+ "bitor_assign",
+ "bitxor",
+ "bitxor_assign",
+ "shl",
+ "shl_assign",
+ "shr",
+ "shr_assign",
+ "index",
+ "index_mut",
+ }
+)
+
+# Java serialization hooks the java.io runtime invokes reflectively during
+# (de)serialization -- never through a call the graph can see -- so they are
+# reachability roots (like Python dunders / Rust trait methods), gated by the .java
+# extension. These names are reserved by the Serializable contract, so rooting them
+# by name under-reports a same-named genuinely-dead method rather than mis-reporting.
+JAVA_SERIALIZATION_METHOD_NAMES: frozenset[str] = frozenset(
+ {
+ "readObject",
+ "writeObject",
+ "readObjectNoData",
+ "readResolve",
+ "writeReplace",
+ }
+)
+
+# C# attributes that mark a method as invoked by a framework/runtime rather
+# than a first-party call the graph can see -- so an attributed method is a
+# reachability root, gated by the .cs extension. Test runners invoke [Fact]/
+# [Theory]/[Test]/... ; ASP.NET routes to [HttpGet]/[Route]/... ; the
+# serialization runtime invokes [OnDeserialized]/... callbacks reflectively.
+# Names are the lowercased, argument-stripped form _norm_decorator produces.
+CSHARP_ROOT_ATTRIBUTES: frozenset[str] = frozenset(
+ {
+ "fact",
+ "theory",
+ "test",
+ "testmethod",
+ "testcase",
+ "setup",
+ "teardown",
+ "onetimesetup",
+ "onetimeteardown",
+ "globalsetup",
+ "apicontroller",
+ "route",
+ "httpget",
+ "httppost",
+ "httpput",
+ "httpdelete",
+ "httppatch",
+ "httphead",
+ "httpoptions",
+ "ondeserialized",
+ "ondeserializing",
+ "onserialized",
+ "onserializing",
+ }
+)
+
+# IDisposable methods the runtime invokes at the close of a `using` block (or
+# `await using`), never through a named call -- reachability roots, gated by
+# the .cs extension and method-ness (name-scoped, like the Java hooks above).
+CSHARP_DISPOSE_METHOD_NAMES: frozenset[str] = frozenset({"Dispose", "DisposeAsync"})
+
+# Base classes that mark a class as a structural interface: its method stubs
+# are never call targets themselves (callers resolve to the implementations),
+# so dead-code analysis roots every method the class defines.
+# ponytail: direct bases only; transitive Protocol subclassing is not chased.
+PROTOCOL_BASE_QNS: tuple[str, ...] = ("typing.Protocol", "typing_extensions.Protocol")
+
+# Substrings in a node's file path that mark it as test code. Covers Python
+# (test_, _test, conftest, /tests/), the JS/TS filename convention
+# (foo.test.ts, foo.spec.tsx), the Jest __tests__/ directory, and the
+# Node.js/mocha singular /test/ dir (express: 34 of 49 dead-code reports
+# were test/ helpers). Matching is segment-anchored via the leading-slash
+# normalization, so contest/ and latest/ do not match. Singular /spec/
+# stays excluded: it collides with product code (a domain "spec" module),
+# which would misclassify live code as test.
+TEST_PATH_PATTERNS: tuple[str, ...] = (
+ "test_",
+ "_test",
+ "conftest",
+ "/tests/",
+ "/test/",
+ ".test.",
+ ".spec.",
+ "__tests__",
+)
+
+# Python Enum protocol hooks: the enum machinery invokes these sunder
+# METHODS by NAME (_generate_next_value_ on auto(), _missing_ on a failed
+# value lookup), never through a call the graph can see -- runtime roots
+# exactly like dunders. A closed set: arbitrary sunder names are not part
+# of the protocol, and _ignore_/_order_ are class ATTRIBUTES consumed at
+# class creation, not methods, so they are deliberately absent.
+PY_ENUM_HOOK_METHOD_NAMES: frozenset[str] = frozenset(
+ {
+ "_generate_next_value_",
+ "_missing_",
+ }
+)
diff --git a/codebase_rag/constants/deps.py b/codebase_rag/constants/deps.py
new file mode 100644
index 000000000..78984aac5
--- /dev/null
+++ b/codebase_rag/constants/deps.py
@@ -0,0 +1,71 @@
+# Dependency-file names and manifest parsing keys.
+
+EXCLUDED_DEPENDENCY_NAMES = frozenset({"python", "php"})
+
+DEPENDENCY_FILES = frozenset(
+ {
+ "pyproject.toml",
+ "requirements.txt",
+ "package.json",
+ "cargo.toml",
+ "go.mod",
+ "gemfile",
+ "composer.json",
+ "pubspec.yaml",
+ }
+)
+CSPROJ_SUFFIX = ".csproj"
+
+DEP_KEY_TOOL = "tool"
+DEP_KEY_POETRY = "poetry"
+DEP_KEY_DEPENDENCIES = "dependencies"
+DEP_KEY_DEV_DEPENDENCIES = "dev-dependencies"
+DEP_KEY_PROJECT = "project"
+DEP_KEY_OPTIONAL_DEPS = "optional-dependencies"
+DEP_KEY_DEV_DEPS_JSON = "devDependencies"
+DEP_KEY_PEER_DEPS = "peerDependencies"
+DEP_KEY_REQUIRE = "require"
+DEP_KEY_REQUIRE_DEV = "require-dev"
+DEP_KEY_VERSION = "version"
+DEP_KEY_GROUP = "group"
+
+DEP_ATTR_INCLUDE = "Include"
+DEP_ATTR_VERSION = "Version"
+DEP_XML_PACKAGE_REF = "PackageReference"
+
+DEP_EXCLUDE_PYTHON = "python"
+DEP_EXCLUDE_PHP = "php"
+
+DEP_FILE_PYPROJECT = "pyproject.toml"
+DEP_FILE_REQUIREMENTS = "requirements.txt"
+DEP_FILE_PACKAGE_JSON = "package.json"
+DEP_FILE_CARGO = "cargo.toml"
+DEP_FILE_GOMOD = "go.mod"
+# The go.mod directive naming the module path that prefixes every import of
+# the module's packages; a same-line comment (incl. `// Deprecated:`) may
+# trail it.
+GO_KEYWORD_MODULE = "module"
+GO_MOD_COMMENT_PREFIX = "//"
+DEP_FILE_GEMFILE = "gemfile"
+DEP_FILE_COMPOSER = "composer.json"
+DEP_FILE_PUBSPEC = "pubspec.yaml"
+
+# pubspec.yaml dependency blocks; a `name: spec` under one of these top-level
+# keys is a package, while a nested block (`flutter:\n sdk: flutter`) has no
+# inline version and is recorded name-only.
+PUBSPEC_DEP_KEYS = frozenset({"dependencies", "dev_dependencies"})
+PUBSPEC_COMMENT_PREFIX = "#"
+PUBSPEC_KEY_SEP = ":"
+
+GOMOD_REQUIRE_BLOCK_START = "require ("
+GOMOD_BLOCK_END = ")"
+GOMOD_REQUIRE_LINE_PREFIX = "require "
+GOMOD_COMMENT_PREFIX = "//"
+
+GEMFILE_GEM_PREFIX = "gem "
+
+IMPORT_CACHE_TTL = 3600
+IMPORT_CACHE_DIR = ".cache/codebase_rag"
+IMPORT_CACHE_FILE = "stdlib_cache.json"
+IMPORT_CACHE_KEY = "cache"
+IMPORT_TIMESTAMPS_KEY = "timestamps"
diff --git a/codebase_rag/constants/fqn_specs.py b/codebase_rag/constants/fqn_specs.py
new file mode 100644
index 000000000..a15c90fcb
--- /dev/null
+++ b/codebase_rag/constants/fqn_specs.py
@@ -0,0 +1,615 @@
+# Per-language FQN scope types and language-spec node tuples.
+
+from .ast_cpp import (
+ TS_CPP_BINARY_EXPRESSION,
+ TS_CPP_CALL_EXPRESSION,
+ TS_CPP_DECLARATION,
+ TS_CPP_DELETE_EXPRESSION,
+ TS_CPP_FIELD_DECLARATION,
+ TS_CPP_FIELD_EXPRESSION,
+ TS_CPP_FUNCTION_DEFINITION,
+ TS_CPP_LAMBDA_EXPRESSION,
+ TS_CPP_LINKAGE_SPECIFICATION,
+ TS_CPP_NEW_EXPRESSION,
+ TS_CPP_SUBSCRIPT_EXPRESSION,
+ TS_CPP_TEMPLATE_DECLARATION,
+ TS_CPP_TRANSLATION_UNIT,
+ TS_CPP_UNARY_EXPRESSION,
+ TS_CPP_UPDATE_EXPRESSION,
+ CppNodeType,
+)
+from .ast_csharp import (
+ TS_CSHARP_CLASS_DECLARATION,
+ TS_CSHARP_COMPILATION_UNIT,
+ TS_CSHARP_CONSTRUCTOR_DECLARATION,
+ TS_CSHARP_CONVERSION_OPERATOR_DECLARATION,
+ TS_CSHARP_DESTRUCTOR_DECLARATION,
+ TS_CSHARP_ENUM_DECLARATION,
+ TS_CSHARP_FILE_SCOPED_NAMESPACE_DECLARATION,
+ TS_CSHARP_IMPLICIT_OBJECT_CREATION_EXPRESSION,
+ TS_CSHARP_INTERFACE_DECLARATION,
+ TS_CSHARP_INVOCATION_EXPRESSION,
+ TS_CSHARP_LOCAL_FUNCTION_STATEMENT,
+ TS_CSHARP_METHOD_DECLARATION,
+ TS_CSHARP_NAMESPACE_DECLARATION,
+ TS_CSHARP_OBJECT_CREATION_EXPRESSION,
+ TS_CSHARP_OPERATOR_DECLARATION,
+ TS_CSHARP_PROPERTY_DECLARATION,
+ TS_CSHARP_RECORD_DECLARATION,
+ TS_CSHARP_STRUCT_DECLARATION,
+ TS_CSHARP_USING_DIRECTIVE,
+)
+from .ast_dart import (
+ TS_DART_CASCADE_SECTION,
+ TS_DART_CLASS_DEFINITION,
+ TS_DART_CONSTANT_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTRUCTOR_SIGNATURE,
+ TS_DART_ENUM_DECLARATION,
+ TS_DART_EXTENSION_DECLARATION,
+ TS_DART_EXTENSION_TYPE_DECLARATION,
+ TS_DART_FACTORY_CONSTRUCTOR_SIGNATURE,
+ TS_DART_FUNCTION_SIGNATURE,
+ TS_DART_GETTER_SIGNATURE,
+ TS_DART_IMPORT_OR_EXPORT,
+ TS_DART_MIXIN_DECLARATION,
+ TS_DART_PART_DIRECTIVE,
+ TS_DART_PART_OF_DIRECTIVE,
+ TS_DART_PROGRAM,
+ TS_DART_SELECTOR,
+ TS_DART_SETTER_SIGNATURE,
+)
+from .ast_go import (
+ TS_GO_CALL_EXPRESSION,
+ TS_GO_FUNCTION_DECLARATION,
+ TS_GO_IMPORT_DECLARATION,
+ TS_GO_METHOD_DECLARATION,
+ TS_GO_SOURCE_FILE,
+ TS_GO_TYPE_ALIAS,
+ TS_GO_TYPE_DECLARATION,
+ TS_GO_TYPE_SPEC,
+)
+from .ast_java import (
+ TS_CONSTRUCTOR_DECLARATION,
+ TS_JAVA_ANNOTATION_TYPE_DECLARATION,
+ TS_JAVA_METHOD_INVOCATION,
+ TS_METHOD_DECLARATION,
+ TS_OBJECT_CREATION_EXPRESSION,
+ TS_PROGRAM,
+ TS_RECORD_DECLARATION,
+)
+from .ast_js import (
+ TS_ARROW_FUNCTION,
+ TS_CLASS_BODY,
+ TS_FUNCTION_DECLARATION,
+ TS_FUNCTION_EXPRESSION,
+ TS_GENERATOR_FUNCTION_DECLARATION,
+ TS_OBJECT,
+)
+from .ast_lua import (
+ TS_LUA_CHUNK,
+ TS_LUA_FUNCTION_CALL,
+ TS_LUA_FUNCTION_DECLARATION,
+ TS_LUA_FUNCTION_DEFINITION,
+)
+from .ast_nodes import (
+ TS_CALL_EXPRESSION,
+ TS_CLASS_DECLARATION,
+ TS_ENUM_DECLARATION,
+ TS_ENUM_SPECIFIER,
+ TS_FUNCTION_SIGNATURE,
+ TS_IMPORT_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+ TS_INTERNAL_MODULE,
+ TS_METHOD_DEFINITION,
+ TS_NAMESPACE_DEFINITION,
+ TS_NEW_EXPRESSION,
+ TS_STATEMENT_BLOCK,
+ TS_STRUCT_SPECIFIER,
+ TS_UNION_SPECIFIER,
+)
+from .ast_php import (
+ TS_PHP_ANONYMOUS_FUNCTION,
+ TS_PHP_ARROW_FUNCTION,
+ TS_PHP_FUNCTION_CALL_EXPRESSION,
+ TS_PHP_FUNCTION_DEFINITION,
+ TS_PHP_INCLUDE_EXPRESSION,
+ TS_PHP_INCLUDE_ONCE_EXPRESSION,
+ TS_PHP_MEMBER_CALL_EXPRESSION,
+ TS_PHP_METHOD_DECLARATION,
+ TS_PHP_NAMESPACE_DEFINITION,
+ TS_PHP_NAMESPACE_USE_DECLARATION,
+ TS_PHP_NULLSAFE_MEMBER_CALL_EXPRESSION,
+ TS_PHP_OBJECT_CREATION_EXPRESSION,
+ TS_PHP_REQUIRE_EXPRESSION,
+ TS_PHP_REQUIRE_ONCE_EXPRESSION,
+ TS_PHP_SCOPED_CALL_EXPRESSION,
+ TS_PHP_TRAIT_DECLARATION,
+)
+from .ast_python import (
+ TS_PY_CALL,
+ TS_PY_CLASS_DEFINITION,
+ TS_PY_FUNCTION_DEFINITION,
+ TS_PY_IMPORT_FROM_STATEMENT,
+ TS_PY_IMPORT_STATEMENT,
+ TS_PY_MODULE,
+ TS_PY_PAIR,
+ TS_PY_WITH_STATEMENT,
+)
+from .ast_rust import (
+ TS_RS_CALL_EXPRESSION,
+ TS_RS_CLOSURE_EXPRESSION,
+ TS_RS_ENUM_ITEM,
+ TS_RS_EXTERN_CRATE_DECLARATION,
+ TS_RS_FUNCTION_ITEM,
+ TS_RS_FUNCTION_SIGNATURE_ITEM,
+ TS_RS_IMPL_ITEM,
+ TS_RS_MACRO_DEFINITION,
+ TS_RS_MACRO_INVOCATION,
+ TS_RS_MOD_ITEM,
+ TS_RS_SOURCE_FILE,
+ TS_RS_STRUCT_ITEM,
+ TS_RS_TRAIT_ITEM,
+ TS_RS_TYPE_ITEM,
+ TS_RS_UNION_ITEM,
+ TS_RS_USE_DECLARATION,
+)
+from .ast_scala import (
+ TS_SCALA_CALL_EXPRESSION,
+ TS_SCALA_CLASS_DEFINITION,
+ TS_SCALA_COMPILATION_UNIT,
+ TS_SCALA_FIELD_EXPRESSION,
+ TS_SCALA_FUNCTION_DECLARATION,
+ TS_SCALA_FUNCTION_DEFINITION,
+ TS_SCALA_GENERIC_FUNCTION,
+ TS_SCALA_IMPORT_DECLARATION,
+ TS_SCALA_INFIX_EXPRESSION,
+ TS_SCALA_OBJECT_DEFINITION,
+ TS_SCALA_TRAIT_DEFINITION,
+)
+from .languages import (
+ PKG_CARGO_TOML,
+ PKG_CMAKE_LISTS,
+ PKG_CONANFILE,
+ PKG_INIT_PY,
+ PKG_MAKEFILE,
+ PKG_VCXPROJ_GLOB,
+)
+
+FQN_PY_SCOPE_TYPES = (
+ TS_PY_CLASS_DEFINITION,
+ TS_PY_MODULE,
+ TS_PY_FUNCTION_DEFINITION,
+)
+FQN_PY_FUNCTION_TYPES = (TS_PY_FUNCTION_DEFINITION,)
+
+FQN_JS_SCOPE_TYPES = (
+ TS_CLASS_DECLARATION,
+ TS_PROGRAM,
+ TS_FUNCTION_DECLARATION,
+ TS_FUNCTION_EXPRESSION,
+ TS_ARROW_FUNCTION,
+ TS_METHOD_DEFINITION,
+)
+FQN_JS_FUNCTION_TYPES = (
+ TS_FUNCTION_DECLARATION,
+ TS_METHOD_DEFINITION,
+ TS_ARROW_FUNCTION,
+ TS_FUNCTION_EXPRESSION,
+)
+
+# The grammar emits `internal_module` for a `namespace`/`module` block; without
+# it a class declared inside a namespace loses the namespace from its qn and
+# collides with a top-level same name.
+FQN_TS_SCOPE_TYPES = (
+ TS_CLASS_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+ TS_NAMESPACE_DEFINITION,
+ TS_INTERNAL_MODULE,
+ TS_PROGRAM,
+ TS_FUNCTION_DECLARATION,
+ TS_FUNCTION_EXPRESSION,
+ TS_ARROW_FUNCTION,
+ TS_METHOD_DEFINITION,
+)
+FQN_TS_FUNCTION_TYPES = (
+ TS_FUNCTION_DECLARATION,
+ TS_METHOD_DEFINITION,
+ TS_ARROW_FUNCTION,
+ TS_FUNCTION_EXPRESSION,
+ TS_FUNCTION_SIGNATURE,
+)
+
+# When climbing a nameless arrow's ancestors for its binding declarator,
+# crossing one of these means the arrow lives INSIDE another function's body
+# (a JSX event handler, a `.map()` callback), not bound to the outer const, so
+# it must not inherit that name. Otherwise the inner arrow becomes
+# `Component.Component`/`utils.getInitials.getInitials`, a double-segment phantom
+# with no incoming edge (dead-code false positive) that also steals the
+# enclosing scope from the real inline handler nodes.
+JS_ARROW_NAME_CLIMB_BOUNDARY = frozenset(
+ {
+ TS_STATEMENT_BLOCK,
+ TS_ARROW_FUNCTION,
+ TS_FUNCTION_DECLARATION,
+ TS_FUNCTION_EXPRESSION,
+ TS_METHOD_DEFINITION,
+ TS_GENERATOR_FUNCTION_DECLARATION,
+ TS_CLASS_BODY,
+ # An OBJECT VALUE (`var pets = { list: function(req, res) {...} }`)
+ # is owned by the pair-key naming path; climbing past it would steal
+ # the enclosing declarator's name and register a phantom function.
+ TS_PY_PAIR,
+ TS_OBJECT,
+ }
+)
+
+FQN_RS_SCOPE_TYPES = (
+ TS_RS_STRUCT_ITEM,
+ TS_RS_ENUM_ITEM,
+ TS_RS_TRAIT_ITEM,
+ TS_RS_IMPL_ITEM,
+ TS_RS_MOD_ITEM,
+ TS_RS_SOURCE_FILE,
+)
+FQN_RS_FUNCTION_TYPES = (
+ TS_RS_FUNCTION_ITEM,
+ TS_RS_FUNCTION_SIGNATURE_ITEM,
+ TS_RS_CLOSURE_EXPRESSION,
+)
+
+FQN_JAVA_SCOPE_TYPES = (
+ TS_CLASS_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+ TS_ENUM_DECLARATION,
+ TS_PROGRAM,
+ # An enclosing method/constructor is a qn scope for a function nested in its
+ # body (a method-body anonymous class's method): the call pass builds
+ # `Class.method.nested`, so the definition pass must include the method too,
+ # else the two disagree and every edge from the nested function dangles onto a
+ # phantom `Class.method.nested`, orphaning its callees. A direct method is the
+ # func itself, not an ancestor, so its own qn is unaffected.
+ TS_METHOD_DECLARATION,
+ TS_CONSTRUCTOR_DECLARATION,
+)
+FQN_JAVA_FUNCTION_TYPES = (
+ TS_METHOD_DECLARATION,
+ TS_CONSTRUCTOR_DECLARATION,
+)
+
+FQN_CPP_SCOPE_TYPES = (
+ CppNodeType.CLASS_SPECIFIER,
+ TS_STRUCT_SPECIFIER,
+ TS_NAMESPACE_DEFINITION,
+ TS_CPP_TRANSLATION_UNIT,
+)
+FQN_CPP_FUNCTION_TYPES = (
+ TS_CPP_FUNCTION_DEFINITION,
+ TS_CPP_DECLARATION,
+ TS_CPP_FIELD_DECLARATION,
+ TS_CPP_TEMPLATE_DECLARATION,
+ TS_CPP_LAMBDA_EXPRESSION,
+)
+
+# compilation_unit is a scope so file-scoped namespaces fold in (see
+# _csharp_get_name). An enclosing method/constructor is a scope so a nested
+# local function's qn agrees between the definition and call passes (cf. Java).
+FQN_CSHARP_SCOPE_TYPES = (
+ TS_CSHARP_COMPILATION_UNIT,
+ TS_CSHARP_NAMESPACE_DECLARATION,
+ TS_CSHARP_FILE_SCOPED_NAMESPACE_DECLARATION,
+ TS_CSHARP_CLASS_DECLARATION,
+ TS_CSHARP_STRUCT_DECLARATION,
+ TS_CSHARP_RECORD_DECLARATION,
+ TS_CSHARP_INTERFACE_DECLARATION,
+ TS_CSHARP_ENUM_DECLARATION,
+ TS_CSHARP_METHOD_DECLARATION,
+ TS_CSHARP_CONSTRUCTOR_DECLARATION,
+)
+FQN_CSHARP_FUNCTION_TYPES = (
+ TS_CSHARP_METHOD_DECLARATION,
+ TS_CSHARP_CONSTRUCTOR_DECLARATION,
+ TS_CSHARP_DESTRUCTOR_DECLARATION,
+ TS_CSHARP_LOCAL_FUNCTION_STATEMENT,
+ TS_CSHARP_OPERATOR_DECLARATION,
+ TS_CSHARP_CONVERSION_OPERATOR_DECLARATION,
+ TS_CSHARP_PROPERTY_DECLARATION,
+)
+
+FQN_LUA_SCOPE_TYPES = (TS_LUA_CHUNK,)
+FQN_LUA_FUNCTION_TYPES = (
+ TS_LUA_FUNCTION_DECLARATION,
+ TS_LUA_FUNCTION_DEFINITION,
+)
+
+# Scopes are the class-like declarations plus the file root; a nested
+# function/method's qn is built through these.
+FQN_DART_SCOPE_TYPES = (
+ TS_DART_CLASS_DEFINITION,
+ TS_DART_MIXIN_DECLARATION,
+ TS_DART_ENUM_DECLARATION,
+ TS_DART_EXTENSION_DECLARATION,
+ TS_DART_EXTENSION_TYPE_DECLARATION,
+ TS_DART_PROGRAM,
+)
+FQN_DART_FUNCTION_TYPES = (
+ TS_DART_FUNCTION_SIGNATURE,
+ TS_DART_GETTER_SIGNATURE,
+ TS_DART_SETTER_SIGNATURE,
+ TS_DART_FACTORY_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTANT_CONSTRUCTOR_SIGNATURE,
+)
+
+FQN_GO_SCOPE_TYPES = (
+ TS_GO_TYPE_DECLARATION,
+ TS_GO_SOURCE_FILE,
+)
+FQN_GO_FUNCTION_TYPES = (
+ TS_GO_FUNCTION_DECLARATION,
+ TS_GO_METHOD_DECLARATION,
+)
+
+FQN_SCALA_SCOPE_TYPES = (
+ TS_SCALA_CLASS_DEFINITION,
+ TS_SCALA_OBJECT_DEFINITION,
+ TS_SCALA_TRAIT_DEFINITION,
+ TS_SCALA_COMPILATION_UNIT,
+)
+FQN_SCALA_FUNCTION_TYPES = (
+ TS_SCALA_FUNCTION_DEFINITION,
+ TS_SCALA_FUNCTION_DECLARATION,
+)
+
+FQN_PHP_SCOPE_TYPES = (
+ TS_CLASS_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+ TS_PHP_TRAIT_DECLARATION,
+ TS_PHP_NAMESPACE_DEFINITION,
+ TS_PROGRAM,
+)
+FQN_PHP_FUNCTION_TYPES = (
+ TS_PHP_FUNCTION_DEFINITION,
+ TS_PHP_METHOD_DECLARATION,
+ TS_PHP_ANONYMOUS_FUNCTION,
+ TS_PHP_ARROW_FUNCTION,
+)
+
+SPEC_PY_FUNCTION_TYPES = (TS_PY_FUNCTION_DEFINITION,)
+SPEC_PY_CLASS_TYPES = (TS_PY_CLASS_DEFINITION,)
+SPEC_PY_MODULE_TYPES = (TS_PY_MODULE,)
+SPEC_PY_CALL_TYPES = (TS_PY_CALL, TS_PY_WITH_STATEMENT)
+SPEC_PY_IMPORT_TYPES = (TS_PY_IMPORT_STATEMENT,)
+SPEC_PY_IMPORT_FROM_TYPES = (TS_PY_IMPORT_FROM_STATEMENT,)
+SPEC_PY_PACKAGE_INDICATORS = (PKG_INIT_PY,)
+
+SPEC_JS_MODULE_TYPES = (TS_PROGRAM,)
+SPEC_JS_CALL_TYPES = (TS_CALL_EXPRESSION, TS_NEW_EXPRESSION)
+
+JS_NAME_NODE_TYPES = (
+ TS_FUNCTION_DECLARATION,
+ TS_CLASS_DECLARATION,
+ TS_METHOD_DEFINITION,
+ # TS `namespace`/`module` block; its `name` field scopes nested classes.
+ TS_INTERNAL_MODULE,
+)
+
+RS_TYPE_NODE_TYPES = (
+ TS_RS_STRUCT_ITEM,
+ TS_RS_ENUM_ITEM,
+ TS_RS_TRAIT_ITEM,
+ TS_RS_TYPE_ITEM,
+)
+RS_IDENT_NODE_TYPES = (TS_RS_FUNCTION_ITEM, TS_RS_MOD_ITEM)
+
+CPP_NAME_NODE_TYPES = (
+ CppNodeType.CLASS_SPECIFIER,
+ TS_STRUCT_SPECIFIER,
+ TS_ENUM_SPECIFIER,
+)
+
+C_NAME_NODE_TYPES = (
+ TS_STRUCT_SPECIFIER,
+ TS_UNION_SPECIFIER,
+ TS_ENUM_SPECIFIER,
+)
+
+SPEC_RS_FUNCTION_TYPES = (
+ TS_RS_FUNCTION_ITEM,
+ TS_RS_FUNCTION_SIGNATURE_ITEM,
+ TS_RS_CLOSURE_EXPRESSION,
+ # macro_rules! definitions register as Function nodes (the cross-language
+ # decision: C/C++/Rust macros all map onto Function), so macro_invocation
+ # call sites (already in SPEC_RS_CALL_TYPES) have a definition to resolve to.
+ TS_RS_MACRO_DEFINITION,
+)
+SPEC_RS_CLASS_TYPES = (
+ TS_RS_STRUCT_ITEM,
+ TS_RS_ENUM_ITEM,
+ TS_RS_UNION_ITEM,
+ TS_RS_TRAIT_ITEM,
+ TS_RS_IMPL_ITEM,
+ TS_RS_TYPE_ITEM,
+)
+SPEC_RS_MODULE_TYPES = (TS_RS_SOURCE_FILE, TS_RS_MOD_ITEM)
+SPEC_RS_CALL_TYPES = (TS_RS_CALL_EXPRESSION, TS_RS_MACRO_INVOCATION)
+SPEC_RS_IMPORT_TYPES = (TS_RS_USE_DECLARATION, TS_RS_EXTERN_CRATE_DECLARATION)
+SPEC_RS_IMPORT_FROM_TYPES = (TS_RS_USE_DECLARATION,)
+SPEC_RS_PACKAGE_INDICATORS = (PKG_CARGO_TOML,)
+
+# LANGUAGE_SPECS node type tuples for Dart. No call node types: the grammar
+# has no call-expression node, so cgr emits no Dart CALLS edges (structural
+# support only). Imports cover import/export, part, and part-of directives.
+SPEC_DART_FUNCTION_TYPES = (
+ TS_DART_FUNCTION_SIGNATURE,
+ TS_DART_GETTER_SIGNATURE,
+ TS_DART_SETTER_SIGNATURE,
+ TS_DART_FACTORY_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTRUCTOR_SIGNATURE,
+ TS_DART_CONSTANT_CONSTRUCTOR_SIGNATURE,
+)
+SPEC_DART_CLASS_TYPES = (
+ TS_DART_CLASS_DEFINITION,
+ TS_DART_MIXIN_DECLARATION,
+ TS_DART_ENUM_DECLARATION,
+ TS_DART_EXTENSION_DECLARATION,
+ TS_DART_EXTENSION_TYPE_DECLARATION,
+)
+SPEC_DART_MODULE_TYPES = (TS_DART_PROGRAM,)
+SPEC_DART_CALL_TYPES = (TS_DART_SELECTOR, TS_DART_CASCADE_SECTION)
+SPEC_DART_IMPORT_TYPES = (
+ TS_DART_IMPORT_OR_EXPORT,
+ TS_DART_PART_DIRECTIVE,
+ TS_DART_PART_OF_DIRECTIVE,
+)
+
+SPEC_GO_FUNCTION_TYPES = (TS_GO_FUNCTION_DECLARATION, TS_GO_METHOD_DECLARATION)
+SPEC_GO_CLASS_TYPES = (TS_GO_TYPE_SPEC, TS_GO_TYPE_ALIAS)
+SPEC_GO_MODULE_TYPES = (TS_GO_SOURCE_FILE,)
+SPEC_GO_CALL_TYPES = (TS_GO_CALL_EXPRESSION,)
+SPEC_GO_IMPORT_TYPES = (TS_GO_IMPORT_DECLARATION,)
+
+SPEC_SCALA_FUNCTION_TYPES = (
+ TS_SCALA_FUNCTION_DEFINITION,
+ TS_SCALA_FUNCTION_DECLARATION,
+)
+SPEC_SCALA_CLASS_TYPES = (
+ TS_SCALA_CLASS_DEFINITION,
+ TS_SCALA_OBJECT_DEFINITION,
+ TS_SCALA_TRAIT_DEFINITION,
+)
+SPEC_SCALA_MODULE_TYPES = (TS_SCALA_COMPILATION_UNIT,)
+SPEC_SCALA_CALL_TYPES = (
+ TS_SCALA_CALL_EXPRESSION,
+ TS_SCALA_GENERIC_FUNCTION,
+ TS_SCALA_FIELD_EXPRESSION,
+ TS_SCALA_INFIX_EXPRESSION,
+)
+SPEC_SCALA_IMPORT_TYPES = (TS_SCALA_IMPORT_DECLARATION,)
+
+SPEC_JAVA_FUNCTION_TYPES = (TS_METHOD_DECLARATION, TS_CONSTRUCTOR_DECLARATION)
+SPEC_JAVA_CLASS_TYPES = (
+ TS_CLASS_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+ TS_ENUM_DECLARATION,
+ TS_JAVA_ANNOTATION_TYPE_DECLARATION,
+ TS_RECORD_DECLARATION,
+)
+SPEC_JAVA_MODULE_TYPES = (TS_PROGRAM,)
+SPEC_JAVA_CALL_TYPES = (TS_JAVA_METHOD_INVOCATION, TS_OBJECT_CREATION_EXPRESSION)
+SPEC_JAVA_IMPORT_TYPES = (TS_IMPORT_DECLARATION,)
+
+SPEC_CSHARP_FUNCTION_TYPES = (
+ TS_CSHARP_METHOD_DECLARATION,
+ TS_CSHARP_CONSTRUCTOR_DECLARATION,
+ TS_CSHARP_DESTRUCTOR_DECLARATION,
+ TS_CSHARP_LOCAL_FUNCTION_STATEMENT,
+ TS_CSHARP_OPERATOR_DECLARATION,
+ TS_CSHARP_CONVERSION_OPERATOR_DECLARATION,
+ TS_CSHARP_PROPERTY_DECLARATION,
+)
+SPEC_CSHARP_CLASS_TYPES = (
+ TS_CSHARP_CLASS_DECLARATION,
+ TS_CSHARP_STRUCT_DECLARATION,
+ TS_CSHARP_RECORD_DECLARATION,
+ TS_CSHARP_INTERFACE_DECLARATION,
+ TS_CSHARP_ENUM_DECLARATION,
+)
+SPEC_CSHARP_MODULE_TYPES = (
+ TS_CSHARP_COMPILATION_UNIT,
+ TS_CSHARP_NAMESPACE_DECLARATION,
+ TS_CSHARP_FILE_SCOPED_NAMESPACE_DECLARATION,
+)
+SPEC_CSHARP_CALL_TYPES = (
+ TS_CSHARP_INVOCATION_EXPRESSION,
+ TS_CSHARP_OBJECT_CREATION_EXPRESSION,
+ TS_CSHARP_IMPLICIT_OBJECT_CREATION_EXPRESSION,
+)
+SPEC_CSHARP_IMPORT_TYPES = (TS_CSHARP_USING_DIRECTIVE,)
+
+SPEC_CPP_FUNCTION_TYPES = (
+ TS_CPP_FUNCTION_DEFINITION,
+ TS_CPP_DECLARATION,
+ TS_CPP_FIELD_DECLARATION,
+ TS_CPP_TEMPLATE_DECLARATION,
+ TS_CPP_LAMBDA_EXPRESSION,
+)
+SPEC_CPP_CLASS_TYPES = (
+ CppNodeType.CLASS_SPECIFIER,
+ TS_STRUCT_SPECIFIER,
+ TS_UNION_SPECIFIER,
+ TS_ENUM_SPECIFIER,
+)
+SPEC_CPP_MODULE_TYPES = (
+ TS_CPP_TRANSLATION_UNIT,
+ TS_NAMESPACE_DEFINITION,
+ TS_CPP_LINKAGE_SPECIFICATION,
+ TS_CPP_DECLARATION,
+)
+SPEC_CPP_CALL_TYPES = (
+ TS_CPP_CALL_EXPRESSION,
+ TS_CPP_FIELD_EXPRESSION,
+ TS_CPP_SUBSCRIPT_EXPRESSION,
+ TS_CPP_NEW_EXPRESSION,
+ TS_CPP_DELETE_EXPRESSION,
+ TS_CPP_BINARY_EXPRESSION,
+ TS_CPP_UNARY_EXPRESSION,
+ TS_CPP_UPDATE_EXPRESSION,
+)
+SPEC_CPP_PACKAGE_INDICATORS = (
+ PKG_CMAKE_LISTS,
+ PKG_MAKEFILE,
+ PKG_VCXPROJ_GLOB,
+ PKG_CONANFILE,
+)
+
+FQN_C_SCOPE_TYPES = (
+ TS_CPP_TRANSLATION_UNIT,
+ TS_STRUCT_SPECIFIER,
+ TS_UNION_SPECIFIER,
+ TS_ENUM_SPECIFIER,
+)
+FQN_C_FUNCTION_TYPES = (TS_CPP_FUNCTION_DEFINITION,)
+
+SPEC_C_FUNCTION_TYPES = (TS_CPP_FUNCTION_DEFINITION,)
+SPEC_C_CLASS_TYPES = (
+ TS_STRUCT_SPECIFIER,
+ TS_UNION_SPECIFIER,
+ TS_ENUM_SPECIFIER,
+)
+SPEC_C_MODULE_TYPES = (TS_CPP_TRANSLATION_UNIT,)
+SPEC_C_CALL_TYPES = (TS_CPP_CALL_EXPRESSION,)
+SPEC_C_PACKAGE_INDICATORS = (PKG_CMAKE_LISTS, PKG_MAKEFILE)
+
+SPEC_PHP_FUNCTION_TYPES = (
+ TS_PHP_FUNCTION_DEFINITION,
+ TS_PHP_METHOD_DECLARATION,
+ TS_PHP_ANONYMOUS_FUNCTION,
+ TS_PHP_ARROW_FUNCTION,
+)
+SPEC_PHP_CLASS_TYPES = (
+ TS_CLASS_DECLARATION,
+ TS_INTERFACE_DECLARATION,
+ TS_PHP_TRAIT_DECLARATION,
+ TS_ENUM_DECLARATION,
+)
+SPEC_PHP_MODULE_TYPES = (TS_PROGRAM,)
+SPEC_PHP_CALL_TYPES = (
+ TS_PHP_FUNCTION_CALL_EXPRESSION,
+ TS_PHP_MEMBER_CALL_EXPRESSION,
+ TS_PHP_SCOPED_CALL_EXPRESSION,
+ TS_PHP_NULLSAFE_MEMBER_CALL_EXPRESSION,
+ TS_PHP_OBJECT_CREATION_EXPRESSION,
+)
+SPEC_PHP_IMPORT_TYPES = (TS_PHP_NAMESPACE_USE_DECLARATION,)
+SPEC_PHP_IMPORT_FROM_TYPES = (
+ TS_PHP_INCLUDE_EXPRESSION,
+ TS_PHP_INCLUDE_ONCE_EXPRESSION,
+ TS_PHP_REQUIRE_EXPRESSION,
+ TS_PHP_REQUIRE_ONCE_EXPRESSION,
+)
+
+SPEC_LUA_FUNCTION_TYPES = (TS_LUA_FUNCTION_DECLARATION, TS_LUA_FUNCTION_DEFINITION)
+SPEC_LUA_CLASS_TYPES: tuple[str, ...] = ()
+SPEC_LUA_MODULE_TYPES = (TS_LUA_CHUNK,)
+SPEC_LUA_CALL_TYPES = (TS_LUA_FUNCTION_CALL,)
+SPEC_LUA_IMPORT_TYPES = (TS_LUA_FUNCTION_CALL,)
diff --git a/codebase_rag/constants/graph.py b/codebase_rag/constants/graph.py
new file mode 100644
index 000000000..d3c583ef7
--- /dev/null
+++ b/codebase_rag/constants/graph.py
@@ -0,0 +1,479 @@
+# Graph schema: node labels, relationships, keys, and Cypher queries.
+
+from enum import StrEnum
+
+KEY_NODES = "nodes"
+KEY_RELATIONSHIPS = "relationships"
+KEY_NODE_ID = "node_id"
+KEY_LABELS = "labels"
+KEY_LABEL = "label"
+KEY_PROPERTIES = "properties"
+KEY_PURGED = "purged"
+KEY_FROM_ID = "from_id"
+KEY_TO_ID = "to_id"
+KEY_TYPE = "type"
+KEY_METADATA = "metadata"
+KEY_TOTAL_NODES = "total_nodes"
+KEY_TOTAL_RELATIONSHIPS = "total_relationships"
+KEY_NODE_LABELS = "node_labels"
+KEY_RELATIONSHIP_TYPES = "relationship_types"
+KEY_EXPORTED_AT = "exported_at"
+KEY_PARSER = "parser"
+KEY_NAME = "name"
+KEY_ROOT_PATH = "root_path"
+KEY_QUALIFIED_NAME = "qualified_name"
+KEY_IS_PROPERTY = "is_property"
+KEY_IS_MACRO = "is_macro"
+KEY_QUERY = "query"
+KEY_RESPONSE = "response"
+KEY_START_LINE = "start_line"
+KEY_END_LINE = "end_line"
+KEY_PATH = "path"
+KEY_ABSOLUTE_PATH = "absolute_path"
+KEY_EXTENSION = "extension"
+KEY_MODULE_TYPE = "module_type"
+KEY_IMPLEMENTS_MODULE = "implements_module"
+KEY_PROPS = "props"
+KEY_CREATED = "created"
+KEY_FROM_VAL = "from_val"
+KEY_TO_VAL = "to_val"
+KEY_FROM_LABEL = "from_label"
+KEY_FROM_QN = "from_qn"
+KEY_REL_TYPE = "rel_type"
+KEY_TO_LABEL = "to_label"
+KEY_TO_QN = "to_qn"
+KEY_PROJECT_PREFIX = "project_prefix"
+KEY_VERSION_SPEC = "version_spec"
+KEY_PREFIX = "prefix"
+KEY_PROJECT_NAME = "project_name"
+# ast-grep finding node properties (issue #413)
+KEY_MESSAGE = "message"
+KEY_SNIPPET = "snippet"
+
+ERR_SUBSTR_ALREADY_EXISTS = "already exists"
+ERR_SUBSTR_CONSTRAINT = "constraint"
+
+PROTOBUF_INDEX_FILE = "index.bin"
+PROTOBUF_NODES_FILE = "nodes.bin"
+PROTOBUF_RELS_FILE = "relationships.bin"
+
+ONEOF_PROJECT = "project"
+ONEOF_PACKAGE = "package"
+ONEOF_FOLDER = "folder"
+ONEOF_MODULE = "module"
+ONEOF_CLASS = "class_node"
+ONEOF_FUNCTION = "function"
+ONEOF_METHOD = "method"
+ONEOF_FILE = "file"
+ONEOF_EXTERNAL_PACKAGE = "external_package"
+ONEOF_EXTERNAL_MODULE = "external_module"
+ONEOF_MODULE_IMPLEMENTATION = "module_implementation"
+ONEOF_MODULE_INTERFACE = "module_interface"
+ONEOF_INTERFACE = "interface_node"
+ONEOF_ENUM = "enum_node"
+ONEOF_TYPE = "type_node"
+ONEOF_UNION = "union_node"
+ONEOF_RESOURCE = "resource"
+
+
+class UniqueKeyType(StrEnum):
+ NAME = KEY_NAME
+ PATH = KEY_PATH
+ ABSOLUTE_PATH = KEY_ABSOLUTE_PATH
+ QUALIFIED_NAME = KEY_QUALIFIED_NAME
+
+
+class NodeLabel(StrEnum):
+ PROJECT = "Project"
+ PACKAGE = "Package"
+ FOLDER = "Folder"
+ FILE = "File"
+ MODULE = "Module"
+ CLASS = "Class"
+ FUNCTION = "Function"
+ METHOD = "Method"
+ INTERFACE = "Interface"
+ ENUM = "Enum"
+ TYPE = "Type"
+ UNION = "Union"
+ MODULE_INTERFACE = "ModuleInterface"
+ MODULE_IMPLEMENTATION = "ModuleImplementation"
+ EXTERNAL_PACKAGE = "ExternalPackage"
+ EXTERNAL_MODULE = "ExternalModule"
+ RESOURCE = "Resource"
+ # ast-grep findings (issue #413): quality/security signals attached to a
+ # Module. Opt-in via CaptureGroup.FINDINGS.
+ PATTERN = "Pattern"
+ CODE_SMELL = "CodeSmell"
+ SECURITY_ISSUE = "SecurityIssue"
+
+
+_NODE_LABEL_UNIQUE_KEYS: dict[NodeLabel, UniqueKeyType] = {
+ NodeLabel.PROJECT: UniqueKeyType.NAME,
+ NodeLabel.PACKAGE: UniqueKeyType.QUALIFIED_NAME,
+ # Folder and File identity must be per checkout: keyed on the bare
+ # relative path, two same-layout projects in the shared graph merge
+ # onto one node and delete-project crosses into the sibling's subtree
+ # (issue #897).
+ NodeLabel.FOLDER: UniqueKeyType.ABSOLUTE_PATH,
+ NodeLabel.FILE: UniqueKeyType.ABSOLUTE_PATH,
+ NodeLabel.MODULE: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.CLASS: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.FUNCTION: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.METHOD: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.INTERFACE: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.ENUM: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.TYPE: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.UNION: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.MODULE_INTERFACE: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.MODULE_IMPLEMENTATION: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.EXTERNAL_PACKAGE: UniqueKeyType.NAME,
+ NodeLabel.EXTERNAL_MODULE: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.RESOURCE: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.PATTERN: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.CODE_SMELL: UniqueKeyType.QUALIFIED_NAME,
+ NodeLabel.SECURITY_ISSUE: UniqueKeyType.QUALIFIED_NAME,
+}
+
+_missing_keys = set(NodeLabel) - set(_NODE_LABEL_UNIQUE_KEYS.keys())
+if _missing_keys:
+ raise RuntimeError(
+ f"NodeLabel(s) missing from _NODE_LABEL_UNIQUE_KEYS: {sorted(_missing_keys)}. "
+ "Every NodeLabel MUST have a unique key defined."
+ )
+
+
+class RelationshipType(StrEnum):
+ CONTAINS_PACKAGE = "CONTAINS_PACKAGE"
+ CONTAINS_FOLDER = "CONTAINS_FOLDER"
+ CONTAINS_FILE = "CONTAINS_FILE"
+ CONTAINS_MODULE = "CONTAINS_MODULE"
+ DEFINES = "DEFINES"
+ DEFINES_METHOD = "DEFINES_METHOD"
+ IMPORTS = "IMPORTS"
+ EXPORTS = "EXPORTS"
+ EXPORTS_MODULE = "EXPORTS_MODULE"
+ IMPLEMENTS_MODULE = "IMPLEMENTS_MODULE"
+ INHERITS = "INHERITS"
+ IMPLEMENTS = "IMPLEMENTS"
+ OVERRIDES = "OVERRIDES"
+ CALLS = "CALLS"
+ REFERENCES = "REFERENCES"
+ INSTANTIATES = "INSTANTIATES"
+ DEPENDS_ON_EXTERNAL = "DEPENDS_ON_EXTERNAL"
+ READS_FROM = "READS_FROM"
+ WRITES_TO = "WRITES_TO"
+ FLOWS_TO = "FLOWS_TO"
+ EXPOSES = "EXPOSES"
+ RESOLVES_TO = "RESOLVES_TO"
+ IMPLEMENTS_PATTERN = "IMPLEMENTS_PATTERN"
+ HAS_SMELL = "HAS_SMELL"
+ HAS_VULNERABILITY = "HAS_VULNERABILITY"
+
+
+class CaptureGroup(StrEnum):
+ STRUCTURE = "structure"
+ CALLS = "calls"
+ TYPES = "types"
+ IMPORTS = "imports"
+ IO = "io"
+ FINDINGS = "findings"
+
+
+# Each relationship type belongs to exactly one capture group. The guard below
+# enforces total coverage, so a new RelationshipType cannot silently escape the
+# capture model.
+CAPTURE_GROUP_RELS: dict[CaptureGroup, frozenset[RelationshipType]] = {
+ CaptureGroup.STRUCTURE: frozenset(
+ {
+ RelationshipType.CONTAINS_PACKAGE,
+ RelationshipType.CONTAINS_FOLDER,
+ RelationshipType.CONTAINS_FILE,
+ RelationshipType.CONTAINS_MODULE,
+ RelationshipType.DEFINES,
+ RelationshipType.DEFINES_METHOD,
+ }
+ ),
+ CaptureGroup.CALLS: frozenset(
+ {
+ RelationshipType.CALLS,
+ RelationshipType.REFERENCES,
+ RelationshipType.INSTANTIATES,
+ }
+ ),
+ CaptureGroup.TYPES: frozenset(
+ {
+ RelationshipType.INHERITS,
+ RelationshipType.IMPLEMENTS,
+ RelationshipType.IMPLEMENTS_MODULE,
+ RelationshipType.OVERRIDES,
+ }
+ ),
+ CaptureGroup.IMPORTS: frozenset(
+ {
+ RelationshipType.IMPORTS,
+ RelationshipType.EXPORTS,
+ RelationshipType.EXPORTS_MODULE,
+ RelationshipType.DEPENDS_ON_EXTERNAL,
+ }
+ ),
+ CaptureGroup.IO: frozenset(
+ {
+ RelationshipType.READS_FROM,
+ RelationshipType.WRITES_TO,
+ RelationshipType.FLOWS_TO,
+ RelationshipType.EXPOSES,
+ RelationshipType.RESOLVES_TO,
+ }
+ ),
+ CaptureGroup.FINDINGS: frozenset(
+ {
+ RelationshipType.IMPLEMENTS_PATTERN,
+ RelationshipType.HAS_SMELL,
+ RelationshipType.HAS_VULNERABILITY,
+ }
+ ),
+}
+
+# Node labels a group exclusively owns; the label is captured only while the
+# owning group has an enabled relationship. Labels owned by no group are always
+# captured.
+CAPTURE_GROUP_NODE_LABELS: dict[CaptureGroup, frozenset[NodeLabel]] = {
+ CaptureGroup.IO: frozenset({NodeLabel.RESOURCE}),
+ CaptureGroup.FINDINGS: frozenset(
+ {NodeLabel.PATTERN, NodeLabel.CODE_SMELL, NodeLabel.SECURITY_ISSUE}
+ ),
+}
+
+# Groups enabled when the user configures nothing. Add-ons (io) are opt-in.
+DEFAULT_CAPTURE_GROUPS: frozenset[CaptureGroup] = frozenset(
+ {
+ CaptureGroup.STRUCTURE,
+ CaptureGroup.CALLS,
+ CaptureGroup.TYPES,
+ CaptureGroup.IMPORTS,
+ }
+)
+
+CAPTURE_TOKEN_ALL = "all"
+CAPTURE_TOKEN_NONE = "none"
+CAPTURE_DROP_PREFIX = "-"
+CAPTURE_ADD_PREFIX = "+"
+CAPTURE_TOKEN_SEPARATORS = ",; "
+
+_capture_covered = frozenset().union(*CAPTURE_GROUP_RELS.values())
+_capture_missing = set(RelationshipType) - _capture_covered
+if _capture_missing:
+ raise RuntimeError(
+ f"RelationshipType(s) missing from CAPTURE_GROUP_RELS: {_capture_missing}. "
+ "Every RelationshipType MUST belong to exactly one capture group."
+ )
+
+
+class AuditCheck(StrEnum):
+ ORPHAN_NODE = "orphan_node"
+ UNDOCUMENTED_LABEL = "undocumented_label"
+ UNDOCUMENTED_PROPERTY = "undocumented_property"
+ MISSING_REQUIRED_PROPERTY = "missing_required_property"
+ UNDOCUMENTED_RELATIONSHIP = "undocumented_relationship"
+ DANGLING_RELATIONSHIP = "dangling_relationship"
+
+
+# Graph audit violation details (issue #646)
+AUDIT_DETAIL_ORPHAN = "{label} '{key}' has no relationships"
+AUDIT_DETAIL_UNDOCUMENTED_LABEL = "label '{label}' is not documented in NODE_SCHEMAS"
+AUDIT_DETAIL_UNDOCUMENTED_PROPERTY = (
+ "{label} '{key}' has undocumented property '{prop}'"
+)
+AUDIT_DETAIL_MISSING_REQUIRED = "{label} '{key}' is missing required property '{prop}'"
+AUDIT_DETAIL_UNDOCUMENTED_RELATIONSHIP = (
+ "({from_label})-[:{rel_type}]->({to_label}) is not documented"
+ " in RELATIONSHIP_SCHEMAS"
+)
+AUDIT_DETAIL_DANGLING = (
+ "({from_label} '{from_key}')-[:{rel_type}]->({to_label} '{to_key}')"
+ " references a nonexistent node and would be dropped by the database"
+)
+
+# Live-graph audit details (doctor)
+AUDIT_DETAIL_ORPHAN_COUNT = "{count} {label} node(s) have no relationships"
+AUDIT_DETAIL_UNDOCUMENTED_PROPERTY_LIVE = (
+ "{label} nodes carry undocumented property '{prop}'"
+)
+AUDIT_DETAIL_MISSING_REQUIRED_LIVE = (
+ "{count} {label} node(s) are missing required properties"
+)
+
+# Node schema property-string tokens ("{name: string, extension: string?}")
+SCHEMA_PROPS_BRACES = "{}"
+SCHEMA_OPTIONAL_SUFFIX = "?"
+
+NODE_PROJECT = NodeLabel.PROJECT
+
+KEY_PARAMETERS = "parameters"
+KEY_DECORATORS = "decorators"
+KEY_MODIFIERS = "modifiers"
+KEY_DOCSTRING = "docstring"
+KEY_IS_EXPORTED = "is_exported"
+# Marks a method that overrides a method of an EXTERNAL stdlib base class
+# (click's textwrap.TextWrapper subclass): invoked by the base's machinery,
+# never by first-party code, so dead-code reachability roots it.
+KEY_OVERRIDES_EXTERNAL = "overrides_external"
+
+CYPHER_DEFAULT_LIMIT = 50
+
+_CYPHER_EMBEDDING_BASE = """
+MATCH (m:Module)-[:DEFINES]->(n)
+WHERE (n:Function OR n:Method)
+ AND m.qualified_name STARTS WITH ($project_name + '.')
+"""
+
+CYPHER_QUERY_EMBEDDINGS = (
+ _CYPHER_EMBEDDING_BASE
+ + """RETURN id(n) AS node_id, n.qualified_name AS qualified_name,
+ n.start_line AS start_line, n.end_line AS end_line,
+ m.path AS path
+"""
+)
+
+CYPHER_QUERY_PROJECT_NODE_IDS = _CYPHER_EMBEDDING_BASE + "RETURN id(n) AS node_id\n"
+
+PAYLOAD_NODE_ID = "node_id"
+PAYLOAD_QUALIFIED_NAME = "qualified_name"
+
+CYPHER_DELETE_MODULE = (
+ # Scoped to the project: two projects in the shared graph can hold the
+ # same relative path, and a path-only match would take the sibling's
+ # module subtree with it. A repository-root __init__.py's module qn IS
+ # the bare project name (no trailing dot), so the prefix test alone
+ # would miss it.
+ "MATCH (m:Module {path: $path}) "
+ "WHERE m.qualified_name = $project_name "
+ "OR m.qualified_name STARTS WITH $project_prefix "
+ "OPTIONAL MATCH (m)-[:DEFINES|DEFINES_METHOD*0..]->(c) "
+ "DETACH DELETE m, c"
+)
+# Keyed on absolute_path: the relative path is shared across same-layout
+# projects, and a path-only delete would take the sibling's node (issue #897).
+CYPHER_DELETE_FILE = "MATCH (f:File {absolute_path: $path}) DETACH DELETE f"
+CYPHER_DELETE_FOLDER = "MATCH (f:Folder {absolute_path: $path}) DETACH DELETE f"
+CYPHER_DELETE_CALLS = "MATCH ()-[r:CALLS]->() DELETE r"
+# Removes external import-target Module nodes that no module imports anymore
+# (e.g. an imported name that was renamed/removed on an incremental rebuild).
+CYPHER_DELETE_ORPHAN_EXTERNAL_MODULES = (
+ "MATCH (m:ExternalModule) WHERE NOT (m)<--() DETACH DELETE m"
+)
+CYPHER_PROJECT_MODULE_PATHS = (
+ # The bare-name alternative covers the repository-root __init__.py,
+ # whose module qn is the project name itself.
+ "MATCH (m:Module) WHERE m.qualified_name = $project_name "
+ "OR m.qualified_name STARTS WITH $project_prefix "
+ "RETURN m.path AS path"
+)
+CYPHER_COUNT_PROJECT_MODULES = (
+ "MATCH (m:Module) WHERE m.qualified_name = $project_name "
+ "OR m.qualified_name STARTS WITH $project_prefix "
+ "RETURN count(m) AS count"
+)
+
+# Queries for orphan pruning: return all paths stored in the graph
+CYPHER_ALL_FILE_PATHS = (
+ "MATCH (f:File) RETURN f.path AS path, f.absolute_path AS absolute_path"
+)
+CYPHER_ALL_MODULE_PATHS_INTERNAL = (
+ "MATCH (m:Module) RETURN m.path AS path, m.qualified_name AS qualified_name"
+)
+CYPHER_ALL_FOLDER_PATHS = (
+ "MATCH (f:Folder) RETURN f.path AS path, f.absolute_path AS absolute_path"
+)
+
+# Rehydrate the in-memory function registry on an incremental run: returns
+# every definition node's qualified name and label so call/instantiation
+# resolution can see symbols defined in files that were not re-parsed. The
+# $project_prefix filter scopes it to the project being indexed; without it,
+# another project's same-named symbols pollute the resolver trie and the
+# bare-name fallback binds calls across the project boundary (issue #711).
+CYPHER_ALL_DEFINITION_QNS = (
+ "MATCH (n) WHERE (n:Function OR n:Method OR n:Class OR n:Interface "
+ "OR n:Enum OR n:Type OR n:Union) "
+ "AND n.qualified_name STARTS WITH $project_prefix "
+ "RETURN n.qualified_name AS qualified_name, head(labels(n)) AS label, "
+ "n.is_property AS is_property, n.is_macro AS is_macro, n.path AS path, "
+ "n.start_line AS start_line, n.end_line AS end_line"
+)
+
+# Module-level qns (plus C++20 module interfaces) for incremental runs:
+# deferred import verification must count modules in UNCHANGED files as
+# targets, or editing one file would drop cross-file IMPORTS edges.
+CYPHER_ALL_MODULE_QNS = (
+ "MATCH (n) WHERE (n:Module OR n:ModuleInterface) "
+ "AND n.qualified_name STARTS WITH $project_prefix "
+ "RETURN n.qualified_name AS qualified_name, head(labels(n)) AS label"
+)
+
+# Inbound reference edges (from unchanged files) into symbols defined in one
+# of $paths. Captured BEFORE a changed file's subtree is deleted so the exact
+# edges can be restored verbatim afterwards (issue #532, inbound half).
+# Re-resolving the callers instead would diverge from a clean index, because
+# cgr's call resolution is context-sensitive (protocol vs concrete receiver,
+# import granularity); the original edges already match a clean re-index.
+CYPHER_INBOUND_EDGES = (
+ "MATCH (caller)-[r:CALLS|REFERENCES|INSTANTIATES|IMPORTS|INHERITS|OVERRIDES]->(target) "
+ "WHERE target.path IN $paths AND caller.qualified_name IS NOT NULL "
+ "AND (caller.path IS NULL OR NOT caller.path IN $paths) "
+ "RETURN head(labels(caller)) AS caller_label, "
+ "caller.qualified_name AS caller_qn, type(r) AS rel, "
+ "head(labels(target)) AS target_label, target.qualified_name AS target_qn"
+)
+# Rehydrate class_inheritance on an incremental run: every INHERITS edge
+# (child -> base) with resolved qns, so protocol dispatch and inherited-method
+# resolution still see the hierarchy of classes defined in files that were not
+# re-parsed. Without it, editing a caller drops the protocol/inheritance
+# redirect (issue #532 residual): a call resolves to the Protocol stub instead
+# of the concrete implementer because _protocol_classes() is empty. Ordered by
+# base_index so multiple-inheritance base order matches the original source,
+# which method resolution and override attribution depend on.
+CYPHER_ALL_INHERITS = (
+ "MATCH (child)-[r:INHERITS]->(base) "
+ "WHERE child.qualified_name IS NOT NULL AND base.qualified_name IS NOT NULL "
+ "AND child.qualified_name STARTS WITH $project_prefix "
+ "RETURN child.qualified_name AS child_qn, base.qualified_name AS base_qn, "
+ "r.base_index AS base_index "
+ "ORDER BY child_qn, base_index"
+)
+KEY_CHILD_QN = "child_qn"
+KEY_BASE_QN = "base_qn"
+KEY_BASE_INDEX = "base_index"
+
+CYPHER_PARAM_PATHS = "paths"
+KEY_CALLER_LABEL = "caller_label"
+KEY_CALLER_QN = "caller_qn"
+KEY_REL = "rel"
+KEY_TARGET_LABEL = "target_label"
+KEY_TARGET_QN = "target_qn"
+
+REL_TYPE_CALLS = "CALLS"
+
+# Rel types where multiple semantically-distinct edges may exist between the
+# same node pair; these props join the MERGE key so parallel edges are not
+# collapsed at write time (issue #722). Props absent from a batch's rows are
+# dropped from the key at flush time, so resource-level FLOWS_TO (no `via`)
+# still dedups on endpoints.
+MERGE_KEY_PROPS_BY_REL: dict[str, tuple[str, ...]] = {
+ RelationshipType.FLOWS_TO.value: ("via", "kind"),
+}
+
+NODE_UNIQUE_CONSTRAINTS: dict[str, str] = {
+ label.value: key.value for label, key in _NODE_LABEL_UNIQUE_KEYS.items()
+}
+
+# Superseded unique constraints that must be dropped from existing shared
+# databases; a leftover Folder/File path constraint would keep rejecting the
+# second same-relative-path node the fix now creates (issue #897).
+LEGACY_NODE_CONSTRAINTS: tuple[tuple[str, str], ...] = (
+ (NodeLabel.FOLDER.value, UniqueKeyType.PATH.value),
+ (NodeLabel.FILE.value, UniqueKeyType.PATH.value),
+)
+
+CYPHER_MEMORY_LIMIT_SUFFIX = " QUERY MEMORY LIMIT {mb} MB"
+CYPHER_MEMORY_LIMIT_TOKEN = "QUERY MEMORY LIMIT"
diff --git a/codebase_rag/constants/health.py b/codebase_rag/constants/health.py
new file mode 100644
index 000000000..a59f73906
--- /dev/null
+++ b/codebase_rag/constants/health.py
@@ -0,0 +1,62 @@
+# Health-check statuses and messages.
+
+HEALTH_CHECK_DOCKER_RUNNING = "Docker daemon is running"
+HEALTH_CHECK_DOCKER_NOT_RUNNING = "Docker daemon is not running"
+HEALTH_CHECK_DOCKER_RUNNING_MSG = "Running (version {version})"
+HEALTH_CHECK_DOCKER_NOT_RESPONDING_MSG = "Not responding"
+HEALTH_CHECK_DOCKER_NOT_INSTALLED_MSG = "Not installed"
+HEALTH_CHECK_DOCKER_NOT_IN_PATH = "docker command not found in PATH"
+HEALTH_CHECK_DOCKER_TIMEOUT_MSG = "Check timed out"
+HEALTH_CHECK_DOCKER_TIMEOUT_ERROR = (
+ "The 'docker info' command took more than 5 seconds to respond."
+)
+HEALTH_CHECK_DOCKER_FAILED_MSG = "Check failed"
+HEALTH_CHECK_DOCKER_EXIT_CODE = "Non-zero exit code"
+
+HEALTH_CHECK_MEMGRAPH_SUCCESSFUL = "Memgraph connection successful"
+HEALTH_CHECK_MEMGRAPH_FAILED = "Memgraph connection failed"
+HEALTH_CHECK_MEMGRAPH_CONNECTED_MSG = "Connected and responsive at {host}:{port}"
+HEALTH_CHECK_MEMGRAPH_CONNECTION_FAILED_MSG = "Connection or query failed"
+HEALTH_CHECK_MEMGRAPH_UNEXPECTED_FAILURE_MSG = "Unexpected failure"
+HEALTH_CHECK_MEMGRAPH_ERROR = "Memgraph error: {error}"
+HEALTH_CHECK_MEMGRAPH_QUERY = "RETURN 1 AS test;"
+
+HEALTH_CHECK_GRAPH_INTEGRITY_OK = "Graph structural integrity verified"
+HEALTH_CHECK_GRAPH_INTEGRITY_FAILED = "Graph structural integrity violations"
+HEALTH_CHECK_GRAPH_INTEGRITY_OK_MSG = "No orphans or schema violations"
+HEALTH_CHECK_GRAPH_INTEGRITY_VIOLATIONS_MSG = "{count} violation(s) found"
+HEALTH_CHECK_GRAPH_INTEGRITY_ERROR_MSG = "Audit queries failed"
+HEALTH_CHECK_GRAPH_INTEGRITY_SEPARATOR = "; "
+
+HEALTH_CHECK_API_KEY_SET = "{display_name} API key is set"
+HEALTH_CHECK_API_KEY_NOT_SET = "{display_name} API key is not set"
+HEALTH_CHECK_API_KEY_CONFIGURED = "Configured"
+HEALTH_CHECK_API_KEY_NOT_CONFIGURED = "Not set"
+HEALTH_CHECK_API_KEY_MISSING_MSG = (
+ "Set the {env_name} environment variable or configure it in your settings."
+)
+
+HEALTH_CHECK_TOOL_INSTALLED = "{tool_name} is installed"
+HEALTH_CHECK_TOOL_NOT_INSTALLED = "{tool_name} is not installed"
+HEALTH_CHECK_TOOL_INSTALLED_MSG = "Installed ({path})"
+HEALTH_CHECK_TOOL_NOT_IN_PATH_MSG = "'{cmd}' not found in PATH"
+HEALTH_CHECK_TOOL_TIMEOUT_MSG = "Check timed out"
+HEALTH_CHECK_TOOL_TIMEOUT_ERROR = (
+ "The command to find '{cmd}' took more than 4 seconds to respond."
+)
+HEALTH_CHECK_TOOL_FAILED_MSG = "Check failed"
+
+HEALTH_CHECK_TOOLS = [
+ ("GEMINI_API_KEY", "Gemini"),
+ ("OPENAI_API_KEY", "OpenAI"),
+ ("ORCHESTRATOR_API_KEY", "Orchestrator"),
+ ("CYPHER_API_KEY", "Cypher"),
+]
+
+HEALTH_CHECK_EXTERNAL_TOOLS = [
+ ("ripgrep", "rg"),
+ ("cmake", "cmake"),
+]
+
+SHELL_CMD_WHERE = "where"
+SHELL_CMD_WHICH = "which"
diff --git a/codebase_rag/constants/lang_tooling.py b/codebase_rag/constants/lang_tooling.py
new file mode 100644
index 000000000..b9a5548bf
--- /dev/null
+++ b/codebase_rag/constants/lang_tooling.py
@@ -0,0 +1,157 @@
+# Add-language grammar tooling messages, prompts, and file names.
+
+LANG_GRAMMARS_DIR = "grammars"
+LANG_CONFIG_FILE = "codebase_rag/language_spec.py"
+LANG_TREE_SITTER_JSON = "tree-sitter.json"
+LANG_NODE_TYPES_JSON = "node-types.json"
+LANG_SRC_DIR = "src"
+LANG_GIT_MODULES_PATH = ".git/modules/{path}"
+LANG_DEFAULT_GRAMMAR_URL = "https://github.com/tree-sitter/tree-sitter-{name}"
+LANG_TREE_SITTER_URL_MARKER = "github.com/tree-sitter/tree-sitter"
+
+LANG_DEFAULT_FUNCTION_NODES = ("function_definition", "method_definition")
+LANG_DEFAULT_CLASS_NODES = ("class_declaration",)
+LANG_DEFAULT_MODULE_NODES = ("compilation_unit",)
+LANG_DEFAULT_CALL_NODES = ("invocation_expression",)
+LANG_FALLBACK_METHOD_NODE = "method_declaration"
+
+LANG_FUNCTION_KEYWORDS = frozenset(
+ {
+ "function",
+ "method",
+ "constructor",
+ "destructor",
+ "lambda",
+ "arrow_function",
+ "anonymous_function",
+ "closure",
+ }
+)
+LANG_CLASS_KEYWORDS = frozenset(
+ {
+ "class",
+ "interface",
+ "struct",
+ "enum",
+ "trait",
+ "object",
+ "type",
+ "impl",
+ "union",
+ }
+)
+LANG_CALL_KEYWORDS = frozenset({"call", "invoke", "invocation"})
+LANG_MODULE_KEYWORDS = frozenset(
+ {"program", "source_file", "compilation_unit", "module", "chunk"}
+)
+LANG_EXCLUSION_KEYWORDS = frozenset({"access", "call"})
+
+LANG_MSG_USING_DEFAULT_URL = "Using default tree-sitter URL: {url}"
+LANG_MSG_CUSTOM_URL_WARNING = (
+ "WARNING: You are adding a grammar from a custom URL. "
+ "This may execute code from the repository. Only proceed if you trust the source."
+)
+LANG_MSG_ADDING_SUBMODULE = "Adding submodule from {url}..."
+LANG_MSG_SUBMODULE_SUCCESS = "Successfully added submodule at {path}"
+LANG_MSG_SUBMODULE_EXISTS = (
+ "Submodule already exists at {path}. Forcing re-installation..."
+)
+LANG_MSG_REMOVING_ENTRY = " -> Removing existing submodule entry..."
+LANG_MSG_READDING_SUBMODULE = " -> Re-adding submodule..."
+LANG_MSG_REINSTALL_SUCCESS = "Successfully re-installed submodule at {path}"
+LANG_MSG_AUTO_DETECTED_LANG = "Auto-detected language: {name}"
+LANG_MSG_USING_LANG_NAME = "Using language name: {name}"
+LANG_MSG_AUTO_DETECTED_EXT = "Auto-detected file extensions: {extensions}"
+LANG_MSG_FOUND_NODE_TYPES = "Found {count} total node types in grammar"
+LANG_MSG_SEMANTIC_CATEGORIES = "Tree-sitter semantic categories:"
+LANG_MSG_CATEGORY_FORMAT = " {category}: {subtypes} ({count} total)"
+LANG_MSG_MAPPED_CATEGORIES = "\nMapped to our categories:"
+LANG_MSG_FUNCTIONS = "Functions: {nodes}"
+LANG_MSG_CLASSES = "Classes: {nodes}"
+LANG_MSG_MODULES = "Modules: {nodes}"
+LANG_MSG_CALLS = "Calls: {nodes}"
+LANG_MSG_LANG_ADDED = "\nLanguage '{name}' has been added to the configuration!"
+LANG_MSG_UPDATED_CONFIG = "Updated {path}"
+LANG_MSG_REVIEW_PROMPT = "Please review the detected node types:"
+LANG_MSG_REVIEW_HINT = " The auto-detection is good but may need manual adjustments."
+LANG_MSG_EDIT_HINT = " Edit the configuration in: {path}"
+LANG_MSG_COMMON_ISSUES = "Look for these common issues:"
+LANG_MSG_ISSUE_MISCLASSIFIED = (
+ " - Remove misclassified types (e.g., table_constructor in functions)"
+)
+LANG_MSG_ISSUE_MISSING = " - Add missing types that should be included"
+LANG_MSG_ISSUE_CLASS_TYPES = (
+ " - Verify class_node_types includes all relevant class-like constructs"
+)
+LANG_MSG_ISSUE_CALL_TYPES = (
+ " - Check call_node_types covers all function call patterns"
+)
+LANG_MSG_LIST_HINT = (
+ "You can run 'cgr language list-languages' to see the current config."
+)
+LANG_MSG_LANG_NOT_FOUND = "Language '{name}' not found."
+LANG_MSG_AVAILABLE_LANGS = "Available languages: {langs}"
+LANG_MSG_REMOVED_FROM_CONFIG = "Removed language '{name}' from configuration file."
+LANG_MSG_REMOVING_SUBMODULE = "Removing git submodule '{path}'..."
+LANG_MSG_CLEANED_MODULES = "Cleaned up git modules directory: {path}"
+LANG_MSG_SUBMODULE_REMOVED = "Successfully removed submodule '{path}'"
+LANG_MSG_NO_SUBMODULE = "No submodule found at '{path}'"
+LANG_MSG_KEEPING_SUBMODULE = "Keeping submodule (--keep-submodule flag used)"
+LANG_MSG_LANG_REMOVED = "Language '{name}' has been removed successfully!"
+LANG_MSG_NO_MODULES_DIR = "No grammars modules directory found."
+LANG_MSG_NO_GITMODULES = "No .gitmodules file found."
+LANG_MSG_NO_ORPHANS = "No orphaned modules found!"
+LANG_MSG_FOUND_ORPHANS = "Found {count} orphaned module(s): {modules}"
+LANG_MSG_REMOVED_ORPHAN = "Removed orphaned module: {module}"
+LANG_MSG_CLEANUP_COMPLETE = "Cleanup complete!"
+LANG_MSG_CLEANUP_CANCELLED = "Cleanup cancelled."
+
+LANG_ERR_MISSING_ARGS = "Error: Either language_name or --grammar-url must be provided"
+LANG_ERR_REINSTALL_FAILED = "Failed to reinstall submodule: {error}"
+LANG_ERR_MANUAL_REMOVE_HINT = "You may need to remove it manually and try again:"
+LANG_ERR_REPO_NOT_FOUND = "Error: Repository not found at {url}"
+LANG_ERR_CUSTOM_URL_HINT = "Try using a custom URL with: --grammar-url "
+LANG_ERR_GIT = "Git error: {error}"
+LANG_ERR_NODE_TYPES_WARNING = (
+ "Warning: node-types.json not found in any expected location for {name}"
+)
+LANG_ERR_TREE_SITTER_JSON_WARNING = "Warning: tree-sitter.json not found in {path}"
+LANG_ERR_NO_GRAMMARS_WARNING = "Warning: No grammars found in tree-sitter.json"
+LANG_ERR_PARSE_NODE_TYPES = "Error parsing node-types.json: {error}"
+LANG_ERR_UPDATE_CONFIG = "Error updating config file: {error}"
+LANG_ERR_CONFIG_NOT_FOUND = "Could not find LANGUAGE_SPECS dictionary end"
+LANG_ERR_REMOVE_CONFIG = "Failed to update config file: {error}"
+LANG_ERR_REMOVE_SUBMODULE = "Failed to remove submodule: {error}"
+
+LANG_PROMPT_LANGUAGE_NAME = "Language name (e.g., 'c-sharp', 'python')"
+LANG_PROMPT_COMMON_NAME = "What is the common name for this language?"
+LANG_PROMPT_EXTENSIONS = (
+ "What file extensions should be associated with this language? (comma-separated)"
+)
+LANG_PROMPT_FUNCTIONS = "Select nodes representing FUNCTIONS (comma-separated)"
+LANG_PROMPT_CLASSES = "Select nodes representing CLASSES (comma-separated)"
+LANG_PROMPT_MODULES = "Select nodes representing MODULES (comma-separated)"
+LANG_PROMPT_CALLS = "Select nodes representing FUNCTION CALLS (comma-separated)"
+LANG_PROMPT_CONTINUE = "Do you want to continue?"
+LANG_PROMPT_REMOVE_ORPHANS = "Do you want to remove these orphaned modules?"
+
+LANG_FALLBACK_MANUAL_ADD = (
+ "FALLBACK: Please manually add the following entry to "
+ "'LANGUAGE_SPECS' in 'codebase_rag/language_spec.py':"
+)
+
+LANG_TABLE_TITLE = "Configured Languages"
+LANG_TABLE_COL_LANGUAGE = "Language"
+LANG_TABLE_COL_EXTENSIONS = "Extensions"
+LANG_TABLE_COL_FUNCTION_TYPES = "Function Types"
+LANG_TABLE_COL_CLASS_TYPES = "Class Types"
+LANG_TABLE_COL_CALL_TYPES = "Call Types"
+LANG_TABLE_PLACEHOLDER = "—"
+
+LANG_MSG_AVAILABLE_NODES = "Available nodes for mapping:"
+LANG_ELLIPSIS = "..."
+LANG_GIT_SUFFIX = ".git"
+LANG_GITMODULES_FILE = ".gitmodules"
+LANG_CALL_KEYWORD_EXCLUDE = "call"
+
+LANG_GITMODULES_REGEX = r"path = (grammars/tree-sitter-[^\\n]+)"
diff --git a/codebase_rag/constants/languages.py b/codebase_rag/constants/languages.py
new file mode 100644
index 000000000..53dfa8938
--- /dev/null
+++ b/codebase_rag/constants/languages.py
@@ -0,0 +1,369 @@
+# Supported languages, file extensions, metadata, and grammar modules.
+
+from enum import StrEnum
+from typing import NamedTuple
+
+BINARY_EXTENSIONS: frozenset[str] = frozenset(
+ {
+ ".pdf",
+ ".png",
+ ".jpg",
+ ".jpeg",
+ ".gif",
+ ".bmp",
+ ".ico",
+ ".tiff",
+ ".webp",
+ }
+)
+
+# Source file extensions by language
+EXT_PY = ".py"
+EXT_JS = ".js"
+EXT_JSX = ".jsx"
+EXT_MJS = ".mjs"
+EXT_CJS = ".cjs"
+EXT_TS = ".ts"
+EXT_MTS = ".mts"
+EXT_CTS = ".cts"
+EXT_TSX = ".tsx"
+EXT_RS = ".rs"
+EXT_GO = ".go"
+EXT_SCALA = ".scala"
+EXT_SC = ".sc"
+EXT_JAVA = ".java"
+EXT_CLASS = ".class"
+EXT_CPP = ".cpp"
+EXT_H = ".h"
+EXT_HPP = ".hpp"
+EXT_CC = ".cc"
+EXT_CXX = ".cxx"
+EXT_HXX = ".hxx"
+EXT_HH = ".hh"
+EXT_IXX = ".ixx"
+EXT_CPPM = ".cppm"
+EXT_CCM = ".ccm"
+EXT_C = ".c"
+EXT_PHP = ".php"
+EXT_LUA = ".lua"
+EXT_CS = ".cs"
+EXT_DART = ".dart"
+
+# File extension tuples by language
+PY_EXTENSIONS = (EXT_PY,)
+JS_EXTENSIONS = (EXT_JS, EXT_JSX, EXT_MJS, EXT_CJS)
+TS_EXTENSIONS = (EXT_TS, EXT_MTS, EXT_CTS)
+TSX_EXTENSIONS = (EXT_TSX,)
+JS_TS_ALL_EXTENSIONS = (
+ EXT_JS,
+ EXT_JSX,
+ EXT_MJS,
+ EXT_CJS,
+ EXT_TS,
+ EXT_MTS,
+ EXT_CTS,
+ EXT_TSX,
+)
+RS_EXTENSIONS = (EXT_RS,)
+GO_EXTENSIONS = (EXT_GO,)
+SCALA_EXTENSIONS = (EXT_SCALA, EXT_SC)
+JAVA_EXTENSIONS = (EXT_JAVA,)
+C_EXTENSIONS = (EXT_C,)
+CPP_EXTENSIONS = (
+ EXT_CPP,
+ EXT_H,
+ EXT_HPP,
+ EXT_CC,
+ EXT_CXX,
+ EXT_HXX,
+ EXT_HH,
+ EXT_IXX,
+ EXT_CPPM,
+ EXT_CCM,
+)
+# Translation-unit sources: only these can define a linkable OS entry point;
+# headers and C++ module interface files never are the entry unit.
+C_CPP_SOURCE_EXTENSIONS = (EXT_C, EXT_CPP, EXT_CC, EXT_CXX)
+PHP_EXTENSIONS = (EXT_PHP,)
+LUA_EXTENSIONS = (EXT_LUA,)
+CS_EXTENSIONS = (EXT_CS,)
+DART_EXTENSIONS = (EXT_DART,)
+
+# Package indicator files
+PKG_INIT_PY = "__init__.py"
+PKG_CARGO_TOML = "Cargo.toml"
+PKG_CMAKE_LISTS = "CMakeLists.txt"
+PKG_MAKEFILE = "Makefile"
+PKG_VCXPROJ_GLOB = "*.vcxproj"
+PKG_CONANFILE = "conanfile.txt"
+PKG_PUBSPEC_YAML = "pubspec.yaml"
+
+
+class CppFrontend(StrEnum):
+ TREESITTER = "treesitter"
+ LIBCLANG = "libclang"
+ HYBRID = "hybrid"
+
+
+class CSharpFrontend(StrEnum):
+ # AUTO resolves at run time: HYBRID where a dotnet toolchain is on PATH,
+ # TREESITTER otherwise (resolve_csharp_frontend). The parser fingerprint
+ # records the RESOLVED mode, so dotnet and non-dotnet graphs never share
+ # an identity.
+ AUTO = "auto"
+ TREESITTER = "treesitter"
+ ROSLYN = "roslyn"
+ HYBRID = "hybrid"
+
+
+# JS/TS import specifier schemes naming genuinely external code (node
+# builtins, registries, URLs). Any OTHER scheme (`ext:` deno aliases,
+# bundler virtual modules) points at first-party code under a non-file-path
+# name, so its unresolved calls defer to the trie.
+JS_EXTERNAL_IMPORT_SCHEMES: frozenset[str] = frozenset(
+ {"node", "npm", "jsr", "bun", "http", "https", "data", "file", "blob"}
+)
+# Module extensions stripped when turning a tsconfig `paths` target into a
+# module qn (`src/util.ts` -> `src/util`), longest first so `.d.ts`-like
+# suffixes match before the bare `.ts`.
+JS_TS_MODULE_EXTENSIONS: tuple[str, ...] = (
+ ".d.mts",
+ ".d.cts",
+ ".d.ts",
+ ".tsx",
+ ".mts",
+ ".cts",
+ ".ts",
+ ".jsx",
+ ".mjs",
+ ".cjs",
+ ".js",
+)
+TSCONFIG_FILENAMES: tuple[str, ...] = (
+ "tsconfig.json",
+ "tsconfig.base.json",
+ "jsconfig.json",
+)
+# When searching subdirectories for tsconfig files (monorepo `frontend/`,
+# `packages/*`), skip dependency/build/VCS trees: their tsconfigs carry
+# unrelated aliases and node_modules holds thousands of them.
+TS_ALIAS_SKIP_DIRS: frozenset[str] = frozenset(
+ {"node_modules", "dist", "build", "out", ".git"}
+)
+JS_INDEX_STEM = "index"
+TS_COMPILER_OPTIONS_KEY = "compilerOptions"
+TS_PATHS_KEY = "paths"
+TS_BASE_URL_KEY = "baseUrl"
+PATH_RELATIVE_PREFIX = "./"
+PATH_PARENT_PREFIX = "../"
+CPP_IMPORT_PARTITION_PREFIX = "import :"
+CPP_PARTITION_PREFIX = "partition_"
+
+
+class SupportedLanguage(StrEnum):
+ PYTHON = "python"
+ JS = "javascript"
+ TS = "typescript"
+ TSX = "tsx"
+ RUST = "rust"
+ GO = "go"
+ SCALA = "scala"
+ JAVA = "java"
+ C = "c"
+ CPP = "cpp"
+ PHP = "php"
+ LUA = "lua"
+ CSHARP = "c_sharp"
+ DART = "dart"
+
+
+class LanguageStatus(StrEnum):
+ FULL = "Fully Supported"
+ DEV = "In Development"
+
+
+class LanguageMetadata(NamedTuple):
+ status: LanguageStatus
+ additional_features: str
+ display_name: str
+
+
+LANGUAGE_METADATA: dict[SupportedLanguage, LanguageMetadata] = {
+ SupportedLanguage.PYTHON: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Type inference, decorators, nested functions",
+ "Python",
+ ),
+ SupportedLanguage.JS: LanguageMetadata(
+ LanguageStatus.FULL,
+ "ES6 modules, CommonJS, prototype methods, object methods, arrow functions",
+ "JavaScript",
+ ),
+ SupportedLanguage.TS: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Interfaces, type aliases, enums, namespaces, ES6/CommonJS modules",
+ "TypeScript",
+ ),
+ SupportedLanguage.TSX: LanguageMetadata(
+ LanguageStatus.FULL,
+ "All TypeScript features plus JSX elements and components",
+ "TypeScript (TSX)",
+ ),
+ SupportedLanguage.C: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Functions, structs, unions, enums, preprocessor includes",
+ "C",
+ ),
+ SupportedLanguage.CPP: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Constructors, destructors, operator overloading, templates, lambdas, C++20 modules, namespaces, preprocessor macros",
+ "C++",
+ ),
+ SupportedLanguage.LUA: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Local/global functions, metatables, closures, coroutines",
+ "Lua",
+ ),
+ SupportedLanguage.RUST: LanguageMetadata(
+ LanguageStatus.FULL,
+ "impl blocks, associated functions, macro_rules! macros",
+ "Rust",
+ ),
+ SupportedLanguage.JAVA: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Generics, annotations, modern features (records/sealed classes), concurrency, reflection",
+ "Java",
+ ),
+ SupportedLanguage.GO: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Receiver methods with cross-file binding, structs, interfaces, type declarations, function-local types",
+ "Go",
+ ),
+ SupportedLanguage.SCALA: LanguageMetadata(
+ LanguageStatus.DEV,
+ "Case classes, objects",
+ "Scala",
+ ),
+ SupportedLanguage.PHP: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Classes, interfaces, traits, enums, namespaces, PHP 8 attributes",
+ "PHP",
+ ),
+ SupportedLanguage.CSHARP: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Namespaces (block and file-scoped), classes/structs/records/interfaces/enums, generics, inheritance/interfaces/overrides, typed call resolution with overloads, using directives",
+ "C#",
+ ),
+ SupportedLanguage.DART: LanguageMetadata(
+ LanguageStatus.FULL,
+ "Classes, mixins, extensions, enhanced enums, factory/named constructors, Flutter widgets, package/relative/dart: imports, part directives, pubspec dependencies",
+ "Dart",
+ ),
+}
+
+# Index file names
+INDEX_INIT = "__init__"
+INDEX_INDEX = "index"
+INDEX_MOD = "mod"
+INDEX_LUA_INIT = "init"
+
+# File stems whose module is importable through the CONTAINING directory's
+# name: pkg/__init__.py, shared/index.js, utils/mod.rs, storage/init.lua.
+MODULE_INDEX_FILE_STEMS = frozenset(
+ {INDEX_INIT, INDEX_INDEX, INDEX_MOD, INDEX_LUA_INIT}
+)
+
+# Parser loader paths and args
+GRAMMARS_DIR = "grammars"
+TREE_SITTER_PREFIX = "tree-sitter-"
+TREE_SITTER_MODULE_PREFIX = "tree_sitter_"
+BINDINGS_DIR = "bindings"
+SETUP_PY = "setup.py"
+BUILD_EXT_CMD = "build_ext"
+INPLACE_FLAG = "--inplace"
+LANG_ATTR_PREFIX = "language_"
+LANG_ATTR_TYPESCRIPT = "language_typescript"
+LANG_ATTR_TSX = "language_tsx"
+LANG_ATTR_PHP = "language_php"
+
+
+class TreeSitterModule(StrEnum):
+ PYTHON = "tree_sitter_python"
+ JS = "tree_sitter_javascript"
+ TS = "tree_sitter_typescript"
+ RUST = "tree_sitter_rust"
+ GO = "tree_sitter_go"
+ SCALA = "tree_sitter_scala"
+ JAVA = "tree_sitter_java"
+ C = "tree_sitter_c"
+ CPP = "tree_sitter_cpp"
+ LUA = "tree_sitter_lua"
+ PHP = "tree_sitter_php"
+ CSHARP = "tree_sitter_c_sharp"
+ DART = "tree_sitter_dart"
+
+
+# Directory names with a context-dependent ignore: `bin` is build output
+# everywhere EXCEPT Cargo's first-party src/bin/ binary layout.
+DIR_BIN = "bin"
+DIR_SRC = "src"
+
+# Patterns detected at repo root and offered as exclude candidates (user picks which)
+IGNORE_PATTERNS = frozenset(
+ {
+ ".cache",
+ ".claude",
+ # Android NDK per-ABI CMake build cache; ships compiler-probe
+ # sources (CMakeCCompilerId.c) that index as project code.
+ ".cxx",
+ # Dart/Flutter tool cache (package_config, generated plugin code).
+ ".dart_tool",
+ ".eclipse",
+ ".eggs",
+ ".env",
+ ".git",
+ ".gradle",
+ ".hg",
+ ".idea",
+ ".maven",
+ ".mypy_cache",
+ ".nox",
+ ".npm",
+ ".nyc_output",
+ ".pnpm-store",
+ ".pytest_cache",
+ ".qdrant_code_embeddings",
+ ".ruff_cache",
+ ".svn",
+ ".tmp",
+ ".tox",
+ ".venv",
+ ".vs",
+ ".vscode",
+ ".yarn",
+ "__pycache__",
+ "bin",
+ "bower_components",
+ "build",
+ "coverage",
+ "dist",
+ "env",
+ "htmlcov",
+ "node_modules",
+ "obj",
+ "out",
+ "Pods",
+ "site-packages",
+ "target",
+ "temp",
+ "tmp",
+ "vendor",
+ "venv",
+ }
+)
+IGNORE_SUFFIXES = frozenset(
+ {".tmp", "~", ".pyc", ".pyo", ".o", ".a", ".so", ".dll", ".class"}
+)
+
+# pathspec style for .cgrignore / --exclude patterns (#495).
+GITWILDMATCH_STYLE = "gitignore"
diff --git a/codebase_rag/constants/mcp.py b/codebase_rag/constants/mcp.py
new file mode 100644
index 000000000..93ab14917
--- /dev/null
+++ b/codebase_rag/constants/mcp.py
@@ -0,0 +1,105 @@
+# MCP server tool names, schema fields, and messages.
+
+from enum import StrEnum
+
+
+class MCPToolName(StrEnum):
+ LIST_PROJECTS = "list_projects"
+ DELETE_PROJECT = "delete_project"
+ WIPE_DATABASE = "wipe_database"
+ INDEX_REPOSITORY = "index_repository"
+ UPDATE_REPOSITORY = "update_repository"
+ QUERY_CODE_GRAPH = "query_code_graph"
+ GET_CODE_SNIPPET = "get_code_snippet"
+ SURGICAL_REPLACE_CODE = "surgical_replace_code"
+ READ_FILE = "read_file"
+ WRITE_FILE = "write_file"
+ LIST_DIRECTORY = "list_directory"
+ SEMANTIC_SEARCH = "semantic_search"
+ STRUCTURAL_SEARCH = "structural_search"
+ STRUCTURAL_REPLACE = "structural_replace"
+ ASK_AGENT = "ask_agent"
+
+
+class MCPTransport(StrEnum):
+ STDIO = "stdio"
+ HTTP = "http"
+
+
+class MCPEnvVar(StrEnum):
+ TARGET_REPO_PATH = "TARGET_REPO_PATH"
+ CLAUDE_PROJECT_ROOT = "CLAUDE_PROJECT_ROOT"
+ PWD = "PWD"
+
+
+class MCPSchemaType(StrEnum):
+ OBJECT = "object"
+ STRING = "string"
+ INTEGER = "integer"
+ BOOLEAN = "boolean"
+
+
+class MCPSchemaField(StrEnum):
+ TYPE = "type"
+ PROPERTIES = "properties"
+ REQUIRED = "required"
+ DESCRIPTION = "description"
+ DEFAULT = "default"
+
+
+class MCPParamName(StrEnum):
+ PROJECT_NAME = "project_name"
+ CONFIRM = "confirm"
+ NATURAL_LANGUAGE_QUERY = "natural_language_query"
+ QUALIFIED_NAME = "qualified_name"
+ FILE_PATH = "file_path"
+ TARGET_CODE = "target_code"
+ REPLACEMENT_CODE = "replacement_code"
+ OFFSET = "offset"
+ LIMIT = "limit"
+ CONTENT = "content"
+ DIRECTORY_PATH = "directory_path"
+ TOP_K = "top_k"
+ QUESTION = "question"
+ PATTERN = "pattern"
+ REWRITE = "rewrite"
+ LANGUAGE = "language"
+ DRY_RUN = "dry_run"
+
+
+# MCP server constants
+MCP_SERVER_NAME = "code-graph-rag"
+MCP_CONTENT_TYPE_TEXT = "text"
+MCP_DEFAULT_DIRECTORY = "."
+MCP_JSON_INDENT = 2
+MCP_LOG_LEVEL_INFO = "INFO"
+MCP_LOG_FORMAT = "{time:YYYY-MM-DD HH:mm:ss} | {level: <8} | {message}"
+MCP_PAGINATION_HEADER = "# Lines {start}-{end} of {total}\n"
+
+# MCP response messages
+MCP_INDEX_SUCCESS = "Successfully indexed repository at {path}. Knowledge graph has been updated (previous data cleared)."
+MCP_INDEX_SUCCESS_PROJECT = "Successfully indexed repository at {path}. Project '{project_name}' has been updated."
+MCP_INDEX_ERROR = "Error indexing repository: {error}"
+MCP_WRITE_SUCCESS = "Successfully wrote file: {path}"
+MCP_UNKNOWN_TOOL_ERROR = "Unknown tool: {name}"
+MCP_TOOL_EXEC_ERROR = "Error executing tool '{name}': {error}"
+MCP_UPDATE_SUCCESS = "Successfully updated repository at {path} (no database wipe)."
+MCP_UPDATE_ERROR = "Error updating repository: {error}"
+MCP_SEMANTIC_NOT_AVAILABLE_RESPONSE = (
+ "Semantic search is not available. Install with: uv sync --extra semantic"
+)
+MCP_ASK_AGENT_ERROR = "Error running ask_agent: {error}"
+MCP_PROJECT_DELETED = "Successfully deleted project '{project_name}'."
+MCP_WIPE_CANCELLED = "Database wipe cancelled. Set confirm=true to proceed."
+MCP_WIPE_SUCCESS = "Database completely wiped. All projects have been removed."
+MCP_WIPE_ERROR = "Error wiping database: {error}"
+
+# MCP dict keys and values
+MCP_KEY_RESULTS = "results"
+MCP_KEY_ERROR = "error"
+MCP_KEY_FOUND = "found"
+MCP_KEY_ERROR_MESSAGE = "error_message"
+MCP_KEY_QUERY_USED = "query_used"
+MCP_KEY_SUMMARY = "summary"
+MCP_NOT_AVAILABLE = "N/A"
+MCP_TOOL_NAME_QUERY = "query"
diff --git a/codebase_rag/constants/providers.py b/codebase_rag/constants/providers.py
new file mode 100644
index 000000000..7caa7bd34
--- /dev/null
+++ b/codebase_rag/constants/providers.py
@@ -0,0 +1,134 @@
+# LLM/embedding provider defaults, env vars, and model metadata.
+
+from enum import StrEnum
+
+
+class ModelRole(StrEnum):
+ ORCHESTRATOR = "orchestrator"
+ CYPHER = "cypher"
+
+
+class Provider(StrEnum):
+ OLLAMA = "ollama"
+ ANTHROPIC = "anthropic"
+ OPENAI = "openai"
+ GOOGLE = "google"
+ AZURE = "azure"
+ LITELLM_PROXY = "litellm_proxy"
+ MINIMAX = "minimax"
+
+
+DEFAULT_MODEL_ROLE = "model"
+
+DEFAULT_REGION = "us-central1"
+DEFAULT_MODEL = "llama3.2"
+DEFAULT_API_KEY = "ollama"
+
+ENV_OPENAI_API_KEY = "OPENAI_API_KEY"
+ENV_GOOGLE_API_KEY = "GOOGLE_API_KEY"
+ENV_ANTHROPIC_API_KEY = "ANTHROPIC_API_KEY"
+ENV_AZURE_API_KEY = "AZURE_API_KEY"
+ENV_AZURE_ENDPOINT = "AZURE_OPENAI_ENDPOINT"
+ENV_AZURE_API_VERSION = "AZURE_API_VERSION"
+ENV_MINIMAX_API_KEY = "MINIMAX_API_KEY"
+
+
+class GoogleProviderType(StrEnum):
+ GLA = "gla"
+ VERTEX = "vertex"
+
+
+# Provider endpoints
+OPENAI_DEFAULT_ENDPOINT = "https://api.openai.com/v1"
+MINIMAX_DEFAULT_ENDPOINT = "https://api.minimax.io/v1"
+MINIMAX_ANTHROPIC_SDK_PATH = "/anthropic"
+OLLAMA_HEALTH_PATH = "/api/tags"
+GOOGLE_CLOUD_SCOPE = "https://www.googleapis.com/auth/cloud-platform"
+V1_PATH = "/v1"
+
+HTTP_OK = 200
+
+UNIXCODER_MODEL = "microsoft/unixcoder-base"
+EMBEDDING_DEFAULT_BATCH_SIZE = 64
+EMBEDDING_CACHE_FILENAME = ".embedding_cache.json"
+
+OPENAI_EMBEDDING_DEFAULT_MODEL = "text-embedding-3-small"
+OPENAI_EMBEDDINGS_PATH = "/embeddings"
+
+
+class EmbeddingProvider(StrEnum):
+ UNIXCODER = "unixcoder"
+ OPENAI = "openai"
+
+
+class EmbeddingDevice(StrEnum):
+ CUDA = "cuda"
+ MPS = "mps"
+ CPU = "cpu"
+
+
+class VectorStoreBackend(StrEnum):
+ QDRANT = "qdrant"
+ MILVUS = "milvus"
+
+
+# Batches between torch.mps.empty_cache() calls: dropping the Metal
+# allocator cache every batch costs ~21% throughput (M-series UniXcoder
+# run), so release it periodically to bound growth.
+EMBEDDING_MPS_CACHE_DROP_INTERVAL = 64
+
+
+# ModelConfig field names
+FIELD_PROVIDER = "provider"
+FIELD_MODEL_ID = "model_id"
+FIELD_API_KEY = "api_key"
+FIELD_ENDPOINT = "endpoint"
+
+ANTHROPIC_COUNT_TOKENS_URL = "https://api.anthropic.com/v1/messages/count_tokens"
+ANTHROPIC_API_VERSION = "2023-06-01"
+ANTHROPIC_HEADER_API_KEY = "x-api-key"
+ANTHROPIC_HEADER_VERSION = "anthropic-version"
+HEADER_CONTENT_TYPE = "content-type"
+CONTENT_TYPE_JSON = "application/json"
+ANTHROPIC_COUNT_TIMEOUT_S = 10.0
+
+DEFAULT_CONTEXT_WINDOW = 200_000
+MODEL_CONTEXT_WINDOWS: dict[str, int] = {
+ "MiniMax-M3": 1_000_000,
+ "MiniMax-M2.7": 204_800,
+ "claude-opus-4-8": 1_000_000,
+ "claude-opus-4-7": 1_000_000,
+ "claude-opus-4-6": 200_000,
+ "claude-opus-4-5": 200_000,
+ "claude-opus-4-1": 200_000,
+ "claude-opus-4-0": 200_000,
+ "claude-sonnet-4-6": 200_000,
+ "claude-sonnet-4-5": 200_000,
+ "claude-sonnet-4-0": 200_000,
+ "claude-haiku-4-5": 200_000,
+ "claude-haiku-4-0": 200_000,
+}
+
+MODULE_TORCH = "torch"
+MODULE_TRANSFORMERS = "transformers"
+MODULE_QDRANT_CLIENT = "qdrant_client"
+MODULE_PYMILVUS = "pymilvus"
+
+SEMANTIC_DEPENDENCIES = (
+ MODULE_PYMILVUS,
+ MODULE_QDRANT_CLIENT,
+ MODULE_TORCH,
+ MODULE_TRANSFORMERS,
+)
+ML_DEPENDENCIES = (MODULE_TORCH, MODULE_TRANSFORMERS)
+
+
+class UniXcoderMode(StrEnum):
+ ENCODER_ONLY = ""
+ DECODER_ONLY = ""
+ ENCODER_DECODER = ""
+
+
+UNIXCODER_MASK_TOKEN = ""
+UNIXCODER_BUFFER_BIAS = "bias"
+UNIXCODER_MAX_CONTEXT = 1024
diff --git a/codebase_rag/constants/security.py b/codebase_rag/constants/security.py
new file mode 100644
index 000000000..47cca2d53
--- /dev/null
+++ b/codebase_rag/constants/security.py
@@ -0,0 +1,176 @@
+# Dangerous shell command and Cypher query guard tables.
+
+# Cypher response cleaning
+CYPHER_PREFIX = "cypher"
+CYPHER_SEMICOLON = ";"
+CYPHER_BACKTICK = "`"
+CYPHER_MATCH_KEYWORD = "MATCH"
+CYPHER_DANGEROUS_KEYWORDS: frozenset[str] = frozenset(
+ {
+ "DELETE",
+ "DETACH",
+ "DROP",
+ "CREATE INDEX",
+ "CREATE CONSTRAINT",
+ "REMOVE",
+ "SET",
+ "MERGE",
+ "CREATE",
+ "LOAD CSV",
+ "FOREACH",
+ }
+)
+
+CYPHER_ALLOWED_PROCEDURE_PREFIXES: frozenset[str] = frozenset(
+ {
+ "algo.",
+ "betweenness_centrality.",
+ "biconnected_components.",
+ "bridges.",
+ "community_detection.",
+ "cycles.",
+ "degree_centrality.",
+ "graph_analyzer.",
+ "graph_util.",
+ "igraphalg.",
+ "katz_centrality.",
+ "leiden_community_detection.",
+ "neighbors.",
+ "node_similarity.",
+ "nxalg.",
+ "pagerank.",
+ "path.",
+ "schema.",
+ "weakly_connected_components.",
+ "wcc.",
+ }
+)
+
+# Shell command constants
+SHELL_CMD_GREP = "grep"
+SHELL_CMD_GIT = "git"
+SHELL_CMD_RM = "rm"
+SHELL_RM_RF_FLAG = "-rf"
+SHELL_RETURN_CODE_ERROR = -1
+SHELL_PIPE_OPERATORS = ("|", "&&", "||", ";")
+SHELL_SUBSHELL_PATTERNS = ("$(", "`")
+SHELL_REDIRECT_OPERATORS = frozenset({">", ">>", "<", "<<"})
+
+# Dangerous commands, absolutely blocked
+SHELL_DANGEROUS_COMMANDS = frozenset(
+ {
+ "dd",
+ "mkfs",
+ "mkfs.ext4",
+ "mkfs.ext3",
+ "mkfs.xfs",
+ "mkfs.btrfs",
+ "mkfs.vfat",
+ "fdisk",
+ "parted",
+ "shred",
+ "wipefs",
+ "mkswap",
+ "swapon",
+ "swapoff",
+ "mount",
+ "umount",
+ "insmod",
+ "rmmod",
+ "modprobe",
+ "shutdown",
+ "reboot",
+ "halt",
+ "poweroff",
+ "init",
+ "telinit",
+ "systemctl",
+ "service",
+ "chroot",
+ "nohup",
+ "disown",
+ "crontab",
+ "at",
+ "batch",
+ }
+)
+
+# Dangerous rm flags
+SHELL_RM_DANGEROUS_FLAGS = frozenset({"-rf", "-fr"})
+SHELL_RM_FORCE_FLAG = "-f"
+
+# System directories to protect from rm -rf
+SHELL_SYSTEM_DIRECTORIES = frozenset(
+ {
+ "bin",
+ "boot",
+ "dev",
+ "etc",
+ "home",
+ "lib",
+ "lib64",
+ "media",
+ "mnt",
+ "opt",
+ "proc",
+ "root",
+ "run",
+ "sbin",
+ "srv",
+ "sys",
+ "tmp",
+ "usr",
+ "var",
+ }
+)
+
+# Dangerous patterns for full pipeline (cross-segment patterns with pipes/operators)
+SHELL_DANGEROUS_PATTERNS_PIPELINE = (
+ (r"(wget|curl)\s+.*\|\s*(sh|bash|zsh|ksh)", "remote script execution"),
+ (r"(wget|curl)\s+.*>\s*.*\.sh\s*&&", "download and execute script"),
+ (r"base64\s+-d.*\|", "base64 decode pipe execution"),
+)
+
+_SYSTEM_DIRS_PATTERN = "|".join(SHELL_SYSTEM_DIRECTORIES)
+
+# Dangerous patterns for individual segments (per-command patterns)
+SHELL_DANGEROUS_PATTERNS_SEGMENT = (
+ (r"rm\s+.*-[rf]+\s+/($|\s)", "rm with root path"),
+ (rf"rm\s+.*-[rf]+\s+/({_SYSTEM_DIRS_PATTERN})($|/|\s)", "rm with system directory"),
+ (r"rm\s+.*-[rf]+\s+~($|\s)", "rm with home directory"),
+ (r"rm\s+.*-[rf]+\s+\*", "rm with wildcard"),
+ (r"rm\s+.*-[rf]+\s+\.\.", "rm with parent directory"),
+ (r"dd\s+.*of=/dev/", "dd writing to device"),
+ (r">\s*/dev/sd[a-z]", "redirect to disk device"),
+ (r">\s*/dev/nvme", "redirect to nvme device"),
+ (r">\s*/dev/null.*<", "null device manipulation"),
+ (r"chmod\s+.*-R\s+777\s+/", "recursive 777 on root"),
+ (r"chmod\s+.*777\s+/($|\s)", "777 on root"),
+ (r"chown\s+.*-R\s+.*\s+/($|\s)", "recursive chown on root"),
+ (r":\(\)\s*\{.*:\s*\|", "fork bomb pattern"),
+ (r"mv\s+.*\s+/dev/null", "move to /dev/null"),
+ (r"ln\s+-[sf]+\s+/dev/null", "symlink to /dev/null"),
+ (r"cat\s+.*/dev/zero", "cat /dev/zero"),
+ (r"cat\s+.*/dev/random", "cat /dev/random"),
+ (r">\s*/etc/passwd", "overwrite passwd"),
+ (r">\s*/etc/shadow", "overwrite shadow"),
+ (r">\s*/etc/sudoers", "overwrite sudoers"),
+ (r"echo\s+.*>\s*/etc/", "write to /etc"),
+ (
+ r"python.*-c.*(import\s+os|__import__\s*\(\s*['\"]os['\"]\s*\))",
+ "python os import in command",
+ ),
+ (r"perl\s+-e", "perl one-liner"),
+ (r"ruby\s+-e", "ruby one-liner"),
+ (r"nc\s+-[el]", "netcat listener"),
+ (r"ncat\s+-[el]", "ncat listener"),
+ (r"/dev/tcp/", "bash tcp device"),
+ (r"eval\s+", "eval command"),
+ (r"exec\s+[0-9]+<>", "exec file descriptor manipulation"),
+ (r"awk\s+.*system\s*\(", "awk system() call"),
+ (r"awk\s+.*getline\s*[<|]", "awk getline file/pipe execution"),
+ (r"sed\s+.*s(.).*?\1.*?\1[gip]*e[gip]*", "sed execute flag"),
+ (r"xargs\s+.*(rm|chmod|chown|mv|dd|mkfs)", "xargs with destructive command"),
+ (r"xargs\s+-I.*sh", "xargs shell execution"),
+ (r"xargs\s+.*bash", "xargs bash execution"),
+)
diff --git a/codebase_rag/constants/stdlib_types.py b/codebase_rag/constants/stdlib_types.py
new file mode 100644
index 000000000..d8dacd7e6
--- /dev/null
+++ b/codebase_rag/constants/stdlib_types.py
@@ -0,0 +1,397 @@
+# Language stdlib entity tables for external-call classification.
+
+# JavaScript built-in types
+JS_BUILTIN_TYPES: frozenset[str] = frozenset(
+ {
+ "Array",
+ "Object",
+ "String",
+ "Number",
+ "Date",
+ "RegExp",
+ "Function",
+ "Map",
+ "Set",
+ "Promise",
+ "Error",
+ "Boolean",
+ }
+)
+
+# JavaScript built-in function patterns
+# JS/TS runtime global classes usable as `extends` bases with no import.
+# A base matching one that resolves to no first-party class is positively
+# external (builtin.), not an unresolvable guess.
+JS_GLOBAL_CLASS_NAMES: frozenset[str] = frozenset(
+ {
+ "Error",
+ "TypeError",
+ "RangeError",
+ "SyntaxError",
+ "ReferenceError",
+ "EvalError",
+ "URIError",
+ "AggregateError",
+ "Object",
+ "Array",
+ "Function",
+ "Promise",
+ "Map",
+ "Set",
+ "WeakMap",
+ "WeakSet",
+ "Date",
+ "RegExp",
+ "ArrayBuffer",
+ "SharedArrayBuffer",
+ "DataView",
+ "EventTarget",
+ "Event",
+ "HTMLElement",
+ }
+)
+
+JS_BUILTIN_PATTERNS: frozenset[str] = frozenset(
+ {
+ "Object.create",
+ "Object.keys",
+ "Object.values",
+ "Object.entries",
+ "Object.assign",
+ "Object.freeze",
+ "Object.seal",
+ "Object.defineProperty",
+ "Object.getPrototypeOf",
+ "Object.setPrototypeOf",
+ "Array.from",
+ "Array.of",
+ "Array.isArray",
+ "parseInt",
+ "parseFloat",
+ "isNaN",
+ "isFinite",
+ "encodeURIComponent",
+ "decodeURIComponent",
+ "setTimeout",
+ "clearTimeout",
+ "setInterval",
+ "clearInterval",
+ "console.log",
+ "console.error",
+ "console.warn",
+ "console.info",
+ "console.debug",
+ "JSON.parse",
+ "JSON.stringify",
+ "Math.random",
+ "Math.floor",
+ "Math.ceil",
+ "Math.round",
+ "Math.abs",
+ "Math.max",
+ "Math.min",
+ "Date.now",
+ "Date.parse",
+ }
+)
+
+JS_METHOD_BIND = "bind"
+JS_METHOD_CALL = "call"
+JS_METHOD_APPLY = "apply"
+JS_SUFFIX_BIND = ".bind"
+JS_SUFFIX_CALL = ".call"
+JS_SUFFIX_APPLY = ".apply"
+JS_FUNCTION_PROTOTYPE_SUFFIXES: dict[str, str] = {
+ JS_SUFFIX_BIND: JS_METHOD_BIND,
+ JS_SUFFIX_CALL: JS_METHOD_CALL,
+ JS_SUFFIX_APPLY: JS_METHOD_APPLY,
+}
+# `fn.bind(ctx)` / `fn.call(...)` / `fn.apply(...)` all use `fn`; in a value
+# position (`onError: handleError.bind(toast)`) the `.bind` resolves to the
+# Function.prototype builtin, so `fn` must be referenced separately or it
+# reports as dead.
+JS_FUNCTION_PROTOTYPE_METHODS = frozenset(
+ {JS_METHOD_BIND, JS_METHOD_CALL, JS_METHOD_APPLY}
+)
+
+# Lua stdlib module names
+LUA_STDLIB_MODULES = frozenset(
+ {
+ "string",
+ "math",
+ "table",
+ "os",
+ "io",
+ "debug",
+ "package",
+ "coroutine",
+ "utf8",
+ "bit32",
+ }
+)
+
+# C++ stdlib namespace and type inference prefixes
+CPP_STD_NAMESPACE = "std"
+CPP_PREFIX_IS = "is_"
+CPP_PREFIX_HAS = "has_"
+
+# C++ stdlib entity names for heuristic detection
+CPP_STDLIB_ENTITIES = frozenset(
+ {
+ "vector",
+ "string",
+ "map",
+ "set",
+ "list",
+ "deque",
+ "unique_ptr",
+ "shared_ptr",
+ "weak_ptr",
+ "thread",
+ "mutex",
+ "condition_variable",
+ "future",
+ "promise",
+ "sort",
+ "find",
+ "copy",
+ "transform",
+ "accumulate",
+ }
+)
+
+# Java stdlib package prefixes for static stdlib detection
+JAVA_STDLIB_PREFIXES = (
+ "java.",
+ "javax.",
+ "jdk.",
+ "com.sun.",
+ "sun.",
+ "org.w3c.",
+ "org.xml.",
+ "org.ietf.",
+ "org.omg.",
+ "netscape.",
+)
+
+# Java common class names for heuristic detection
+JAVA_STDLIB_CLASSES = frozenset(
+ {
+ "String",
+ "Object",
+ "Integer",
+ "Double",
+ "Boolean",
+ "ArrayList",
+ "HashMap",
+ "HashSet",
+ "LinkedList",
+ "File",
+ "URL",
+ "Pattern",
+ "LocalDateTime",
+ "BigDecimal",
+ }
+)
+
+# Java type inference constants
+JAVA_LANG_PREFIX = "java.lang."
+JAVA_ARRAY_SUFFIX = "[]"
+JAVA_SUFFIX_EXCEPTION = "Exception"
+JAVA_SUFFIX_ERROR = "Error"
+JAVA_SUFFIX_INTERFACE = "Interface"
+JAVA_SUFFIX_BUILDER = "Builder"
+JAVA_PRIMITIVE_TYPES = frozenset(
+ {
+ "int",
+ "long",
+ "double",
+ "float",
+ "boolean",
+ "char",
+ "byte",
+ "short",
+ }
+)
+JAVA_WRAPPER_TYPES = frozenset(
+ {
+ "String",
+ "Object",
+ "Integer",
+ "Long",
+ "Double",
+ "Boolean",
+ }
+)
+
+# java.lang types Java code names WITHOUT an import (the implicit java.lang
+# import). A bare `extends`/`implements` base matching one that resolves to
+# no first-party type is positively external (java.lang.); mirrors the
+# JS global class rule (JS_GLOBAL_CLASS_NAMES -> builtin.).
+JAVA_LANG_CLASS_NAMES = JAVA_WRAPPER_TYPES | frozenset(
+ {
+ "Byte",
+ "Short",
+ "Float",
+ "Character",
+ "Number",
+ "Void",
+ "Enum",
+ "Record",
+ "Thread",
+ "ThreadLocal",
+ "ClassLoader",
+ "SecurityManager",
+ "StringBuilder",
+ "StringBuffer",
+ "Throwable",
+ "Exception",
+ "RuntimeException",
+ "Error",
+ "IllegalArgumentException",
+ "IllegalStateException",
+ "UnsupportedOperationException",
+ "NullPointerException",
+ "IndexOutOfBoundsException",
+ "ArrayIndexOutOfBoundsException",
+ "StringIndexOutOfBoundsException",
+ "ClassCastException",
+ "ArithmeticException",
+ "NumberFormatException",
+ "InterruptedException",
+ "CloneNotSupportedException",
+ "ReflectiveOperationException",
+ "ClassNotFoundException",
+ "NoSuchMethodException",
+ "NoSuchFieldException",
+ "InstantiationException",
+ "IllegalAccessException",
+ "SecurityException",
+ "AssertionError",
+ "StackOverflowError",
+ "OutOfMemoryError",
+ "LinkageError",
+ "NoClassDefFoundError",
+ "Runnable",
+ "Comparable",
+ "Iterable",
+ "Cloneable",
+ "AutoCloseable",
+ "CharSequence",
+ "Appendable",
+ "Readable",
+ }
+)
+
+# C# base class library / framework roots. A qualified name under one of
+# these namespaces (`System.Collections.Generic.List`) is external stdlib,
+# not first-party code, so stdlib extraction folds the trailing PascalCase
+# type into its namespace path.
+CSHARP_STDLIB_PREFIXES = (
+ "System.",
+ "Microsoft.",
+ "Windows.",
+ "Mono.",
+)
+
+# Recognized BCL types. ONLY a name in this set folds into its namespace
+# (`System.Collections.Generic.List` -> `System.Collections.Generic`); every
+# other PascalCase leaf is kept whole as a namespace, because C# namespaces
+# are PascalCase too and a case heuristic would misfold them
+# (`Microsoft.Extensions.Logging`).
+CSHARP_STDLIB_CLASSES = frozenset(
+ {
+ # System primitives / core types
+ "Object",
+ "String",
+ "Int32",
+ "Int64",
+ "Boolean",
+ "Double",
+ "Decimal",
+ "Single",
+ "Byte",
+ "Char",
+ "Guid",
+ "DateTime",
+ "DateTimeOffset",
+ "TimeSpan",
+ "Uri",
+ "Exception",
+ "Nullable",
+ "Type",
+ "Action",
+ "Func",
+ "Console",
+ # System.Threading.Tasks
+ "Task",
+ "ValueTask",
+ "CancellationToken",
+ # System.Collections.Generic
+ "List",
+ "Dictionary",
+ "HashSet",
+ "Queue",
+ "Stack",
+ "SortedList",
+ "SortedDictionary",
+ "LinkedList",
+ "IEnumerable",
+ "ICollection",
+ "IList",
+ "IDictionary",
+ "IReadOnlyList",
+ "IReadOnlyDictionary",
+ "KeyValuePair",
+ # System.Linq
+ "Enumerable",
+ "IQueryable",
+ # System interfaces
+ "IDisposable",
+ "IAsyncDisposable",
+ "IComparable",
+ "IEquatable",
+ # Other ubiquitous BCL types (curated common set; a complete list
+ # is unbounded, so the tail stays as full type paths rather than risk
+ # a case heuristic that would misfold PascalCase namespaces).
+ "Math",
+ "MathF",
+ "Random",
+ "Convert",
+ "Environment",
+ "Array",
+ "Span",
+ "Memory",
+ "Tuple",
+ "Lazy",
+ "GC",
+ "StringBuilder",
+ "StringComparer",
+ "Regex",
+ "Match",
+ "Encoding",
+ "File",
+ "Directory",
+ "Path",
+ "Stream",
+ "MemoryStream",
+ "FileStream",
+ "StreamReader",
+ "StreamWriter",
+ "TextReader",
+ "TextWriter",
+ "HttpClient",
+ "HttpResponseMessage",
+ "HttpRequestMessage",
+ "JsonSerializer",
+ "Thread",
+ "Mutex",
+ "SemaphoreSlim",
+ "Stopwatch",
+ "Timer",
+ "CultureInfo",
+ "IServiceProvider",
+ "IServiceCollection",
+ "ILogger",
+ }
+)
diff --git a/codebase_rag/constants/structural.py b/codebase_rag/constants/structural.py
new file mode 100644
index 000000000..2fb5baf1d
--- /dev/null
+++ b/codebase_rag/constants/structural.py
@@ -0,0 +1,69 @@
+# Constants for the ast-grep structural search/replace toolkit (#415).
+from __future__ import annotations
+
+import re
+
+from .languages import SupportedLanguage
+
+# Optional dependency module name for the dependency guard.
+MODULE_AST_GREP = "ast_grep_py"
+
+# cgr language -> ast-grep language id. scala and dart are intentionally
+# absent: ast-grep ships no grammar for them, so files in those languages
+# are skipped by the tools rather than crashing the Rust binding.
+AST_GREP_LANGUAGES: dict[SupportedLanguage, str] = {
+ SupportedLanguage.PYTHON: "python",
+ SupportedLanguage.JS: "javascript",
+ SupportedLanguage.TS: "typescript",
+ SupportedLanguage.TSX: "tsx",
+ SupportedLanguage.RUST: "rust",
+ SupportedLanguage.GO: "go",
+ SupportedLanguage.JAVA: "java",
+ SupportedLanguage.C: "c",
+ SupportedLanguage.CPP: "cpp",
+ SupportedLanguage.PHP: "php",
+ SupportedLanguage.LUA: "lua",
+ SupportedLanguage.CSHARP: "csharp",
+}
+
+# Metavariable tokens in a rewrite template. ast-grep's node.replace() does
+# NOT interpolate metavars, so the service does it: $$$NAME (multi),
+# $NAME (single). One pattern matched left-to-right in a single pass, so
+# text inserted for one metavar is never re-scanned for another.
+AST_GREP_METAVAR_RE = re.compile(r"\$\$\$([A-Z_][A-Z0-9_]*)|\$([A-Z_][A-Z0-9_]*)")
+
+# Cap on matches from a single structural search, so an over-broad pattern
+# cannot flood the agent context. Truncation is reported, not silent.
+AST_GREP_MAX_RESULTS = 200
+
+# Result dict keys (StructuralSearchMatch / StructuralReplaceChange).
+STRUCT_KEY_FILE = "file"
+STRUCT_KEY_LINE = "line"
+STRUCT_KEY_COLUMN = "column"
+STRUCT_KEY_END_LINE = "end_line"
+STRUCT_KEY_END_COLUMN = "end_column"
+STRUCT_KEY_TEXT = "text"
+STRUCT_KEY_MATCHES = "matches"
+STRUCT_KEY_DIFF = "diff"
+STRUCT_KEY_APPLIED = "applied"
+
+# User-facing messages.
+AST_GREP_NOT_AVAILABLE = (
+ "Structural search/replace is not available. Install with: uv sync --extra ast-grep"
+)
+AST_GREP_NO_MATCHES = "No structural matches for pattern: {pattern}"
+AST_GREP_UNKNOWN_LANGUAGE = (
+ "Unknown or unsupported language '{language}'. Supported: {supported}"
+)
+AST_GREP_INVALID_PATTERN = "Invalid ast-grep pattern '{pattern}': {error}"
+AST_GREP_TRUNCATED = (
+ "Result cap of {limit} reached; narrow the pattern or raise the limit for more."
+)
+AST_GREP_DRY_RUN_HEADER = "Dry run: {count} file(s) would change. No files written."
+AST_GREP_APPLIED_HEADER = "Applied: rewrote {count} file(s)."
+
+# Approval-prompt display for a structural rewrite.
+AST_GREP_APPROVAL_HEADER = "Structural replace"
+AST_GREP_APPROVAL_PATTERN = "pattern: {pattern}"
+AST_GREP_APPROVAL_REWRITE = "rewrite: {rewrite}"
+AST_GREP_APPROVAL_DRY_RUN = "dry_run: {dry_run}"
diff --git a/codebase_rag/cypher_queries.py b/codebase_rag/cypher_queries.py
index 8d70bae4e..c59d626bd 100644
--- a/codebase_rag/cypher_queries.py
+++ b/codebase_rag/cypher_queries.py
@@ -1,8 +1,31 @@
-from .constants import CYPHER_DEFAULT_LIMIT
+from .constants import CYPHER_DEFAULT_LIMIT, NodeLabel, RelationshipType
CYPHER_DELETE_ALL = "MATCH (n) DETACH DELETE n;"
-CYPHER_LIST_PROJECTS = "MATCH (p:Project) RETURN p.name AS name ORDER BY p.name"
+# Graph structural integrity audit (issue #646). A zero-degree Project is a
+# valid empty-repo graph, so the orphan scan exempts it.
+CYPHER_AUDIT_ORPHANS = (
+ "MATCH (n) WHERE NOT (n)--() AND NOT n:Project "
+ "RETURN labels(n)[0] AS label, count(n) AS orphans"
+)
+CYPHER_AUDIT_LABELS = "MATCH (n) UNWIND labels(n) AS label RETURN DISTINCT label"
+CYPHER_AUDIT_REL_TRIPLES = (
+ "MATCH (a)-[r]->(b) "
+ "RETURN DISTINCT labels(a)[0] AS src, type(r) AS rel, labels(b)[0] AS dst"
+)
+CYPHER_AUDIT_LABEL_PROPS = (
+ "MATCH (n) UNWIND labels(n) AS label UNWIND keys(n) AS key "
+ "RETURN DISTINCT label AS label, key AS key"
+)
+CYPHER_AUDIT_MISSING_REQUIRED = (
+ "MATCH (n:{label}) WHERE {conditions} RETURN count(n) AS missing"
+)
+CYPHER_AUDIT_IS_NULL = "n.{prop} IS NULL"
+CYPHER_AUDIT_OR = " OR "
+
+CYPHER_LIST_PROJECTS = (
+ "MATCH (p:Project) RETURN p.name AS name, p.root_path AS root_path ORDER BY p.name"
+)
CYPHER_DELETE_PROJECT = """
MATCH (p:Project {name: $project_name})
@@ -11,6 +34,45 @@
DETACH DELETE p, container, defined
"""
+CYPHER_SHOW_CONSTRAINTS = "SHOW CONSTRAINT INFO;"
+
+# Damage detectors for the issue #897 migration. Sharing always leaves a
+# single-hop signature: the topmost merged node has containment parents in
+# two projects (Project roots are never merged, so the parents are distinct
+# nodes). Keyless rows match the second purge's predicate directly.
+CYPHER_ANY_SHARED_STRUCTURE = (
+ "MATCH (parent)-[:CONTAINS_FOLDER|CONTAINS_FILE]->(n) "
+ "WHERE (n:Folder OR n:File) "
+ "WITH n, count(parent) AS parents "
+ "WHERE parents > 1 "
+ "RETURN 1 AS damaged LIMIT 1"
+)
+
+CYPHER_ANY_KEYLESS_STRUCTURE = (
+ "MATCH (n) WHERE (n:Folder OR n:File) AND n.absolute_path IS NULL "
+ "RETURN 1 AS damaged LIMIT 1"
+)
+
+# The superseded relative-path key merged same-layout projects onto shared
+# Folder/File nodes (issue #897). A merged node cannot be split, so anything
+# the containment walk reaches from more than one Project is purged; the
+# next re-index rebuilds it with per-project identity.
+CYPHER_PURGE_CROSS_PROJECT_STRUCTURE = (
+ "MATCH (p:Project)"
+ "-[:CONTAINS_PACKAGE|CONTAINS_FOLDER|CONTAINS_FILE|CONTAINS_MODULE*]->(n) "
+ "WHERE (n:Folder OR n:File) "
+ "WITH n, count(DISTINCT p) AS owners "
+ "WHERE owners > 1 "
+ "DETACH DELETE n RETURN count(n) AS purged"
+)
+
+# Rows written before absolute_path existed can never match the current
+# delete queries; they are unmanageable and must go with the migration.
+CYPHER_PURGE_KEYLESS_STRUCTURE = (
+ "MATCH (n) WHERE (n:Folder OR n:File) AND n.absolute_path IS NULL "
+ "DETACH DELETE n RETURN count(n) AS purged"
+)
+
CYPHER_EXAMPLE_DECORATED_FUNCTIONS = f"""MATCH (n:Function|Method)
WHERE ANY(d IN n.decorators WHERE toLower(d) IN ['flow', 'task'])
RETURN n.name AS name, n.qualified_name AS qualified_name, labels(n) AS type
@@ -51,9 +113,30 @@
CYPHER_EXAMPLE_LIMIT_ONE = """MATCH (f:File) RETURN f.path as path, f.name as name, labels(f) as type LIMIT 1"""
+CYPHER_EXAMPLE_PROJECT_SCOPED = f"""MATCH (c:Class)
+WHERE c.qualified_name STARTS WITH 'myproject.'
+RETURN c.name AS name, c.qualified_name AS qualified_name, labels(c) AS type
+LIMIT {CYPHER_DEFAULT_LIMIT}"""
+
CYPHER_EXAMPLE_CLASS_METHODS = f"""MATCH (c:Class)-[:DEFINES_METHOD]->(m:Method)
-WHERE c.qualified_name ENDS WITH '.UserService'
-RETURN m.name AS name, m.qualified_name AS qualified_name, labels(m) AS type
+WHERE c.name = 'UserService'
+RETURN c.name AS className, m.name AS methodName, m.qualified_name AS qualified_name, labels(m) AS type
+LIMIT {CYPHER_DEFAULT_LIMIT}"""
+
+# ast-grep findings (issue #413): Pattern/CodeSmell/SecurityIssue nodes hang
+# off a Module via IMPLEMENTS_PATTERN/HAS_SMELL/HAS_VULNERABILITY. The finding
+# node's name is the rule id; start_line locates the site.
+CYPHER_EXAMPLE_FIND_PATTERN = f"""MATCH (m:Module)-[:IMPLEMENTS_PATTERN]->(p:Pattern)
+WHERE p.name = 'singleton'
+RETURN m.path AS path, p.name AS pattern, p.start_line AS line, p.message AS message
+LIMIT {CYPHER_DEFAULT_LIMIT}"""
+
+CYPHER_EXAMPLE_SECURITY_ISSUES = f"""MATCH (m:Module)-[:HAS_VULNERABILITY]->(s:SecurityIssue)
+RETURN m.path AS path, s.name AS rule, s.start_line AS line, s.message AS message
+LIMIT {CYPHER_DEFAULT_LIMIT}"""
+
+CYPHER_EXAMPLE_CODE_SMELLS = f"""MATCH (m:Module)-[:HAS_SMELL]->(c:CodeSmell)
+RETURN m.path AS path, c.name AS smell, c.start_line AS line, c.message AS message
LIMIT {CYPHER_DEFAULT_LIMIT}"""
CYPHER_EXPORT_NODES = """
@@ -73,17 +156,73 @@
MATCH (m:Module)-[:DEFINES]->(n)
WHERE id(n) = $node_id
RETURN n.qualified_name AS qualified_name, n.start_line AS start_line,
- n.end_line AS end_line, m.path AS path
+ n.end_line AS end_line, m.path AS path, n.absolute_path AS absolute_path
"""
CYPHER_FIND_BY_QUALIFIED_NAME = """
MATCH (n) WHERE n.qualified_name = $qn
OPTIONAL MATCH (m:Module)-[*]-(n)
-RETURN n.name AS name, n.start_line AS start, n.end_line AS end, m.path AS path, n.docstring AS docstring
+RETURN n.name AS name, n.start_line AS start, n.end_line AS end, m.path AS path,
+ n.absolute_path AS absolute_path, n.docstring AS docstring
LIMIT 1
"""
+CYPHER_STATS_NODE_COUNTS = """
+MATCH (n)
+RETURN labels(n) AS labels, count(*) AS count
+ORDER BY count DESC
+"""
+
+CYPHER_STATS_RELATIONSHIP_COUNTS = """
+MATCH ()-[r]->()
+RETURN type(r) AS type, count(*) AS count
+ORDER BY count DESC
+"""
+
+
+# Dead-code fetch queries. Reachability itself runs client-side in
+# codebase_rag/dead_code.py: the previous single-query formulation expanded
+# *BFS from every root, which is O(roots x graph) and hit memgraph's 600s
+# query timeout on big projects (django: 31k roots, 101k CALLS edges). These
+# two linear scans fetch the project's nodes and edges instead; the target
+# of a relationship is deliberately unfiltered so INHERITS to an external
+# base (typing.Protocol) and OVERRIDES of external methods stay visible.
+_DEAD_CODE_NODE_LABELS = "|".join(
+ (
+ NodeLabel.FUNCTION.value,
+ NodeLabel.METHOD.value,
+ NodeLabel.CLASS.value,
+ NodeLabel.MODULE.value,
+ )
+)
+_DEAD_CODE_REL_TYPES = "|".join(
+ (
+ RelationshipType.CALLS.value,
+ RelationshipType.REFERENCES.value,
+ RelationshipType.INSTANTIATES.value,
+ RelationshipType.INHERITS.value,
+ RelationshipType.DEFINES.value,
+ RelationshipType.DEFINES_METHOD.value,
+ RelationshipType.OVERRIDES.value,
+ )
+)
+
+CYPHER_DEAD_CODE_NODES = f"""MATCH (n:{_DEAD_CODE_NODE_LABELS})
+WHERE n.qualified_name STARTS WITH $project_prefix
+RETURN labels(n)[0] AS label, n.qualified_name AS qualified_name,
+ n.name AS name, n.path AS path,
+ n.start_line AS start_line, n.end_line AS end_line,
+ n.decorators AS decorators, n.is_exported AS is_exported,
+ n.overrides_external AS overrides_external"""
+
+CYPHER_DEAD_CODE_RELS = f"""MATCH (a:{_DEAD_CODE_NODE_LABELS})-[r:{_DEAD_CODE_REL_TYPES}]->(b)
+WHERE a.qualified_name STARTS WITH $project_prefix
+RETURN labels(a)[0] AS from_label, a.qualified_name AS from_qn,
+ type(r) AS rel_type, labels(b)[0] AS to_label,
+ b.qualified_name AS to_qn"""
+
+
def wrap_with_unwind(query: str) -> str:
return f"UNWIND $batch AS row\n{query}"
@@ -103,6 +242,10 @@ def build_constraint_query(label: str, prop: str) -> str:
return f"CREATE CONSTRAINT ON (n:{label}) ASSERT n.{prop} IS UNIQUE;"
+def build_drop_constraint_query(label: str, prop: str) -> str:
+ return f"DROP CONSTRAINT ON (n:{label}) ASSERT n.{prop} IS UNIQUE;"
+
+
def build_index_query(label: str, prop: str) -> str:
return f"CREATE INDEX ON :{label}({prop});"
@@ -118,11 +261,40 @@ def build_merge_relationship_query(
to_label: str,
to_key: str,
has_props: bool = False,
+ merge_key_props: tuple[str, ...] = (),
+) -> str:
+ # merge_key_props: properties that distinguish parallel edges between the
+ # same node pair (e.g. FLOWS_TO's `via`). Including them in the MERGE
+ # pattern keeps each variant as its own edge instead of collapsing them
+ # into one (issue #722). Every row in the batch must carry these keys.
+ key_map = ""
+ if merge_key_props:
+ key_map = " {" + ", ".join(f"{p}: row.props.{p}" for p in merge_key_props) + "}"
+ query = (
+ f"MATCH (a:{from_label} {{{from_key}: row.from_val}}), "
+ f"(b:{to_label} {{{to_key}: row.to_val}})\n"
+ f"MERGE (a)-[r:{rel_type}{key_map}]->(b)\n"
+ )
+ query += CYPHER_SET_PROPS_RETURN_COUNT if has_props else CYPHER_RETURN_COUNT
+ return query
+
+
+def build_create_node_query(label: str, id_key: str) -> str:
+ return f"CREATE (n:{label} {{{id_key}: row.id}})\nSET n += row.props"
+
+
+def build_create_relationship_query(
+ from_label: str,
+ from_key: str,
+ rel_type: str,
+ to_label: str,
+ to_key: str,
+ has_props: bool = False,
) -> str:
query = (
f"MATCH (a:{from_label} {{{from_key}: row.from_val}}), "
f"(b:{to_label} {{{to_key}: row.to_val}})\n"
- f"MERGE (a)-[r:{rel_type}]->(b)\n"
+ f"CREATE (a)-[r:{rel_type}]->(b)\n"
)
query += CYPHER_SET_PROPS_RETURN_COUNT if has_props else CYPHER_RETURN_COUNT
return query
diff --git a/codebase_rag/dead_code.py b/codebase_rag/dead_code.py
new file mode 100644
index 000000000..37b4cd796
--- /dev/null
+++ b/codebase_rag/dead_code.py
@@ -0,0 +1,472 @@
+# Dead-code reachability engine. Roots (entry points, framework hooks,
+# module-load callees, test code) expand over CALLS/REFERENCES edges;
+# whatever is never reached is reported. Reachability runs client-side in
+# Python: the per-root *BFS Cypher formulation is O(roots x graph) and hit
+# memgraph's 600s timeout on big projects (django: 31k roots, 101k CALLS
+# edges), whereas a multi-source walk over the fetched edges is linear and
+# finishes in milliseconds.
+from collections import defaultdict
+from fnmatch import fnmatch
+
+from . import constants as cs
+from . import cypher_queries as cq
+from .types_defs import (
+ DeadCodeConfig,
+ GraphQueryClient,
+ PropertyDict,
+ PropertyValue,
+ ResultRow,
+ ResultValue,
+)
+
+_MODULE = cs.NodeLabel.MODULE.value
+_FUNCTION = cs.NodeLabel.FUNCTION.value
+_METHOD = cs.NodeLabel.METHOD.value
+_CLASS = cs.NodeLabel.CLASS.value
+_CALLS = cs.RelationshipType.CALLS.value
+_REFERENCES = cs.RelationshipType.REFERENCES.value
+_INSTANTIATES = cs.RelationshipType.INSTANTIATES.value
+_INHERITS = cs.RelationshipType.INHERITS.value
+_DEFINES = cs.RelationshipType.DEFINES.value
+_DEFINES_METHOD = cs.RelationshipType.DEFINES_METHOD.value
+_OVERRIDES = cs.RelationshipType.OVERRIDES.value
+_NodeId = tuple[str, PropertyValue]
+_RelTuple = tuple[str, PropertyValue, str, str, PropertyValue]
+
+
+def default_dead_code_config(
+ include_tests: bool,
+ include_classes: bool,
+ exclude_patterns: tuple[str, ...] = (),
+) -> DeadCodeConfig:
+ return DeadCodeConfig(
+ include_tests=include_tests,
+ include_classes=include_classes,
+ root_decorators=frozenset(d.lower() for d in cs.DEFAULT_ROOT_DECORATORS),
+ entry_points=(),
+ test_patterns=tuple(cs.TEST_PATH_PATTERNS),
+ exclude_patterns=exclude_patterns,
+ )
+
+
+def _norm_decorator(decorator: str) -> str:
+ # Drop '@' and any surrounding attribute brackets, take the text before
+ # '(', then the last dotted segment, lowercased -> `@app.route(...)` and a
+ # C# `[Route("x")]` both become `route`. Bracket-stripping keeps the
+ # normalization robust to whatever a highlight query captures.
+ cleaned = decorator.replace(cs.DECORATOR_AT, "").strip("[] ")
+ head = cleaned.split(cs.CHAR_PAREN_OPEN)[0]
+ return head.split(cs.SEPARATOR_DOT)[-1].strip("[]").lower()
+
+
+def _is_dunder(name: str) -> bool:
+ # A __dunder__ method is invoked by the Python runtime (async with,
+ # iteration, operators), never by an explicit call the graph can see, so it
+ # is a reachability root, not dead code.
+ return (
+ len(name) > len(cs.PY_NAME_DUNDER) * 2
+ and name.startswith(cs.PY_NAME_DUNDER)
+ and name.endswith(cs.PY_NAME_DUNDER)
+ )
+
+
+def _is_rust_runtime_root(name: str, is_method: bool, path: str) -> bool:
+ # A Rust `.rs` symbol the language/runtime invokes with no call site: `fn
+ # main()` (entry) or a trait-impl method (Display::fmt, Iterator::next).
+ # Name-scoped like Python dunders; trait methods must be methods.
+ if not path.endswith(cs.EXT_RS):
+ return False
+ # `main` is only the entry point as a receiverless `fn main()`; a method
+ # named main is not, so gate it to non-methods. Trait methods are the reverse.
+ if name in cs.RUST_ROOT_FUNCTION_NAMES:
+ return not is_method
+ return is_method and name in cs.RUST_TRAIT_METHOD_NAMES
+
+
+def _is_c_cpp_entry_root(
+ name: str, is_method: bool, path: str, qn: str, project_prefix: str
+) -> bool:
+ # A C/C++ program entry (`main`, Windows' `wWinMain`/`WinMain`/`wmain`, a
+ # DLL's `DllMain`) is invoked by the OS runtime, never by a call the graph
+ # sees, so it roots its whole call tree (an unrooted wWinMain reported all
+ # 34 windows/runner symbols of a Flutter desktop shim dead). Only a free
+ # function at FILE scope in a translation-unit source counts: a method, a
+ # namespace-scoped `main`, or a header-defined `WinMain` is ordinary code
+ # the OS cannot invoke. (Linkage is not captured in the graph, so a
+ # file-scope `static DllMain` in a source file still roots.)
+ if is_method or name not in cs.C_CPP_ENTRY_FUNCTION_NAMES:
+ return False
+ if not path.endswith(cs.C_CPP_SOURCE_EXTENSIONS):
+ return False
+ # File scope, exactly: a file-scope definition's qn is the project prefix
+ # plus the path's dotted form (extension dropped) plus the name
+ # (`proj.runner.main.wWinMain` for runner/main.cpp). A namespace inserts
+ # its own segment, even one named like the file stem (`namespace main`
+ # in main.cpp), so nothing short of the exact qn earns the root.
+ dotted_module = path.rsplit(cs.SEPARATOR_DOT, 1)[0].replace(
+ cs.SEPARATOR_SLASH, cs.SEPARATOR_DOT
+ )
+ return qn == f"{project_prefix}{dotted_module}{cs.SEPARATOR_DOT}{name}"
+
+
+def _is_cpp_operator_root(name: str, path: str) -> bool:
+ # A C++ operator overload / user-defined literal (`operator==`, `operator[]`,
+ # `operator""_json`) is invoked by operator/literal SYNTAX, not a named call
+ # the graph sees, so it is a reachability root (like Python dunders / Rust
+ # trait methods). `operator` heads every such definition (member or free),
+ # so the name prefix on a C++ file identifies them.
+ return name.startswith(cs.CPP_OPERATOR_PREFIX) and path.endswith(cs.CPP_EXTENSIONS)
+
+
+def _is_java_serialization_root(name: str, is_method: bool, path: str) -> bool:
+ # A Java serialization hook (`readObject`/`writeObject`/`writeReplace`/
+ # `readResolve`/`readObjectNoData`) is invoked reflectively by the java.io
+ # runtime, never by a named call the graph sees, so it is a reachability root
+ # (like Python dunders / Rust trait methods). Gated to methods on a .java
+ # file; `name` is the bare method name (signature stripped by caller).
+ return (
+ is_method
+ and path.endswith(cs.EXT_JAVA)
+ and name in cs.JAVA_SERIALIZATION_METHOD_NAMES
+ )
+
+
+def _is_csharp_attribute_root(props: PropertyDict, path: str) -> bool:
+ # A C# method carrying a framework/runtime attribute ([Fact], [HttpGet],
+ # [OnDeserialized]) is invoked reflectively, never by a call the graph sees,
+ # so it is a reachability root. Gated to .cs; the decorator set matches via
+ # the normalized (lowercased, arg-stripped) form.
+ return path.endswith(cs.EXT_CS) and _has_root_decorator(
+ props, cs.CSHARP_ROOT_ATTRIBUTES
+ )
+
+
+def _is_csharp_dispose_root(name: str, is_method: bool, path: str) -> bool:
+ # `Dispose`/`DisposeAsync` are invoked by a `using` block's teardown, not
+ # a named call; a reachability root on a .cs method (like the Java hooks).
+ return (
+ is_method
+ and path.endswith(cs.EXT_CS)
+ and name in cs.CSHARP_DISPOSE_METHOD_NAMES
+ )
+
+
+def _is_csharp_operator_or_finalizer_root(name: str, path: str) -> bool:
+ # An operator overload is invoked by operator SYNTAX (`a + b`) and a
+ # finalizer (`~Foo`) by the GC, never a named call the graph sees, so both
+ # are reachability roots on a .cs file (cf. the C++ operator root). The
+ # synthesized leaf carries the `operator_`/`~` prefix.
+ return path.endswith(cs.EXT_CS) and (
+ name.startswith(cs.TS_CSHARP_OPERATOR_NAME_PREFIX)
+ or name.startswith(cs.TS_CSHARP_DESTRUCTOR_NAME_PREFIX)
+ )
+
+
+def _matches_test_path(path: str, patterns: tuple[str, ...]) -> bool:
+ # Match test-path patterns against a leading-slash-normalized path so a dir
+ # pattern like `/tests/` also matches a ROOT `tests/` dir (Rust integration
+ # tests, a top-level tests/ folder), not just a nested `src/tests/`. The
+ # leading slash keeps `contests/` from matching `/tests/`.
+ normalized = (
+ path if path.startswith(cs.SEPARATOR_SLASH) else cs.SEPARATOR_SLASH + path
+ )
+ return any(pattern in normalized for pattern in patterns)
+
+
+def _has_root_decorator(props: PropertyDict, root_decorators: frozenset[str]) -> bool:
+ decorators = props.get(cs.KEY_DECORATORS)
+ if not isinstance(decorators, list):
+ return False
+ return any(_norm_decorator(str(d)) in root_decorators for d in decorators)
+
+
+def _walk(
+ frontier: set[str],
+ adjacency: dict[str, set[str]],
+ live: set[str],
+ added: set[str] | None = None,
+) -> None:
+ stack = list(frontier)
+ while stack:
+ current = stack.pop()
+ for nxt in adjacency.get(current, ()):
+ if nxt not in live:
+ live.add(nxt)
+ if added is not None:
+ added.add(nxt)
+ stack.append(nxt)
+
+
+def dead_code_from_graph(
+ nodes: dict[_NodeId, PropertyDict],
+ rels: list[_RelTuple],
+ project_prefix: str,
+ config: DeadCodeConfig,
+) -> set[str]:
+ labels = {_FUNCTION, _METHOD}
+ traversal = {_CALLS, _REFERENCES}
+ module_rels = {_CALLS, _REFERENCES}
+ if config.include_classes:
+ labels.add(_CLASS)
+ traversal |= {_INSTANTIATES, _INHERITS}
+ module_rels.add(_INSTANTIATES)
+
+ candidates: set[str] = set()
+ props_by_qn: dict[str, PropertyDict] = {}
+ method_qns: set[str] = set()
+ module_path: dict[str, str] = {}
+ for (label, uid), props in nodes.items():
+ if label == _MODULE:
+ module_path[str(uid)] = str(props.get(cs.KEY_PATH, ""))
+ elif label in labels and str(uid).startswith(project_prefix):
+ # With tests excluded, a test-file symbol's only callers are
+ # excluded as roots, so reporting it is noise (test helpers and
+ # mocks are infrastructure, not dead production code).
+ if not config.include_tests and _matches_test_path(
+ str(props.get(cs.KEY_PATH) or ""), config.test_patterns
+ ):
+ continue
+ candidates.add(str(uid))
+ props_by_qn[str(uid)] = props
+ if label == _METHOD:
+ method_qns.add(str(uid))
+
+ roots: set[str] = set()
+ # A method of a typing.Protocol subclass is an interface stub whose callers
+ # resolve to the implementations; DEFINES edges from functions/methods feed
+ # the live-owner registration round below.
+ defines_pairs: list[tuple[str, str]] = []
+ protocol_classes: set[str] = set()
+ class_methods: list[tuple[str, str]] = []
+ nested_class_pairs: list[tuple[str, str]] = []
+ for from_label, from_val, rel_type, to_label, to_val in rels:
+ if rel_type == _DEFINES and from_label in (_FUNCTION, _METHOD):
+ defines_pairs.append((str(from_val), str(to_val)))
+ if to_label == _CLASS:
+ nested_class_pairs.append((str(from_val), str(to_val)))
+ elif rel_type == _INHERITS and str(to_val) in cs.PROTOCOL_BASE_QNS:
+ protocol_classes.add(str(from_val))
+ elif rel_type == _DEFINES_METHOD:
+ class_methods.append((str(from_val), str(to_val)))
+ if from_label != _MODULE or rel_type not in module_rels:
+ continue
+ target_qn = str(to_val)
+ if target_qn not in candidates:
+ continue
+ path = module_path.get(str(from_val), "")
+ is_test = _matches_test_path(path, config.test_patterns)
+ if config.include_tests or not is_test:
+ roots.add(target_qn)
+ protocol_stubs = {m for c, m in class_methods if c in protocol_classes}
+
+ for qn in candidates:
+ if qn in roots:
+ continue
+ props = props_by_qn[qn]
+ # The duplicate-qn marker (`init@51`, a SECOND Go init() in one file)
+ # is a registration artifact, never part of the written name; strip it
+ # so every name-scoped root rule sees the real leaf (kubernetes
+ # pkg.apis.abac register.init@51 reported dead).
+ leaf = qn.rsplit(cs.SEPARATOR_DOT, 1)[-1].split(cs.DUP_QN_MARKER, 1)[0]
+ path = str(props.get(cs.KEY_PATH, ""))
+ if _has_root_decorator(props, config.root_decorators):
+ roots.add(qn)
+ elif props.get(cs.KEY_IS_EXPORTED) is True:
+ roots.add(qn)
+ # A method overriding an EXTERNAL stdlib base's method (click's
+ # textwrap.TextWrapper subclass) is invoked by the base's machinery,
+ # never by a first-party call, so it is a root.
+ elif props.get(cs.KEY_OVERRIDES_EXTERNAL) is True:
+ roots.add(qn)
+ elif qn in protocol_stubs:
+ roots.add(qn)
+ elif qn in method_qns and _is_dunder(leaf) and path.endswith(cs.EXT_PY):
+ roots.add(qn)
+ # Python Enum protocol hooks (_generate_next_value_, _missing_) are
+ # invoked by the enum machinery by NAME, like dunders: roots, not
+ # dead code (django's TextChoices._generate_next_value_).
+ elif (
+ qn in method_qns
+ and leaf in cs.PY_ENUM_HOOK_METHOD_NAMES
+ and path.endswith(cs.EXT_PY)
+ ):
+ roots.add(qn)
+ elif (
+ qn not in method_qns
+ and leaf in cs.GO_ROOT_FUNCTION_NAMES
+ and path.endswith(cs.EXT_GO)
+ ):
+ roots.add(qn)
+ elif _is_rust_runtime_root(leaf, qn in method_qns, path):
+ roots.add(qn)
+ elif _is_cpp_operator_root(leaf, path) or _is_c_cpp_entry_root(
+ leaf, qn in method_qns, path, qn, project_prefix
+ ):
+ roots.add(qn)
+ elif _is_java_serialization_root(
+ leaf.split(cs.CHAR_PAREN_OPEN, 1)[0], qn in method_qns, path
+ ):
+ roots.add(qn)
+ elif _is_csharp_attribute_root(props, path):
+ roots.add(qn)
+ elif _is_csharp_dispose_root(
+ leaf.split(cs.CHAR_PAREN_OPEN, 1)[0], qn in method_qns, path
+ ):
+ roots.add(qn)
+ elif _is_csharp_operator_or_finalizer_root(leaf, path):
+ roots.add(qn)
+ elif any(qn.endswith(entry) for entry in config.entry_points):
+ roots.add(qn)
+ elif config.include_tests and _matches_test_path(path, config.test_patterns):
+ roots.add(qn)
+
+ adjacency: dict[str, set[str]] = defaultdict(set)
+ # OVERRIDES is recorded overrider -> overridden; keep the REVERSE mapping
+ # (overridden -> overriders) to expand virtual-dispatch targets below.
+ override_rev: dict[str, set[str]] = defaultdict(set)
+ for from_label, from_val, rel_type, _to_label, to_val in rels:
+ if rel_type in traversal:
+ adjacency[str(from_val)].add(str(to_val))
+ elif rel_type == _OVERRIDES:
+ override_rev[str(to_val)].add(str(from_val))
+
+ live = set(roots)
+ _walk(roots, adjacency, live)
+
+ # Second expansion: a decorated function DEFINED by a LIVE owner is
+ # framework-registered when the owner runs, so it and its callees are
+ # live; the closure of a DEAD owner never registers and stays in the
+ # reported cluster. ponytail: one round, so a registration chain nested
+ # two closures deep is missed; iterate to fixed point if real code ever
+ # registers closures from inside registered closures.
+ closure_roots = {
+ c
+ for o, c in defines_pairs
+ if o in live
+ and c not in live
+ and c in props_by_qn
+ and props_by_qn[c].get(cs.KEY_DECORATORS)
+ }
+ live |= closure_roots
+ _walk(closure_roots, adjacency, live)
+
+ # Factory-class and override expansions, iterated together to a fixed
+ # point because they feed each other (a factory revived only via an
+ # override's callee still needs its class rooted, and vice versa).
+ #
+ # Factory-class rule: a class defined inside a LIVE function
+ # (django's create_reverse_many_to_one_manager) escapes via its return
+ # value or arguments, so no call edge lands on its methods. Treat them as
+ # dispatch surface and revive their callee closure; a DEAD factory's
+ # class stays dead.
+ #
+ # Override rule: a call to a base or interface method dispatches at
+ # runtime to any override, so every transitive override of a LIVE method
+ # is a reachable dispatch target, as is its callee closure. `override_rev`
+ # walks all multi-level overriders (Base<-Sub<-SubSub); an override of a
+ # DEAD base stays dead.
+ #
+ # Each round scans only nodes revived since the last (the pair maps and
+ # override_rev are static, so a rescanned node yields nothing new),
+ # keeping the loop O(live) total; a round that adds nothing ends it.
+ classes_by_owner: dict[str, set[str]] = defaultdict(set)
+ for owner, cls in nested_class_pairs:
+ classes_by_owner[owner].add(cls)
+ methods_by_class: dict[str, set[str]] = defaultdict(set)
+ for cls, m in class_methods:
+ methods_by_class[cls].add(m)
+
+ frontier = set(live)
+ while frontier:
+ added: set[str] = set()
+
+ factory_method_roots: set[str] = set()
+ for owner in frontier:
+ for cls in classes_by_owner.get(owner, ()):
+ if cls not in live:
+ live.add(cls)
+ added.add(cls)
+ factory_method_roots |= methods_by_class[cls] - live
+ live |= factory_method_roots
+ added |= factory_method_roots
+ _walk(factory_method_roots, adjacency, live, added=added)
+
+ override_roots: set[str] = set()
+ stack = list(frontier | added)
+ while stack:
+ for overrider in override_rev.get(stack.pop(), ()):
+ if overrider not in live and overrider not in override_roots:
+ override_roots.add(overrider)
+ stack.append(overrider)
+ live |= override_roots
+ added |= override_roots
+ _walk(override_roots, adjacency, live, added=added)
+
+ frontier = added
+
+ dead = candidates - live
+ # Suppress generated files (openapi-ts client/core, routeTree.gen.ts) from
+ # the REPORT only, after reachability: they stay full participants as roots
+ # and callers, so a real function invoked only from generated glue is not
+ # newly flagged; excluding earlier would drop those live edges.
+ if config.exclude_patterns:
+ dead = {
+ qn
+ for qn in dead
+ if not any(
+ fnmatch(str(props_by_qn[qn].get(cs.KEY_PATH) or ""), pattern)
+ for pattern in config.exclude_patterns
+ )
+ }
+ return dead
+
+
+def _as_str_list(value: ResultValue | None) -> list[str]:
+ if not isinstance(value, list):
+ return []
+ return [str(item) for item in value]
+
+
+def _node_props(row: ResultRow) -> PropertyDict:
+ # Coalesce NULL column values at the fetch boundary so the engine never
+ # sees None where a str/list/bool is expected. Only properties the engine
+ # reads are kept; the report is built from the raw rows.
+ return {
+ cs.KEY_PATH: str(row.get(cs.KEY_PATH) or ""),
+ cs.KEY_DECORATORS: _as_str_list(row.get(cs.KEY_DECORATORS)),
+ cs.KEY_IS_EXPORTED: row.get(cs.KEY_IS_EXPORTED) is True,
+ cs.KEY_OVERRIDES_EXTERNAL: row.get(cs.KEY_OVERRIDES_EXTERNAL) is True,
+ }
+
+
+def _row_qn(row: ResultRow) -> str:
+ return str(row.get(cs.KEY_QUALIFIED_NAME) or "")
+
+
+def collect_dead_code(
+ ingestor: GraphQueryClient, project_name: str, config: DeadCodeConfig
+) -> list[ResultRow]:
+ prefix = project_name + cs.SEPARATOR_DOT
+ params: dict[str, PropertyValue] = {cs.KEY_PROJECT_PREFIX: prefix}
+
+ node_rows = ingestor.fetch_all(cq.CYPHER_DEAD_CODE_NODES, params)
+ nodes: dict[_NodeId, PropertyDict] = {
+ (str(row.get(cs.KEY_LABEL) or ""), _row_qn(row)): _node_props(row)
+ for row in node_rows
+ }
+
+ rels: list[_RelTuple] = [
+ (
+ str(row.get(cs.KEY_FROM_LABEL) or ""),
+ str(row.get(cs.KEY_FROM_QN) or ""),
+ str(row.get(cs.KEY_REL_TYPE) or ""),
+ str(row.get(cs.KEY_TO_LABEL) or ""),
+ str(row.get(cs.KEY_TO_QN) or ""),
+ )
+ for row in ingestor.fetch_all(cq.CYPHER_DEAD_CODE_RELS, params)
+ ]
+
+ dead = dead_code_from_graph(nodes, rels, prefix, config)
+ rows = [row for row in node_rows if _row_qn(row) in dead]
+ rows.sort(key=_row_qn)
+ return rows
diff --git a/codebase_rag/docker-compose.yaml b/codebase_rag/docker-compose.yaml
new file mode 100644
index 000000000..1b394c873
--- /dev/null
+++ b/codebase_rag/docker-compose.yaml
@@ -0,0 +1,27 @@
+services:
+ memgraph:
+ image: memgraph/memgraph-mage
+ ports:
+ - "${MEMGRAPH_PORT:-7687}:7687"
+ - "${MEMGRAPH_HTTP_PORT:-7444}:7444"
+ volumes:
+ - memgraph_data:/var/lib/memgraph
+ - memgraph_log:/var/log/memgraph
+ lab:
+ image: memgraph/lab
+ ports:
+ - "${LAB_PORT:-3000}:3000"
+ environment:
+ QUICK_CONNECT_MG_HOST: memgraph
+ qdrant:
+ image: qdrant/qdrant
+ ports:
+ - "${QDRANT_HTTP_PORT:-6333}:6333"
+ - "${QDRANT_GRPC_PORT:-6334}:6334"
+ volumes:
+ - qdrant_storage:/qdrant/storage
+
+volumes:
+ qdrant_storage:
+ memgraph_data:
+ memgraph_log:
diff --git a/codebase_rag/embedder.py b/codebase_rag/embedder.py
index 0928cae97..ba8e385fd 100644
--- a/codebase_rag/embedder.py
+++ b/codebase_rag/embedder.py
@@ -1,19 +1,183 @@
-# ┌────────────────────────────────────────────────────────────────────────┐
-# │ UniXcoder Model Singleton via LRU Cache │
-# ├────────────────────────────────────────────────────────────────────────┤
-# │ get_model() provides: │
-# │ - Singleton behavior without global variables │
-# │ - Thread-safe lazy initialization │
-# │ - Easy testability with cache_clear() method │
-# │ - Memory efficient with maxsize=1 │
-# └────────────────────────────────────────────────────────────────────────┘
+from __future__ import annotations
+
+import hashlib
+import json
+import os
from functools import lru_cache
+from pathlib import Path
+
+import httpx
+from loguru import logger
+from . import constants as cs
from . import exceptions as ex
-from .config import settings
-from .constants import UNIXCODER_MODEL
+from . import logs as ls
+from .config import API_KEY_INFO, settings
from .utils.dependencies import has_torch, has_transformers
+
+def _cache_namespace() -> str:
+ # Embeddings from different providers/models live in different vector
+ # spaces; namespacing cache keys prevents a provider or model switch
+ # from replaying stale vectors of the wrong space.
+ if settings.EMBEDDING_PROVIDER == cs.EmbeddingProvider.OPENAI:
+ namespace = f"{cs.EmbeddingProvider.OPENAI}:{settings.OPENAI_EMBEDDING_MODEL}"
+ if settings.OPENAI_EMBEDDING_DIMENSIONS is not None:
+ namespace = f"{namespace}:{settings.OPENAI_EMBEDDING_DIMENSIONS}"
+ return namespace
+ return f"{cs.EmbeddingProvider.UNIXCODER}:{cs.UNIXCODER_MODEL}"
+
+
+class EmbeddingCache:
+ __slots__ = ("_cache", "_path")
+
+ def __init__(self, path: Path | None = None) -> None:
+ self._cache: dict[str, list[float]] = {}
+ self._path = path
+
+ @staticmethod
+ def _content_hash(content: str) -> str:
+ return hashlib.sha256(f"{_cache_namespace()}\x00{content}".encode()).hexdigest()
+
+ def get(self, content: str) -> list[float] | None:
+ return self._cache.get(self._content_hash(content))
+
+ def put(self, content: str, embedding: list[float]) -> None:
+ self._cache[self._content_hash(content)] = embedding
+
+ def get_many(self, snippets: list[str]) -> dict[int, list[float]]:
+ results: dict[int, list[float]] = {}
+ for i, snippet in enumerate(snippets):
+ if (cached := self.get(snippet)) is not None:
+ results[i] = cached
+ return results
+
+ def put_many(self, snippets: list[str], embeddings: list[list[float]]) -> None:
+ for snippet, embedding in zip(snippets, embeddings):
+ self.put(snippet, embedding)
+
+ def save(self) -> None:
+ if self._path is None:
+ return
+ try:
+ self._path.parent.mkdir(parents=True, exist_ok=True)
+ with self._path.open("w", encoding="utf-8") as f:
+ json.dump(self._cache, f)
+ except Exception as e:
+ logger.warning(ls.EMBEDDING_CACHE_SAVE_FAILED, path=self._path, error=e)
+
+ def load(self) -> None:
+ if self._path is None or not self._path.exists():
+ return
+ try:
+ with self._path.open("r", encoding="utf-8") as f:
+ self._cache = json.load(f)
+ logger.debug(
+ ls.EMBEDDING_CACHE_LOADED, count=len(self._cache), path=self._path
+ )
+ except Exception as e:
+ logger.warning(ls.EMBEDDING_CACHE_LOAD_FAILED, path=self._path, error=e)
+ self._cache = {}
+
+ def clear(self) -> None:
+ self._cache.clear()
+
+ def __len__(self) -> int:
+ return len(self._cache)
+
+
+_embedding_cache: EmbeddingCache | None = None
+
+
+def get_embedding_cache() -> EmbeddingCache:
+ global _embedding_cache
+ if _embedding_cache is None:
+ cache_path = Path(settings.QDRANT_DB_PATH) / cs.EMBEDDING_CACHE_FILENAME
+ _embedding_cache = EmbeddingCache(path=cache_path)
+ _embedding_cache.load()
+ return _embedding_cache
+
+
+def clear_embedding_cache() -> None:
+ global _embedding_cache
+ if _embedding_cache is not None:
+ _embedding_cache.clear()
+ _embedding_cache = None
+
+
+def _openai_client() -> httpx.Client:
+ headers: dict[str, str] = {}
+ api_key = settings.OPENAI_EMBEDDING_API_KEY or os.environ.get(
+ API_KEY_INFO[cs.Provider.OPENAI]["env_var"]
+ )
+ if api_key:
+ headers["Authorization"] = f"Bearer {api_key}"
+ return httpx.Client(
+ base_url=settings.OPENAI_EMBEDDING_BASE_URL,
+ headers=headers,
+ timeout=settings.OPENAI_EMBEDDING_TIMEOUT,
+ )
+
+
+def _openai_embed_batch(snippets: list[str]) -> list[list[float]]:
+ embeddings: list[list[float]] = []
+ batch_size = settings.OPENAI_EMBEDDING_BATCH_SIZE
+ with _openai_client() as client:
+ for start in range(0, len(snippets), batch_size):
+ batch = snippets[start : start + batch_size]
+ payload: dict[str, object] = {
+ "model": settings.OPENAI_EMBEDDING_MODEL,
+ "input": batch,
+ }
+ if settings.OPENAI_EMBEDDING_DIMENSIONS is not None:
+ payload["dimensions"] = settings.OPENAI_EMBEDDING_DIMENSIONS
+ response = client.post(cs.OPENAI_EMBEDDINGS_PATH, json=payload)
+ if response.status_code != cs.HTTP_OK:
+ raise RuntimeError(
+ ex.OPENAI_EMBEDDING_HTTP_ERROR.format(
+ status=response.status_code, body=response.text[:500]
+ )
+ )
+ embeddings.extend(_parse_embedding_rows(response, len(batch)))
+ return embeddings
+
+
+def _parse_embedding_rows(response: httpx.Response, expected: int) -> list[list[float]]:
+ try:
+ rows = response.json()["data"]
+ except (json.JSONDecodeError, KeyError, TypeError) as e:
+ raise RuntimeError(
+ ex.OPENAI_EMBEDDING_MALFORMED_RESPONSE.format(error=e)
+ ) from e
+ if not isinstance(rows, list):
+ raise RuntimeError(
+ ex.OPENAI_EMBEDDING_MALFORMED_RESPONSE.format(error="'data' is not a list")
+ )
+ if len(rows) != expected:
+ raise RuntimeError(
+ ex.OPENAI_EMBEDDING_COUNT_MISMATCH.format(got=len(rows), expected=expected)
+ )
+ # Place each row at its declared index instead of sorting: a buggy
+ # server emitting duplicate or out-of-range indices must fail loudly
+ # rather than pair snippets with the wrong vectors.
+ placed: list[list[float] | None] = [None] * expected
+ for row in rows:
+ try:
+ index, embedding = row["index"], row["embedding"]
+ except (KeyError, TypeError) as e:
+ raise RuntimeError(
+ ex.OPENAI_EMBEDDING_MALFORMED_RESPONSE.format(error=e)
+ ) from e
+ if (
+ not isinstance(index, int)
+ or not 0 <= index < expected
+ or placed[index] is not None
+ ):
+ raise RuntimeError(ex.OPENAI_EMBEDDING_BAD_INDEX.format(index=index))
+ placed[index] = embedding
+ return [row for row in placed if row is not None]
+
+
if has_torch() and has_transformers():
import numpy as np
import torch
@@ -21,15 +185,53 @@
from .unixcoder import UniXcoder
+ def _device_available(device: cs.EmbeddingDevice) -> bool:
+ match device:
+ case cs.EmbeddingDevice.CUDA:
+ return torch.cuda.is_available()
+ case cs.EmbeddingDevice.MPS:
+ return torch.backends.mps.is_available()
+ case _:
+ return True
+
+ def _select_device() -> cs.EmbeddingDevice:
+ if (override := settings.EMBEDDING_DEVICE) is not None:
+ if _device_available(override):
+ return override
+ logger.warning(ls.EMBEDDING_DEVICE_UNAVAILABLE.format(device=override))
+ if torch.cuda.is_available():
+ return cs.EmbeddingDevice.CUDA
+ if torch.backends.mps.is_available():
+ return cs.EmbeddingDevice.MPS
+ return cs.EmbeddingDevice.CPU
+
+ _batches_since_cache_drop = 0
+
+ def _sync_after_batch(device: torch.device | str) -> None:
+ # MPS wedges inside Metal's waitUntilCompleted when command buffers
+ # accumulate across thousands of batches (issue #689); draining the
+ # stream after every batch keeps each command buffer short-lived,
+ # and a periodic (not per-batch: ~21% throughput cost) cache drop
+ # bounds Metal allocator growth over monorepo-scale runs.
+ if torch.device(device).type != cs.EmbeddingDevice.MPS:
+ return
+ torch.mps.synchronize()
+ global _batches_since_cache_drop
+ _batches_since_cache_drop += 1
+ if _batches_since_cache_drop >= cs.EMBEDDING_MPS_CACHE_DROP_INTERVAL:
+ torch.mps.empty_cache()
+ _batches_since_cache_drop = 0
+
@lru_cache(maxsize=1)
def get_model() -> UniXcoder:
- model = UniXcoder(UNIXCODER_MODEL)
+ model = UniXcoder(cs.UNIXCODER_MODEL)
model.eval()
- if torch.cuda.is_available():
- model = model.cuda()
+ device = _select_device()
+ if device != cs.EmbeddingDevice.CPU:
+ model = model.to(device)
return model
- def embed_code(code: str, max_length: int | None = None) -> list[float]:
+ def _unixcoder_embed_code(code: str, max_length: int | None) -> list[float]:
if max_length is None:
max_length = settings.EMBEDDING_MAX_LENGTH
model = get_model()
@@ -39,10 +241,89 @@ def embed_code(code: str, max_length: int | None = None) -> list[float]:
with torch.no_grad():
_, sentence_embeddings = model(tokens_tensor)
embedding: NDArray[np.float32] = sentence_embeddings.cpu().numpy()
+ _sync_after_batch(device)
result: list[float] = embedding[0].tolist()
return result
+ def _unixcoder_embed_batch(
+ snippets: list[str], max_length: int | None, batch_size: int
+ ) -> list[list[float]]:
+ if max_length is None:
+ max_length = settings.EMBEDDING_MAX_LENGTH
+ model = get_model()
+ device = next(model.parameters()).device
+
+ all_new_embeddings: list[list[float]] = []
+ for start in range(0, len(snippets), batch_size):
+ batch = snippets[start : start + batch_size]
+ tokens_list = model.tokenize(batch, max_length=max_length, padding=True)
+ tokens_tensor = torch.tensor(tokens_list).to(device)
+ with torch.no_grad():
+ _, sentence_embeddings = model(tokens_tensor)
+ batch_np: NDArray[np.float32] = sentence_embeddings.cpu().numpy()
+ _sync_after_batch(device)
+ for row in batch_np:
+ all_new_embeddings.append(row.tolist())
+ return all_new_embeddings
+
else:
- def embed_code(code: str, max_length: int | None = None) -> list[float]:
+ def _unixcoder_embed_code(code: str, max_length: int | None) -> list[float]:
raise RuntimeError(ex.SEMANTIC_EXTRA)
+
+ def _unixcoder_embed_batch(
+ snippets: list[str], max_length: int | None, batch_size: int
+ ) -> list[list[float]]:
+ raise RuntimeError(ex.SEMANTIC_EXTRA)
+
+
+def embed_code(code: str, max_length: int | None = None) -> list[float]:
+ cache = get_embedding_cache()
+ if (cached := cache.get(code)) is not None:
+ return cached
+ try:
+ if settings.EMBEDDING_PROVIDER == cs.EmbeddingProvider.OPENAI:
+ result = _openai_embed_batch([code])[0]
+ else:
+ result = _unixcoder_embed_code(code, max_length)
+ except Exception:
+ logger.exception(ls.EMBEDDING_SNIPPET_FAILED, length=len(code))
+ raise
+ cache.put(code, result)
+ return result
+
+
+def embed_code_batch(
+ snippets: list[str],
+ max_length: int | None = None,
+ batch_size: int = cs.EMBEDDING_DEFAULT_BATCH_SIZE,
+) -> list[list[float]]:
+ if not snippets:
+ return []
+
+ cache = get_embedding_cache()
+ cached_results = cache.get_many(snippets)
+
+ if len(cached_results) == len(snippets):
+ logger.debug(ls.EMBEDDING_CACHE_HIT, count=len(snippets))
+ return [cached_results[i] for i in range(len(snippets))]
+
+ uncached_indices = [i for i in range(len(snippets)) if i not in cached_results]
+ uncached_snippets = [snippets[i] for i in uncached_indices]
+
+ if settings.EMBEDDING_PROVIDER == cs.EmbeddingProvider.OPENAI:
+ all_new_embeddings = _openai_embed_batch(uncached_snippets)
+ else:
+ all_new_embeddings = _unixcoder_embed_batch(
+ uncached_snippets, max_length, batch_size
+ )
+
+ cache.put_many(uncached_snippets, all_new_embeddings)
+
+ results: list[list[float]] = [[] for _ in snippets]
+ for i, emb in cached_results.items():
+ results[i] = emb
+ for idx, orig_i in enumerate(uncached_indices):
+ results[orig_i] = all_new_embeddings[idx]
+
+ return results
diff --git a/codebase_rag/exceptions.py b/codebase_rag/exceptions.py
index f30202395..6a5638600 100644
--- a/codebase_rag/exceptions.py
+++ b/codebase_rag/exceptions.py
@@ -1,4 +1,4 @@
-# (H) Provider validation errors
+# Provider validation errors
GOOGLE_GLA_NO_KEY = (
"Gemini GLA provider requires api_key. "
"Set ORCHESTRATOR_API_KEY or CYPHER_API_KEY in .env file."
@@ -11,16 +11,51 @@
"OpenAI provider requires api_key. "
"Set ORCHESTRATOR_API_KEY or CYPHER_API_KEY in .env file."
)
+ANTHROPIC_NO_KEY = (
+ "Anthropic provider requires api_key. "
+ "Set ORCHESTRATOR_API_KEY or CYPHER_API_KEY in .env file."
+)
+AZURE_NO_KEY = "Azure OpenAI provider requires api_key. Set AZURE_API_KEY in .env file."
+AZURE_NO_ENDPOINT = (
+ "Azure OpenAI provider requires endpoint. Set AZURE_OPENAI_ENDPOINT in .env file."
+)
+MINIMAX_NO_KEY = (
+ "MiniMax provider requires api_key. "
+ "Set ORCHESTRATOR_API_KEY or CYPHER_API_KEY or MINIMAX_API_KEY in .env file."
+)
OLLAMA_NOT_RUNNING = (
"Ollama server not responding at {endpoint}. "
"Make sure Ollama is running: ollama serve"
)
+LITELLM_NO_ENDPOINT = (
+ "LiteLLM provider requires endpoint. "
+ "Set ORCHESTRATOR_ENDPOINT or CYPHER_ENDPOINT in .env file."
+)
+LITELLM_NOT_RUNNING = (
+ "LiteLLM proxy server not responding at {endpoint}. "
+ "Make sure LiteLLM proxy is running and API key is valid."
+)
UNKNOWN_PROVIDER = "Unknown provider '{provider}'. Available providers: {available}"
-# (H) Dependency errors
+# Dependency errors
SEMANTIC_EXTRA = "Semantic search requires 'semantic' extra: uv sync --extra semantic"
-# (H) Configuration errors
+# OpenAI-compatible embedding errors
+OPENAI_EMBEDDING_HTTP_ERROR = (
+ "OpenAI-compatible embedding request failed with status {status}: {body}"
+)
+OPENAI_EMBEDDING_COUNT_MISMATCH = (
+ "OpenAI-compatible embedding response returned {got} embeddings "
+ "for {expected} inputs"
+)
+OPENAI_EMBEDDING_MALFORMED_RESPONSE = (
+ "OpenAI-compatible embedding response is malformed: {error}"
+)
+OPENAI_EMBEDDING_BAD_INDEX = (
+ "OpenAI-compatible embedding response has an invalid or duplicate index: {index}"
+)
+
+# Configuration errors
PROVIDER_EMPTY = "Provider name cannot be empty in 'provider:model' format."
MODEL_ID_EMPTY = "Model ID cannot be empty."
MODEL_FORMAT_INVALID = (
@@ -29,31 +64,44 @@
BATCH_SIZE_POSITIVE = "batch_size must be a positive integer"
CONFIG = "{role} configuration error: {error}"
-# (H) Graph loading errors
+# Graph loading errors
GRAPH_FILE_NOT_FOUND = "Graph file not found: {path}"
FAILED_TO_LOAD_DATA = "Failed to load data from file"
NODES_NOT_LOADED = "Nodes should be loaded"
RELATIONSHIPS_NOT_LOADED = "Relationships should be loaded"
DATA_NOT_LOADED = "Data should be loaded"
-# (H) Parser errors
+# Parser errors
NO_LANGUAGES = "No Tree-sitter languages available."
-# (H) LLM errors
+# LLM errors
LLM_INIT_CYPHER = "Failed to initialize CypherGenerator: {error}"
LLM_INVALID_QUERY = "LLM did not generate a valid query. Output: {output}"
+LLM_DANGEROUS_QUERY = "LLM generated a destructive Cypher query (found '{keyword}'). Query rejected: {query}"
+LLM_UNBOUNDED_PATH = (
+ "LLM generated an unbounded variable-length path pattern "
+ "(e.g. [:TYPE*] or [:TYPE*N..]) which causes memory exhaustion on cyclic graphs. "
+ "Add an upper bound such as [:TYPE*1..6]. Query rejected: {query}"
+)
+LLM_DISALLOWED_PROCEDURE = (
+ "LLM generated a CALL to procedure '{name}' which is outside the read-only "
+ "MAGE allowlist. Query rejected: {query}"
+)
LLM_GENERATION_FAILED = "Cypher generation failed: {error}"
LLM_INIT_ORCHESTRATOR = "Failed to initialize RAG Orchestrator: {error}"
-# (H) Graph service errors
+# Graph service errors
BATCH_SIZE = "batch_size must be a positive integer"
CONN = "Not connected to Memgraph."
+AUTH_INCOMPLETE = (
+ "Both username and password are required for authentication. "
+ "Either provide both or neither."
+)
-# (H) Access control errors (used with raise)
+# Access control errors (used with raise)
ACCESS_DENIED = "Access denied: Cannot access files outside the project root."
-DOC_UNSUPPORTED_PROVIDER = "DocumentAnalyzer does not support the 'local' LLM provider."
-# (H) Exception classes
+# Exception classes
class LLMGenerationError(Exception):
pass
diff --git a/codebase_rag/function_registry.py b/codebase_rag/function_registry.py
new file mode 100644
index 000000000..dd990d7de
--- /dev/null
+++ b/codebase_rag/function_registry.py
@@ -0,0 +1,264 @@
+# Trie-backed registry of every defined function/method qualified name, with
+# the auxiliary indices resolution needs: simple-name lookup, ending-with
+# cache, duplicate-QN variants, property/abstract markers, callable params.
+
+import sys
+from collections.abc import Callable, ItemsView, KeysView
+
+from . import constants as cs
+from .types_defs import (
+ FunctionRegistry,
+ NodeType,
+ QualifiedName,
+ SimpleNameLookup,
+ TrieNode,
+)
+
+
+class FunctionRegistryTrie:
+ __slots__ = (
+ "root",
+ "_entries",
+ "_simple_name_lookup",
+ "_ending_with_cache",
+ "_duplicates",
+ "_properties",
+ "_property_names",
+ "_abstracts",
+ "_callable_params",
+ )
+
+ def __init__(self, simple_name_lookup: SimpleNameLookup | None = None) -> None:
+ self.root: TrieNode = {}
+ self._entries: FunctionRegistry = {}
+ self._simple_name_lookup = simple_name_lookup
+ self._ending_with_cache: dict[str, list[QualifiedName]] = {}
+ self._duplicates: dict[QualifiedName, list[QualifiedName]] = {}
+ self._properties: set[QualifiedName] = set()
+ self._property_names: set[str] = set()
+ self._abstracts: set[QualifiedName] = set()
+ self._callable_params: dict[QualifiedName, dict[str, int]] = {}
+
+ def mark_callable_params(
+ self, qualified_name: QualifiedName, params: dict[str, int]
+ ) -> None:
+ if params:
+ self._callable_params[qualified_name] = params
+
+ def callable_params(self, qualified_name: QualifiedName) -> dict[str, int] | None:
+ return self._callable_params.get(qualified_name)
+
+ def mark_property(self, qualified_name: QualifiedName) -> None:
+ self._properties.add(qualified_name)
+ self._property_names.add(qualified_name.rsplit(cs.SEPARATOR_DOT, 1)[-1])
+
+ def is_property(self, qualified_name: QualifiedName) -> bool:
+ return qualified_name in self._properties
+
+ def property_names(self) -> set[str]:
+ return self._property_names
+
+ def mark_abstract(self, qualified_name: QualifiedName) -> None:
+ self._abstracts.add(qualified_name)
+
+ def is_abstract(self, qualified_name: QualifiedName) -> bool:
+ return qualified_name in self._abstracts
+
+ def register_unique_qn(
+ self, natural_qn: QualifiedName, start_line: int
+ ) -> QualifiedName:
+ if natural_qn not in self._entries:
+ return natural_qn
+ variant = f"{natural_qn}{cs.DUP_QN_MARKER}{start_line}"
+ bucket = self._duplicates.setdefault(natural_qn, [natural_qn])
+ if variant not in bucket:
+ bucket.append(variant)
+ return variant
+
+ def variants(self, qualified_name: QualifiedName) -> list[QualifiedName]:
+ return self._duplicates.get(qualified_name, [qualified_name])
+
+ def insert(self, qualified_name: QualifiedName, func_type: NodeType) -> None:
+ qualified_name = sys.intern(qualified_name)
+ self._entries[qualified_name] = func_type
+
+ simple_name = qualified_name.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ if self._simple_name_lookup is not None:
+ self._simple_name_lookup[simple_name].add(qualified_name)
+ self._invalidate_ending_with_cache(qualified_name, simple_name)
+
+ parts = qualified_name.split(cs.SEPARATOR_DOT)
+ current: TrieNode = self.root
+
+ for part in parts:
+ if part not in current:
+ current[part] = {}
+ child = current[part]
+ assert isinstance(child, dict)
+ current = child
+
+ current[cs.TRIE_TYPE_KEY] = func_type
+ current[cs.TRIE_QN_KEY] = qualified_name
+
+ def get(
+ self, qualified_name: QualifiedName, default: NodeType | None = None
+ ) -> NodeType | None:
+ return self._entries.get(qualified_name, default)
+
+ def __contains__(self, qualified_name: QualifiedName) -> bool:
+ return qualified_name in self._entries
+
+ def __getitem__(self, qualified_name: QualifiedName) -> NodeType:
+ return self._entries[qualified_name]
+
+ def __setitem__(self, qualified_name: QualifiedName, func_type: NodeType) -> None:
+ self.insert(qualified_name, func_type)
+
+ def __delitem__(self, qualified_name: QualifiedName) -> None:
+ if qualified_name not in self._entries:
+ return
+
+ del self._entries[qualified_name]
+ self._duplicates.pop(qualified_name, None)
+ for natural, bucket in list(self._duplicates.items()):
+ if qualified_name in bucket:
+ bucket.remove(qualified_name)
+ if len(bucket) <= 1:
+ self._duplicates.pop(natural, None)
+ simple_name = qualified_name.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+
+ if qualified_name in self._properties:
+ self._properties.discard(qualified_name)
+ if not any(
+ p.rsplit(cs.SEPARATOR_DOT, 1)[-1] == simple_name
+ for p in self._properties
+ ):
+ self._property_names.discard(simple_name)
+ self._abstracts.discard(qualified_name)
+ self._callable_params.pop(qualified_name, None)
+
+ self._invalidate_ending_with_cache(qualified_name, simple_name)
+
+ if self._simple_name_lookup is not None:
+ if simple_name in self._simple_name_lookup:
+ self._simple_name_lookup[simple_name].discard(qualified_name)
+
+ parts = qualified_name.split(cs.SEPARATOR_DOT)
+ self._cleanup_trie_path(parts, self.root)
+
+ def _cleanup_trie_path(self, parts: list[str], node: TrieNode) -> bool:
+ if not parts:
+ node.pop(cs.TRIE_QN_KEY, None)
+ node.pop(cs.TRIE_TYPE_KEY, None)
+ return not node
+
+ part = parts[0]
+ if part not in node:
+ return False
+
+ child = node[part]
+ assert isinstance(child, dict)
+ if self._cleanup_trie_path(parts[1:], child):
+ del node[part]
+
+ is_endpoint = cs.TRIE_QN_KEY in node
+ has_children = any(not key.startswith(cs.TRIE_INTERNAL_PREFIX) for key in node)
+ return not has_children and not is_endpoint
+
+ def _navigate_to_prefix(self, prefix: str) -> TrieNode | None:
+ parts = prefix.split(cs.SEPARATOR_DOT) if prefix else []
+ current: TrieNode = self.root
+ for part in parts:
+ if part not in current:
+ return None
+ child = current[part]
+ assert isinstance(child, dict)
+ current = child
+ return current
+
+ def _collect_from_subtree(
+ self,
+ node: TrieNode,
+ filter_fn: Callable[[QualifiedName], bool] | None = None,
+ ) -> list[tuple[QualifiedName, NodeType]]:
+ results: list[tuple[QualifiedName, NodeType]] = []
+
+ def dfs(n: TrieNode) -> None:
+ if cs.TRIE_QN_KEY in n:
+ qn = n[cs.TRIE_QN_KEY]
+ func_type = n[cs.TRIE_TYPE_KEY]
+ assert isinstance(qn, str) and isinstance(func_type, NodeType)
+ if filter_fn is None or filter_fn(qn):
+ results.append((qn, func_type))
+
+ for key, child in n.items():
+ if not key.startswith(cs.TRIE_INTERNAL_PREFIX):
+ assert isinstance(child, dict)
+ dfs(child)
+
+ dfs(node)
+ return results
+
+ def keys(self) -> KeysView[QualifiedName]:
+ return self._entries.keys()
+
+ def items(self) -> ItemsView[QualifiedName, NodeType]:
+ return self._entries.items()
+
+ def __len__(self) -> int:
+ return len(self._entries)
+
+ def find_with_prefix_and_suffix(
+ self, prefix: str, suffix: str
+ ) -> list[QualifiedName]:
+ node = self._navigate_to_prefix(prefix)
+ if node is None:
+ return []
+ suffix_pattern = f".{suffix}"
+ matches = self._collect_from_subtree(
+ node, lambda qn: qn.endswith(suffix_pattern)
+ )
+ return [qn for qn, _ in matches]
+
+ def _invalidate_ending_with_cache(
+ self, qualified_name: QualifiedName, simple_name: str
+ ) -> None:
+ if not self._ending_with_cache:
+ return
+ self._ending_with_cache.pop(simple_name, None)
+ # dotted suffixes are cached too (#513); drop any the qn ends with.
+ for key in [
+ k
+ for k in self._ending_with_cache
+ if cs.SEPARATOR_DOT in k and qualified_name.endswith(f".{k}")
+ ]:
+ del self._ending_with_cache[key]
+
+ def find_ending_with(self, suffix: str) -> list[QualifiedName]:
+ cached = self._ending_with_cache.get(suffix)
+ if cached is not None:
+ return cached
+ if self._simple_name_lookup is not None:
+ if suffix in self._simple_name_lookup:
+ result = sorted(self._simple_name_lookup[suffix])
+ elif cs.SEPARATOR_DOT in suffix:
+ # #513: the index only holds last segments, so a dotted
+ # suffix ("Class.method") always misses it; fall back to
+ # the linear scan instead of dropping the match.
+ result = sorted(
+ qn for qn in self._entries.keys() if qn.endswith(f".{suffix}")
+ )
+ else:
+ # dot-free miss is authoritative: insert() indexes every
+ # entry's last segment, so nothing can end with ".suffix".
+ result = []
+ else:
+ result = sorted(
+ qn for qn in self._entries.keys() if qn.endswith(f".{suffix}")
+ )
+ self._ending_with_cache[suffix] = result
+ return result
+
+ def find_with_prefix(self, prefix: str) -> list[tuple[QualifiedName, NodeType]]:
+ node = self._navigate_to_prefix(prefix)
+ return [] if node is None else self._collect_from_subtree(node)
diff --git a/codebase_rag/graph_audit.py b/codebase_rag/graph_audit.py
new file mode 100644
index 000000000..4847ae1d8
--- /dev/null
+++ b/codebase_rag/graph_audit.py
@@ -0,0 +1,281 @@
+# Structural integrity audit of a generated knowledge graph (issue #646).
+# Validates recorded node and relationship batches against the documented
+# schema (types_defs.NODE_SCHEMAS / RELATIONSHIP_SCHEMAS): orphan nodes,
+# required-property completeness, and undocumented labels, properties, and
+# relationship endpoint triples.
+from collections.abc import Callable, Sequence
+
+from . import constants as cs
+from . import cypher_queries as cq
+from .types_defs import (
+ NODE_SCHEMAS,
+ RELATIONSHIP_SCHEMAS,
+ AuditViolation,
+ GraphNodeRecord,
+ GraphRelRecord,
+ PropertyValue,
+ ResultRow,
+)
+
+
+def documented_node_properties() -> dict[str, dict[str, bool]]:
+ """Parse NODE_SCHEMAS property strings into {label: {prop: required}}."""
+ parsed: dict[str, dict[str, bool]] = {}
+ for schema in NODE_SCHEMAS:
+ props: dict[str, bool] = {}
+ for entry in schema.properties.strip(cs.SCHEMA_PROPS_BRACES).split(
+ cs.CHAR_COMMA
+ ):
+ # A trailing comma or stray whitespace in a schema string must
+ # not register an empty-named required property.
+ if not (entry := entry.strip()):
+ continue
+ name, _, type_part = entry.partition(cs.CHAR_COLON)
+ props[name.strip()] = not type_part.strip().endswith(
+ cs.SCHEMA_OPTIONAL_SUFFIX
+ )
+ parsed[schema.label.value] = props
+ return parsed
+
+
+def documented_relationship_triples() -> frozenset[tuple[str, str, str]]:
+ return frozenset(
+ (source.value, schema.rel_type.value, target.value)
+ for schema in RELATIONSHIP_SCHEMAS
+ for source in schema.sources
+ for target in schema.targets
+ )
+
+
+def _node_key(node: GraphNodeRecord) -> PropertyValue:
+ if key_prop := cs.NODE_UNIQUE_CONSTRAINTS.get(node.label):
+ return node.properties.get(key_prop)
+ return None
+
+
+def merge_node_records(
+ nodes: Sequence[GraphNodeRecord],
+) -> list[GraphNodeRecord]:
+ """Collapse repeated ensures of one node, merging properties.
+
+ Mirrors the ingestor's MERGE ... SET n += props semantics: the stored
+ node carries the union of all batched property dicts.
+ """
+ merged: dict[tuple[str, PropertyValue], GraphNodeRecord] = {}
+ for node in nodes:
+ identity = (node.label, _node_key(node))
+ if existing := merged.get(identity):
+ existing.properties.update(node.properties)
+ else:
+ merged[identity] = GraphNodeRecord(node.label, dict(node.properties))
+ return list(merged.values())
+
+
+def _node_identities(
+ nodes: Sequence[GraphNodeRecord],
+) -> set[tuple[str, PropertyValue]]:
+ return {(node.label, _node_key(node)) for node in merge_node_records(nodes)}
+
+
+def find_orphans(
+ nodes: Sequence[GraphNodeRecord], relationships: Sequence[GraphRelRecord]
+) -> list[GraphNodeRecord]:
+ identities = _node_identities(nodes)
+ connected: set[tuple[str, PropertyValue]] = set()
+ for rel in relationships:
+ # The database MERGEs a relationship by MATCHing both endpoints, so a
+ # dangling edge is silently dropped and must not count as connectivity
+ # for the endpoint that does exist.
+ endpoints = [(label, value) for label, _, value in (rel.from_spec, rel.to_spec)]
+ if all(endpoint in identities for endpoint in endpoints):
+ connected.update(endpoints)
+ return [
+ node
+ for node in merge_node_records(nodes)
+ # A repo with no indexable content is just its Project root, so a
+ # zero-degree Project is valid rather than a construction failure.
+ if node.label != cs.NodeLabel.PROJECT.value
+ and (node.label, _node_key(node)) not in connected
+ ]
+
+
+def find_dangling_relationships(
+ nodes: Sequence[GraphNodeRecord], relationships: Sequence[GraphRelRecord]
+) -> list[AuditViolation]:
+ identities = _node_identities(nodes)
+ violations: list[AuditViolation] = []
+ seen: set[tuple[str, PropertyValue, str, str, PropertyValue]] = set()
+ for rel in relationships:
+ from_label, _, from_key = rel.from_spec
+ to_label, _, to_key = rel.to_spec
+ if (from_label, from_key) in identities and (to_label, to_key) in identities:
+ continue
+ signature = (from_label, from_key, rel.rel_type, to_label, to_key)
+ if signature in seen:
+ continue
+ seen.add(signature)
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.DANGLING_RELATIONSHIP,
+ cs.AUDIT_DETAIL_DANGLING.format(
+ from_label=from_label,
+ from_key=from_key,
+ rel_type=rel.rel_type,
+ to_label=to_label,
+ to_key=to_key,
+ ),
+ )
+ )
+ return violations
+
+
+def find_property_violations(
+ nodes: Sequence[GraphNodeRecord],
+) -> list[AuditViolation]:
+ documented = documented_node_properties()
+ violations: list[AuditViolation] = []
+ for node in merge_node_records(nodes):
+ schema_props = documented.get(node.label)
+ if schema_props is None:
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.UNDOCUMENTED_LABEL,
+ cs.AUDIT_DETAIL_UNDOCUMENTED_LABEL.format(label=node.label),
+ )
+ )
+ continue
+ key = _node_key(node)
+ for prop in node.properties:
+ if prop not in schema_props:
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.UNDOCUMENTED_PROPERTY,
+ cs.AUDIT_DETAIL_UNDOCUMENTED_PROPERTY.format(
+ label=node.label, key=key, prop=prop
+ ),
+ )
+ )
+ for prop, required in schema_props.items():
+ if required and node.properties.get(prop) is None:
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.MISSING_REQUIRED_PROPERTY,
+ cs.AUDIT_DETAIL_MISSING_REQUIRED.format(
+ label=node.label, key=key, prop=prop
+ ),
+ )
+ )
+ return violations
+
+
+def find_relationship_violations(
+ relationships: Sequence[GraphRelRecord],
+) -> list[AuditViolation]:
+ documented = documented_relationship_triples()
+ violations: list[AuditViolation] = []
+ seen: set[tuple[str, str, str]] = set()
+ for rel in relationships:
+ triple = (rel.from_spec[0], rel.rel_type, rel.to_spec[0])
+ if triple in documented or triple in seen:
+ continue
+ seen.add(triple)
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.UNDOCUMENTED_RELATIONSHIP,
+ cs.AUDIT_DETAIL_UNDOCUMENTED_RELATIONSHIP.format(
+ from_label=triple[0], rel_type=triple[1], to_label=triple[2]
+ ),
+ )
+ )
+ return violations
+
+
+def build_missing_required_query(label: str, required_props: Sequence[str]) -> str:
+ conditions = cq.CYPHER_AUDIT_OR.join(
+ cq.CYPHER_AUDIT_IS_NULL.format(prop=prop) for prop in required_props
+ )
+ return cq.CYPHER_AUDIT_MISSING_REQUIRED.format(label=label, conditions=conditions)
+
+
+def collect_live_violations(
+ fetch_all: Callable[[str], Sequence[ResultRow]],
+) -> list[AuditViolation]:
+ """Run the structural audit against a live graph via Cypher (doctor)."""
+ violations: list[AuditViolation] = []
+ for row in fetch_all(cq.CYPHER_AUDIT_ORPHANS):
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.ORPHAN_NODE,
+ cs.AUDIT_DETAIL_ORPHAN_COUNT.format(
+ count=row["orphans"], label=row["label"]
+ ),
+ )
+ )
+ documented_props = documented_node_properties()
+ for row in fetch_all(cq.CYPHER_AUDIT_LABELS):
+ if row["label"] not in documented_props:
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.UNDOCUMENTED_LABEL,
+ cs.AUDIT_DETAIL_UNDOCUMENTED_LABEL.format(label=row["label"]),
+ )
+ )
+ documented_triples = documented_relationship_triples()
+ for row in fetch_all(cq.CYPHER_AUDIT_REL_TRIPLES):
+ if (row["src"], row["rel"], row["dst"]) not in documented_triples:
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.UNDOCUMENTED_RELATIONSHIP,
+ cs.AUDIT_DETAIL_UNDOCUMENTED_RELATIONSHIP.format(
+ from_label=row["src"],
+ rel_type=row["rel"],
+ to_label=row["dst"],
+ ),
+ )
+ )
+ for row in fetch_all(cq.CYPHER_AUDIT_LABEL_PROPS):
+ # Unknown labels are already reported above; only grade documented ones.
+ if (schema_props := documented_props.get(str(row["label"]))) and str(
+ row["key"]
+ ) not in schema_props:
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.UNDOCUMENTED_PROPERTY,
+ cs.AUDIT_DETAIL_UNDOCUMENTED_PROPERTY_LIVE.format(
+ label=row["label"], prop=row["key"]
+ ),
+ )
+ )
+ for label, schema_props in documented_props.items():
+ required = [prop for prop, is_required in schema_props.items() if is_required]
+ # An all-optional schema would render an empty WHERE clause, which
+ # is a Cypher syntax error.
+ if not required:
+ continue
+ rows = fetch_all(build_missing_required_query(label, required))
+ if rows and (count := rows[0]["missing"]):
+ violations.append(
+ AuditViolation(
+ cs.AuditCheck.MISSING_REQUIRED_PROPERTY,
+ cs.AUDIT_DETAIL_MISSING_REQUIRED_LIVE.format(
+ count=count, label=label
+ ),
+ )
+ )
+ return violations
+
+
+def collect_violations(
+ nodes: Sequence[GraphNodeRecord], relationships: Sequence[GraphRelRecord]
+) -> list[AuditViolation]:
+ violations = [
+ AuditViolation(
+ cs.AuditCheck.ORPHAN_NODE,
+ cs.AUDIT_DETAIL_ORPHAN.format(label=node.label, key=_node_key(node)),
+ )
+ for node in find_orphans(nodes, relationships)
+ ]
+ violations.extend(find_property_violations(nodes))
+ violations.extend(find_relationship_violations(relationships))
+ violations.extend(find_dangling_relationships(nodes, relationships))
+ return violations
diff --git a/codebase_rag/graph_loader.py b/codebase_rag/graph_loader.py
index b69635755..6a210c6d5 100644
--- a/codebase_rag/graph_loader.py
+++ b/codebase_rag/graph_loader.py
@@ -13,6 +13,18 @@
class GraphLoader:
+ __slots__ = (
+ "file_path",
+ "_data",
+ "_nodes",
+ "_relationships",
+ "_nodes_by_id",
+ "_nodes_by_label",
+ "_outgoing_rels",
+ "_incoming_rels",
+ "_property_indexes",
+ )
+
def __init__(self, file_path: str):
self.file_path = Path(file_path)
self._data: GraphData | None = None
diff --git a/codebase_rag/graph_updater.py b/codebase_rag/graph_updater.py
index 2620d2bcb..0a6d12394 100644
--- a/codebase_rag/graph_updater.py
+++ b/codebase_rag/graph_updater.py
@@ -1,250 +1,302 @@
+"""Orchestrate parsing a repository into graph nodes and edges and ingest them."""
+
+import hashlib
+import json
+import os
import sys
-from collections import OrderedDict, defaultdict
-from collections.abc import Callable, ItemsView, KeysView
+from collections import defaultdict
+from collections.abc import Callable, Mapping
from pathlib import Path
from loguru import logger
-from tree_sitter import Node, Parser
+from rich.progress import Progress, SpinnerColumn, TextColumn
+from tree_sitter import Node, Parser, QueryCursor
from . import constants as cs
from . import logs as ls
+from .analyzers import FindingAnalyzer
+from .ast_cache import BoundedASTCache
+from .capture import CaptureSelection, default_capture
from .config import settings
-from .language_spec import LANGUAGE_FQN_SPECS, get_language_spec
+from .function_registry import FunctionRegistryTrie
+from .language_spec import (
+ LANGUAGE_FQN_SPECS,
+ get_language_for_extension,
+ get_language_spec,
+)
+from .parser_fingerprint import compute_parser_fingerprint
+from .parser_loader import COMBINED_FUNC_CLASS_IMPORT_QUERIES
+from .parsers.ast_grep_tier import AstGrepTier
+from .parsers.cpp.preproc_recovery import parse_with_preproc_recovery
+from .parsers.cpp_frontend import (
+ cpp_frontend_available,
+ find_compile_commands,
+ run_cpp_frontend,
+ run_cpp_frontend_hybrid,
+)
+from .parsers.csharp_frontend import (
+ CSharpQueryCall,
+ csharp_frontend_available,
+ find_csharp_project,
+ run_csharp_frontend,
+)
+from .parsers.endpoint_prefixes import (
+ CYPHER_DELETE_HANDLER_EXPOSES,
+ CYPHER_PROJECT_PY_MODULES,
+ CYPHER_PROJECT_ROUTE_HANDLERS,
+ build_router_registry,
+)
+from .parsers.endpoint_routes import (
+ CYPHER_DELETE_MODULE_EXPOSES,
+ CYPHER_PROJECT_MODULES,
+ ROUTE_CALL_LANGUAGES,
+ ROUTE_MODULE_EXTENSIONS,
+ RouteRegistration,
+ collect_route_registrations,
+)
+from .parsers.endpoints import (
+ _emit_endpoint,
+ emit_endpoints,
+ link_endpoints,
+ parse_route_decorator,
+)
from .parsers.factory import ProcessorFactory
-from .services import IngestorProtocol, QueryProtocol
+from .parsers.utils import sorted_captures
+from .services import FilteringIngestor, IngestorProtocol, QueryProtocol
+from .services.resource_cleanup import prune_unanchored_resources
from .types_defs import (
+ CppDefinitionSpan,
EmbeddingQueryResult,
- FunctionRegistry,
+ FunctionLocation,
LanguageQueries,
NodeType,
- QualifiedName,
+ PendingExpansionCall,
+ PendingMacroCall,
ResultRow,
SimpleNameLookup,
- TrieNode,
)
from .utils.dependencies import has_semantic_dependencies
from .utils.fqn_resolver import find_function_source_by_fqn
-from .utils.path_utils import should_skip_path
+from .utils.path_utils import (
+ cached_relative_path,
+ matches_ignore_patterns,
+ should_skip_path,
+ should_skip_rel_file,
+ unignore_could_match_within,
+)
from .utils.source_extraction import extract_source_with_fallback
-class FunctionRegistryTrie:
- def __init__(self, simple_name_lookup: SimpleNameLookup | None = None) -> None:
- self.root: TrieNode = {}
- self._entries: FunctionRegistry = {}
- self._simple_name_lookup = simple_name_lookup
-
- def insert(self, qualified_name: QualifiedName, func_type: NodeType) -> None:
- self._entries[qualified_name] = func_type
-
- parts = qualified_name.split(cs.SEPARATOR_DOT)
- current: TrieNode = self.root
-
- for part in parts:
- if part not in current:
- current[part] = {}
- child = current[part]
- assert isinstance(child, dict)
- current = child
-
- current[cs.TRIE_TYPE_KEY] = func_type
- current[cs.TRIE_QN_KEY] = qualified_name
-
- def get(
- self, qualified_name: QualifiedName, default: NodeType | None = None
- ) -> NodeType | None:
- return self._entries.get(qualified_name, default)
-
- def __contains__(self, qualified_name: QualifiedName) -> bool:
- return qualified_name in self._entries
-
- def __getitem__(self, qualified_name: QualifiedName) -> NodeType:
- return self._entries[qualified_name]
-
- def __setitem__(self, qualified_name: QualifiedName, func_type: NodeType) -> None:
- self.insert(qualified_name, func_type)
-
- def __delitem__(self, qualified_name: QualifiedName) -> None:
- if qualified_name not in self._entries:
- return
-
- del self._entries[qualified_name]
-
- parts = qualified_name.split(cs.SEPARATOR_DOT)
- self._cleanup_trie_path(parts, self.root)
-
- def _cleanup_trie_path(self, parts: list[str], node: TrieNode) -> bool:
- if not parts:
- node.pop(cs.TRIE_QN_KEY, None)
- node.pop(cs.TRIE_TYPE_KEY, None)
- return not node
-
- part = parts[0]
- if part not in node:
- return False
-
- child = node[part]
- assert isinstance(child, dict)
- if self._cleanup_trie_path(parts[1:], child):
- del node[part]
-
- is_endpoint = cs.TRIE_QN_KEY in node
- has_children = any(not key.startswith(cs.TRIE_INTERNAL_PREFIX) for key in node)
- return not has_children and not is_endpoint
-
- def _navigate_to_prefix(self, prefix: str) -> TrieNode | None:
- parts = prefix.split(cs.SEPARATOR_DOT) if prefix else []
- current: TrieNode = self.root
- for part in parts:
- if part not in current:
- return None
- child = current[part]
- assert isinstance(child, dict)
- current = child
- return current
-
- def _collect_from_subtree(
- self,
- node: TrieNode,
- filter_fn: Callable[[QualifiedName], bool] | None = None,
- ) -> list[tuple[QualifiedName, NodeType]]:
- results: list[tuple[QualifiedName, NodeType]] = []
-
- def dfs(n: TrieNode) -> None:
- if cs.TRIE_QN_KEY in n:
- qn = n[cs.TRIE_QN_KEY]
- func_type = n[cs.TRIE_TYPE_KEY]
- assert isinstance(qn, str) and isinstance(func_type, NodeType)
- if filter_fn is None or filter_fn(qn):
- results.append((qn, func_type))
-
- for key, child in n.items():
- if not key.startswith(cs.TRIE_INTERNAL_PREFIX):
- assert isinstance(child, dict)
- dfs(child)
-
- dfs(node)
- return results
-
- def keys(self) -> KeysView[QualifiedName]:
- return self._entries.keys()
-
- def items(self) -> ItemsView[QualifiedName, NodeType]:
- return self._entries.items()
-
- def __len__(self) -> int:
- return len(self._entries)
-
- def find_with_prefix_and_suffix(
- self, prefix: str, suffix: str
- ) -> list[QualifiedName]:
- node = self._navigate_to_prefix(prefix)
- if node is None:
- return []
- suffix_pattern = f".{suffix}"
- matches = self._collect_from_subtree(
- node, lambda qn: qn.endswith(suffix_pattern)
- )
- return [qn for qn, _ in matches]
-
- def find_ending_with(self, suffix: str) -> list[QualifiedName]:
- if self._simple_name_lookup is not None and suffix in self._simple_name_lookup:
- # (H) O(1) lookup using the simple_name_lookup index
- return list(self._simple_name_lookup[suffix])
- # (H) Fallback to linear scan if no index available
- return [qn for qn in self._entries.keys() if qn.endswith(f".{suffix}")]
-
- def find_with_prefix(self, prefix: str) -> list[tuple[QualifiedName, NodeType]]:
- node = self._navigate_to_prefix(prefix)
- return [] if node is None else self._collect_from_subtree(node)
-
-
-class BoundedASTCache:
- def __init__(
- self,
- max_entries: int | None = None,
- max_memory_mb: int | None = None,
- ):
- self.cache: OrderedDict[Path, tuple[Node, cs.SupportedLanguage]] = OrderedDict()
- self.max_entries = (
- max_entries if max_entries is not None else settings.CACHE_MAX_ENTRIES
- )
- max_mem = (
- max_memory_mb if max_memory_mb is not None else settings.CACHE_MAX_MEMORY_MB
- )
- self.max_memory_bytes = max_mem * cs.BYTES_PER_MB
-
- def __setitem__(self, key: Path, value: tuple[Node, cs.SupportedLanguage]) -> None:
- if key in self.cache:
- del self.cache[key]
-
- self.cache[key] = value
-
- self._enforce_limits()
+def _owning_module_qn(qn: str, module_qns: set[str]) -> str | None:
+ owner: str | None = None
+ for candidate in module_qns:
+ if qn.startswith(candidate + cs.SEPARATOR_DOT) and (
+ owner is None or len(candidate) > len(owner)
+ ):
+ owner = candidate
+ return owner
+
+
+def _route_handler_entry(
+ row: Mapping[str, object], already_pending: set[str], module_qns: set[str]
+) -> tuple[cs.NodeLabel, str, list[str], str | None] | None:
+ qn = row.get(cs.KEY_QUALIFIED_NAME)
+ decorators = row.get(cs.KEY_DECORATORS)
+ if not isinstance(qn, str) or qn in already_pending:
+ return None
+ if not isinstance(decorators, list):
+ return None
+ routes = [d for d in decorators if isinstance(d, str) and parse_route_decorator(d)]
+ if not routes:
+ return None
+ labels = row.get("labels")
+ label = (
+ cs.NodeLabel.METHOD
+ if isinstance(labels, list) and cs.NodeLabel.METHOD.value in labels
+ else cs.NodeLabel.FUNCTION
+ )
+ return (label, qn, routes, _owning_module_qn(qn, module_qns))
+
+
+type FileHashCache = dict[str, str]
+type DirMtimesCache = dict[str, float]
+
+
+_CPP_SPAN_FILE_EXTENSIONS = frozenset(cs.CPP_EXTENSIONS) | frozenset(cs.C_EXTENSIONS)
+
+
+def _hash_file(filepath: Path) -> str:
+ data = filepath.read_bytes()
+ return hashlib.md5(data, usedforsecurity=False).hexdigest()
+
+
+def _hash_file_with_bytes(filepath: Path) -> tuple[str, bytes] | None:
+ try:
+ with open(filepath, "rb") as f:
+ data = f.read()
+ except OSError as e:
+ logger.warning(ls.FILE_UNREADABLE, path=filepath, error=e)
+ return None
+ return hashlib.md5(data, usedforsecurity=False).hexdigest(), data
+
+
+def _load_hash_cache(cache_path: Path) -> FileHashCache:
+ if not cache_path.is_file():
+ return {}
+ try:
+ with cache_path.open(encoding="utf-8") as f:
+ data = json.load(f)
+ if isinstance(data, dict):
+ logger.info(ls.HASH_CACHE_LOADED, count=len(data), path=cache_path)
+ return data
+ except (json.JSONDecodeError, OSError) as e:
+ logger.warning(ls.HASH_CACHE_LOAD_FAILED, path=cache_path, error=e)
+ return {}
+
+
+def _save_hash_cache(cache_path: Path, hashes: FileHashCache) -> None:
+ try:
+ cache_path.parent.mkdir(parents=True, exist_ok=True)
+ with cache_path.open("w", encoding="utf-8") as f:
+ json.dump(hashes, f, indent=2)
+ logger.info(ls.HASH_CACHE_SAVED, count=len(hashes), path=cache_path)
+ except OSError as e:
+ logger.warning(ls.HASH_CACHE_SAVE_FAILED, path=cache_path, error=e)
+
+
+def _load_parser_fingerprint(stamp_path: Path) -> str | None:
+ try:
+ return stamp_path.read_text(encoding="utf-8").strip() or None
+ except OSError:
+ return None
+
+
+def _save_parser_fingerprint(stamp_path: Path, fingerprint: str) -> None:
+ try:
+ stamp_path.write_text(fingerprint, encoding="utf-8")
+ except OSError as e:
+ logger.warning(ls.PARSER_FINGERPRINT_SAVE_FAILED, path=stamp_path, error=e)
+
+
+def _load_dir_mtimes(cache_path: Path) -> DirMtimesCache:
+ if not cache_path.is_file():
+ return {}
+ try:
+ with cache_path.open(encoding="utf-8") as f:
+ data = json.load(f)
+ if isinstance(data, dict):
+ return {k: float(v) for k, v in data.items() if isinstance(v, int | float)}
+ except (json.JSONDecodeError, OSError, ValueError):
+ pass
+ return {}
+
+
+def _save_dir_mtimes(cache_path: Path, mtimes: DirMtimesCache) -> None:
+ try:
+ cache_path.parent.mkdir(parents=True, exist_ok=True)
+ with cache_path.open("w", encoding="utf-8") as f:
+ json.dump(mtimes, f)
+ except OSError:
+ pass
+
+
+def _touch_empty_json(cache_path: Path) -> None:
+ if cache_path.exists():
+ return
+ try:
+ cache_path.parent.mkdir(parents=True, exist_ok=True)
+ with cache_path.open("w", encoding="utf-8") as f:
+ f.write(cs.JSON_EMPTY_OBJECT)
+ except OSError:
+ pass
- def __getitem__(self, key: Path) -> tuple[Node, cs.SupportedLanguage]:
- value = self.cache[key]
- self.cache.move_to_end(key)
- return value
- def __delitem__(self, key: Path) -> None:
- if key in self.cache:
- del self.cache[key]
-
- def __contains__(self, key: Path) -> bool:
- return key in self.cache
-
- def items(self) -> ItemsView[Path, tuple[Node, cs.SupportedLanguage]]:
- return self.cache.items()
-
- def _enforce_limits(self) -> None:
- while len(self.cache) > self.max_entries:
- self.cache.popitem(last=False) # (H) Remove least recently used
-
- if self._should_evict_for_memory():
- entries_to_remove = max(
- 1, len(self.cache) // settings.CACHE_EVICTION_DIVISOR
- )
- for _ in range(entries_to_remove):
- if self.cache:
- self.cache.popitem(last=False)
-
- def _should_evict_for_memory(self) -> bool:
- try:
- cache_size = sum(sys.getsizeof(v) for v in self.cache.values())
- return cache_size > self.max_memory_bytes
- except Exception:
- return (
- len(self.cache)
- > self.max_entries * settings.CACHE_MEMORY_THRESHOLD_RATIO
- )
+class GraphUpdater:
+ """Drive a full or incremental ingest of a repository into the graph.
+ Parses each supported source file into definitions, imports, and calls,
+ resolves them across files, and streams the resulting nodes and edges to
+ the configured ingestor.
+ """
-class GraphUpdater:
def __init__(
self,
ingestor: IngestorProtocol,
repo_path: Path,
- parsers: dict[cs.SupportedLanguage, Parser],
- queries: dict[cs.SupportedLanguage, LanguageQueries],
+ parsers: Mapping[cs.SupportedLanguage, Parser],
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
unignore_paths: frozenset[str] | None = None,
exclude_paths: frozenset[str] | None = None,
+ project_name: str | None = None,
+ capture: CaptureSelection | None = None,
+ skip_embeddings: bool | None = None,
):
+ self.capture = capture if capture is not None else default_capture()
+ # `ingestor` stays the raw object for DB queries (QueryProtocol),
+ # flushes, and test introspection. `_sink` is a filtering wrapper that
+ # drops disabled relationships/nodes at one choke point, so the ~20
+ # parser emission sites stay untouched. All emission goes through
+ # `_sink`; everything else uses `ingestor`.
self.ingestor = ingestor
+ self._sink: IngestorProtocol = FilteringIngestor(ingestor, self.capture)
+ self._single_file: Path | None = None
+ # True while the current sync re-parses EVERY file (no cache/force):
+ # such a run re-resolves all edges itself, so graph reads may degrade
+ # on failure; an incremental run's correctness depends on them.
+ self._is_full_build = False
+ if repo_path.is_file():
+ resolved = repo_path.resolve()
+ self._single_file = resolved
+ repo_path = resolved.parent
self.repo_path = repo_path
self.parsers = parsers
self.queries = queries
- self.project_name = repo_path.resolve().name
+ self.project_name = (
+ project_name and project_name.strip()
+ ) or repo_path.resolve().name
self.simple_name_lookup: SimpleNameLookup = defaultdict(set)
self.function_registry = FunctionRegistryTrie(
simple_name_lookup=self.simple_name_lookup
)
- self.ast_cache = BoundedASTCache()
+ self.ast_cache = BoundedASTCache(loader=self._load_ast_from_disk)
+ # Every file parsed this run, in parse order. The AST cache is bounded
+ # and evicts on large repos, so Pass 3 must iterate this full list (not
+ # the cache) and re-parse evicted files, or their calls are dropped.
+ self._parsed_files: list[tuple[Path, cs.SupportedLanguage]] = []
self.unignore_paths = unignore_paths
self.exclude_paths = exclude_paths
+ # None defers to the CGR_SKIP_EMBEDDINGS setting so env-configured
+ # callers (MCP, workspace sync) opt out without a CLI flag.
+ self.skip_embeddings = (
+ settings.SKIP_EMBEDDINGS if skip_embeddings is None else skip_embeddings
+ )
+ self.skipped_because_in_sync = False
+ self._collected_dir_mtimes: DirMtimesCache = {}
+ self._cpp_frontend_covered: frozenset[str] = frozenset()
+ # Hybrid-mode macro uses awaiting a caller: attribution needs the
+ # tree-sitter definition spans, which exist only after Pass 2.
+ self._pending_cpp_macro_calls: list[PendingMacroCall] = []
+ # Hybrid-mode expansion-produced calls (call text lives inside a
+ # macro body): BOTH ends join to tree-sitter spans after Pass 2.
+ self._pending_cpp_expansion_calls: list[PendingExpansionCall] = []
+ # Definition spans read back from the graph on incremental runs:
+ # an expansion call's CALLEE may live in an UNCHANGED file whose
+ # spans Pass 2 never recorded this run.
+ self._rehydrated_cpp_spans: dict[str, list[CppDefinitionSpan]] = {}
+ # C# Roslyn hybrid facts awaiting their join point: partial
+ # declaration groups join to Class qns after Pass 2, and LINQ
+ # query-operator calls join to function locations after Pass 3.
+ self._csharp_partial_decls: list[list[tuple[str, int]]] = []
+ self._csharp_query_calls: list[CSharpQueryCall] = []
+ # Files (re)parsed by Pass 2 this run: the only files whose
+ # definition spans exist for hybrid macro-call attribution.
+ self._reparsed_file_keys: set[str] = set()
+ # Module qns read back from the graph on incremental runs; deferred
+ # import verification counts them as real internal targets.
+ self._rehydrated_module_qns: set[str] = set()
self.factory = ProcessorFactory(
- ingestor=self.ingestor,
+ ingestor=self._sink,
repo_path=self.repo_path,
project_name=self.project_name,
queries=self.queries,
@@ -253,45 +305,947 @@ def __init__(
ast_cache=self.ast_cache,
unignore_paths=self.unignore_paths,
exclude_paths=self.exclude_paths,
+ capture=self.capture,
+ )
+ # Fallback structural tier for languages with no tree-sitter
+ # LanguageSpec (e.g. Ruby), driven by ast-grep pattern configs.
+ self.ast_grep_tier = AstGrepTier(self._sink, self.repo_path, self.project_name)
+ # Opt-in ast-grep finding analyzer (issue #413): Pattern/CodeSmell/
+ # SecurityIssue nodes from categorized YAML rules, run as a post-pass.
+ self.finding_analyzer = FindingAnalyzer(
+ self._sink, self.repo_path, self.capture
+ )
+
+ def _run_cpp_frontend(self) -> None:
+ # Optional libclang C++ pre-pass when a compile_commands.json is
+ # discoverable. LIBCLANG: emit macro-accurate C/C++ nodes/edges
+ # directly (tree-sitter cannot expand macros) and skip covered files
+ # in the definition pass. HYBRID: tree-sitter stays the backbone
+ # (nothing skipped); libclang layers on only macro Function nodes and
+ # #include IMPORTS, whose qns are scheme-identical, and hands back
+ # macro uses for span attribution after Pass 2. Missing either
+ # condition falls back to tree-sitter.
+ self._cpp_frontend_covered = frozenset()
+ self._pending_cpp_macro_calls = []
+ if settings.CPP_FRONTEND not in (
+ cs.CppFrontend.LIBCLANG,
+ cs.CppFrontend.HYBRID,
+ ):
+ return
+ if not self._repo_has_c_or_cpp_files():
+ # HYBRID is the default, so a repo with no C/C++ sources must
+ # skip silently instead of warning about libclang or a missing
+ # compile_commands.json on every index of a Python/Go project.
+ return
+ if not cpp_frontend_available():
+ logger.warning(ls.CPP_FRONTEND_UNAVAILABLE)
+ return
+ compdb_dir = find_compile_commands(self.repo_path)
+ if compdb_dir is None:
+ logger.warning(ls.CPP_FRONTEND_NO_COMPDB)
+ return
+ logger.info(ls.CPP_FRONTEND_RUNNING.format(path=compdb_dir))
+ if settings.CPP_FRONTEND == cs.CppFrontend.HYBRID:
+ (
+ self._pending_cpp_macro_calls,
+ self._pending_cpp_expansion_calls,
+ ) = run_cpp_frontend_hybrid(
+ self._sink,
+ self.repo_path,
+ self.project_name,
+ compdb_dir,
+ function_registry=self.function_registry,
+ simple_name_lookup=self.simple_name_lookup,
+ structural_elements=(
+ self.factory.structure_processor.structural_elements
+ ),
+ )
+ logger.info(
+ ls.CPP_FRONTEND_HYBRID_PENDING.format(
+ count=len(self._pending_cpp_macro_calls)
+ + len(self._pending_cpp_expansion_calls)
+ )
+ )
+ return
+ self._cpp_frontend_covered = run_cpp_frontend(
+ self._sink,
+ self.repo_path,
+ self.project_name,
+ compdb_dir,
+ function_registry=self.function_registry,
+ simple_name_lookup=self.simple_name_lookup,
+ structural_elements=self.factory.structure_processor.structural_elements,
+ )
+ logger.info(
+ ls.CPP_FRONTEND_COVERED.format(count=len(self._cpp_frontend_covered))
+ )
+
+ def _run_csharp_frontend(self) -> None:
+ # Optional Roslyn semantic pre-pass. ROSLYN/HYBRID: load the repo's
+ # real .csproj/.sln via MSBuildWorkspace and collect facts syntax
+ # alone cannot derive: exact INHERITS-vs-IMPLEMENTS base kinds (Pass
+ # 2), exact per-invocation call targets (Pass 3), partial-type
+ # identity groups (joined after Pass 2), and LINQ query-operator
+ # calls (after Pass 3). Missing dotnet, project, or a build/restore
+ # failure all fall back to pure tree-sitter (empty facts). Reset first
+ # so a reused updater (watch mode) that previously ran hybrid does not
+ # keep applying stale facts on a later run with the frontend off.
+ # csharp_call_sites is mutated in place because the type-inference
+ # engine holds a reference.
+ dp = self.factory.definition_processor
+ dp.csharp_base_kinds = {}
+ dp.csharp_call_sites.clear()
+ dp.csharp_external_sites.clear()
+ self._csharp_partial_decls = []
+ self._csharp_query_calls = []
+ if settings.CSHARP_FRONTEND == cs.CSharpFrontend.TREESITTER:
+ return
+ project = find_csharp_project(self.repo_path)
+ if project is None:
+ # Skip silently when there is no C# project: nothing to augment,
+ # and building the net tool for a non-C# repo would be wasteful.
+ return
+ if not csharp_frontend_available():
+ # AUTO promises hybrid only where the toolchain exists, so a
+ # missing dotnet is the expected fallback (info); an EXPLICIT
+ # hybrid/roslyn request that cannot run stays a warning.
+ if settings.CSHARP_FRONTEND == cs.CSharpFrontend.AUTO:
+ logger.info(ls.CSHARP_FRONTEND_AUTO_FALLBACK)
+ else:
+ logger.warning(ls.CSHARP_FRONTEND_UNAVAILABLE)
+ return
+ logger.info(ls.CSHARP_FRONTEND_RUNNING.format(path=project))
+ facts = run_csharp_frontend(self.repo_path)
+ dp.csharp_base_kinds = facts.base_kinds
+ dp.csharp_call_sites.update(facts.call_sites)
+ dp.csharp_external_sites.update(facts.external_sites)
+ self._csharp_partial_decls = facts.partial_groups
+ self._csharp_query_calls = facts.query_calls
+ logger.info(ls.CSHARP_FRONTEND_TYPES.format(count=len(facts.base_kinds)))
+ logger.info(
+ ls.CSHARP_FRONTEND_FACTS.format(
+ calls=len(facts.call_sites),
+ partials=len(facts.partial_groups),
+ queries=len(facts.query_calls),
+ externals=len(facts.external_sites),
+ )
)
+ def _join_csharp_partials(self) -> None:
+ # Replace the directory-keyed syntactic partial grouping with the
+ # Roslyn symbol-identity groups wherever Roslyn saw the type: parts
+ # in DIFFERENT directories of one project merge (the syntactic rule
+ # deliberately under-merges there), and unrelated same-name types a
+ # syntactic merge would conflate split apart (each arrives as its
+ # own group). Group lists stay SHARED objects because the resolver
+ # compares them by identity.
+ if not self._csharp_partial_decls:
+ return
+ dp = self.factory.definition_processor
+ groups: list[list[str]] = []
+ covered: set[str] = set()
+ for decls in self._csharp_partial_decls:
+ qns = sorted({qn for d in decls if (qn := dp.csharp_type_locations.get(d))})
+ covered.update(qns)
+ if len(qns) > 1:
+ groups.append(qns)
+ for qn in covered:
+ old = dp.csharp_partial_groups.pop(qn, None)
+ if old is not None and qn in old:
+ # Also shrink the shared syntactic list so members NOT
+ # covered by Roslyn stop spanning to this part.
+ old.remove(qn)
+ for group in groups:
+ for qn in group:
+ dp.csharp_partial_groups[qn] = group
+ if groups:
+ logger.info(ls.CSHARP_FRONTEND_PARTIALS_JOINED.format(count=len(groups)))
+
+ def _emit_csharp_query_calls(self) -> None:
+ # LINQ query syntax has no invocation nodes for tree-sitter to see;
+ # each Roslyn query-operator fact becomes a direct CALLS edge, both
+ # ends resolved through the Pass-2 function-location registry (a miss
+ # on either end drops the fact rather than risk a dangling edge).
+ if not self._csharp_query_calls:
+ return
+ dp = self.factory.definition_processor
+ rel_to_module = {
+ cached_relative_path(path, self.repo_path).as_posix(): qn
+ for qn, path in dp.module_qn_to_file_path.items()
+ }
+
+ def located(rel_file: str, line: int, col: int) -> FunctionLocation | None:
+ module_qn = rel_to_module.get(rel_file)
+ if module_qn is None:
+ return None
+ return dp.function_locations.get((module_qn, line, col))
+
+ emitted = 0
+ for fact in self._csharp_query_calls:
+ caller = located(fact.caller_file, fact.caller_line, fact.caller_col)
+ target = located(fact.target_file, fact.target_line, fact.target_col)
+ if caller is None or target is None:
+ continue
+ self.ingestor.ensure_relationship_batch(
+ (caller.label, cs.KEY_QUALIFIED_NAME, caller.qualified_name),
+ cs.RelationshipType.CALLS,
+ (target.label, cs.KEY_QUALIFIED_NAME, target.qualified_name),
+ )
+ emitted += 1
+ if emitted:
+ logger.info(ls.CSHARP_FRONTEND_QUERY_EDGES.format(count=emitted))
+
+ def _tightest_containing_span(
+ self, rel_path: str, line: int
+ ) -> CppDefinitionSpan | None:
+ spans = self.factory.definition_processor.cpp_definition_spans
+ candidates = spans.get(rel_path)
+ if candidates is None and rel_path not in self._reparsed_file_keys:
+ # An unchanged file on an incremental run has no fresh spans;
+ # its definitions (and their lines) are unchanged too, so the
+ # graph-rehydrated spans are exact.
+ candidates = self._rehydrated_cpp_spans.get(rel_path)
+ containing = [s for s in candidates or () if s.start_line <= line <= s.end_line]
+ if not containing:
+ return None
+ return min(containing, key=lambda s: s.end_line - s.start_line)
+
+ def _resolve_hybrid_macro_calls(self) -> None:
+ # Attribute each hybrid macro use to the tightest enclosing
+ # TREE-SITTER definition span (recorded during Pass 2), falling back
+ # to the use site's Module: the mirror of the libclang frontend's own
+ # span resolution, but against the qn scheme the rest of the graph
+ # actually uses.
+ if not self._pending_cpp_macro_calls:
+ return
+ spans = self.factory.definition_processor.cpp_definition_spans
+ emitted = 0
+ for call in self._pending_cpp_macro_calls:
+ # The frontend parses every TU each run, but an incremental run
+ # records spans only for re-parsed files. An unchanged file has no
+ # spans here AND already carries its caller->macro edges, so
+ # resolving it would re-attribute in-function uses to the Module.
+ if call.rel_path not in self._reparsed_file_keys:
+ continue
+ containing = [
+ s
+ for s in spans.get(call.rel_path, ())
+ if s.start_line <= call.line <= s.end_line
+ and s.qualified_name != call.callee_qn
+ ]
+ if containing:
+ tightest = min(containing, key=lambda s: s.end_line - s.start_line)
+ caller_label, caller_qn = tightest.label, tightest.qualified_name
+ else:
+ caller_label = cs.NodeLabel.MODULE.value
+ caller_qn = call.fallback_module_qn
+ self._sink.ensure_relationship_batch(
+ (caller_label, cs.KEY_QUALIFIED_NAME, caller_qn),
+ cs.RelationshipType.CALLS,
+ (cs.NodeLabel.FUNCTION, cs.KEY_QUALIFIED_NAME, call.callee_qn),
+ )
+ emitted += 1
+ logger.info(ls.CPP_FRONTEND_MACRO_CALLS.format(count=emitted))
+
+ def _resolve_hybrid_expansion_calls(self) -> None:
+ # A call whose text lives inside a macro body exists only after
+ # expansion, so tree-sitter never emits it. Join BOTH ends to
+ # tree-sitter definition spans: the caller by expansion site (Module
+ # fallback, like macro uses), the callee by its referenced
+ # definition's location (dropped when no span contains it, since an
+ # unindexed or template-only definition has no tree-sitter node to
+ # target).
+ if not self._pending_cpp_expansion_calls:
+ return
+ emitted = 0
+ for call in self._pending_cpp_expansion_calls:
+ if call.caller_rel_path not in self._reparsed_file_keys:
+ continue
+ callee = self._tightest_containing_span(
+ call.callee_rel_path, call.callee_line
+ )
+ if callee is None:
+ continue
+ caller = self._tightest_containing_span(
+ call.caller_rel_path, call.caller_line
+ )
+ if caller is not None:
+ caller_label: str = caller.label
+ caller_qn = caller.qualified_name
+ else:
+ caller_label = cs.NodeLabel.MODULE.value
+ caller_qn = call.fallback_module_qn
+ self._sink.ensure_relationship_batch(
+ (caller_label, cs.KEY_QUALIFIED_NAME, caller_qn),
+ cs.RelationshipType.CALLS,
+ (callee.label, cs.KEY_QUALIFIED_NAME, callee.qualified_name),
+ )
+ emitted += 1
+ logger.info(ls.CPP_FRONTEND_EXPANSION_CALLS.format(count=emitted))
+
+ def _repo_has_c_or_cpp_files(self) -> bool:
+ # Cheap early-exit scan: the frontend (and its warnings) only make
+ # sense when there is C/C++ to index.
+ extensions = set(cs.CPP_EXTENSIONS) | set(cs.C_EXTENSIONS)
+ for _root, dirs, files in os.walk(self.repo_path):
+ dirs[:] = [d for d in dirs if not d.startswith(cs.SEPARATOR_DOT)]
+ # splitext, not Path(): no object allocation per repo file.
+ if any(os.path.splitext(f)[1].lower() in extensions for f in files):
+ return True
+ return False
+
def _is_dependency_file(self, file_name: str, filepath: Path) -> bool:
return (
file_name.lower() in cs.DEPENDENCY_FILES
or filepath.suffix.lower() == cs.CSPROJ_SUFFIX
)
- def run(self) -> None:
- self.ingestor.ensure_node_batch(
- cs.NODE_PROJECT, {cs.KEY_NAME: self.project_name}
+ def run(self, force: bool = False) -> None:
+ """Ingest the repository; ``force`` rebuilds instead of updating incrementally."""
+ py_engine = self.factory.type_inference._python_type_inference
+ if py_engine is not None:
+ py_engine._available_classes_cache.clear()
+ py_engine._return_stmt_cache.clear()
+ py_engine._method_return_type_cache.clear()
+ py_engine._self_assignment_cache.clear()
+ # Reset per-run parse tracking so a reused updater does not reprocess
+ # a previous run's files in Pass 3.
+ self._parsed_files.clear()
+ self._sink.ensure_node_batch(
+ cs.NODE_PROJECT,
+ {
+ cs.KEY_NAME: self.project_name,
+ cs.KEY_ROOT_PATH: str(self.repo_path.resolve()),
+ },
)
- logger.info(ls.ENSURING_PROJECT.format(name=self.project_name))
+ logger.info(ls.ENSURING_PROJECT, name=self.project_name)
+
+ if not force and self._single_file is None:
+ self._drop_cache_if_graph_lost()
+ self._warn_if_parser_changed()
+
+ if not force and self._is_already_in_sync():
+ logger.info(ls.GRAPH_ALREADY_IN_SYNC)
+ self.skipped_because_in_sync = True
+ self.ingestor.flush_all()
+ return
logger.info(ls.PASS_1_STRUCTURE)
self.factory.structure_processor.identify_structure()
+ # LIBCLANG must run before Pass 2: _process_files consumes the
+ # covered-file set to skip those files.
+ if settings.CPP_FRONTEND != cs.CppFrontend.HYBRID:
+ self._run_cpp_frontend()
+
+ # The C# Roslyn frontend must run before Pass 2: it produces a
+ # base-classification oracle that split_csharp_bases consults while
+ # ingesting each type's INHERITS/IMPLEMENTS edges during Pass 2.
+ self._run_csharp_frontend()
+
logger.info(ls.PASS_2_FILES)
- self._process_files()
+ self._process_files(force=force)
+
+ # Partial groups join AFTER Pass 2: the Roslyn declaration
+ # locations resolve against the Class qns Pass 2 just registered.
+ self._join_csharp_partials()
+
+ # HYBRID must run after Pass 2: an incremental run deletes each
+ # changed file's Module subtree before re-parsing it, so macro
+ # nodes and include IMPORTS emitted earlier would be deleted with
+ # it and vanish until a forced rebuild.
+ if settings.CPP_FRONTEND == cs.CppFrontend.HYBRID:
+ self._run_cpp_frontend()
+
+ corrected = self.factory.definition_processor.resolve_deferred_cpp_methods()
+ if corrected:
+ logger.info("Resolved {} deferred C++ out-of-class methods", corrected)
+
+ contained = self.factory.definition_processor.resolve_deferred_cpp_containment()
+ if contained:
+ logger.info("Resolved {} deferred C++ nested containments", contained)
+
+ # After resolve_deferred_cpp_methods: an out-of-class method's span
+ # is recorded only once its class binding resolves, and a macro use
+ # inside such a method must attribute to it, not the Module.
+ self._resolve_hybrid_macro_calls()
+
+ go_methods = self.factory.definition_processor.resolve_deferred_go_methods()
+ if go_methods:
+ logger.info("Resolved {} Go receiver methods", go_methods)
+
+ if not force:
+ self._rehydrate_registry_from_graph()
+
+ # After rehydration: an expansion call's callee join needs spans
+ # for unchanged files too.
+ self._resolve_hybrid_expansion_calls()
+
+ # After rehydration so the "does a real definition exist?" check sees
+ # definitions in files an incremental run did not re-parse; otherwise a
+ # forward declaration whose definition lives in an unchanged file is
+ # kept as a phantom and re-fragments the class.
+ kept_forwards = (
+ self.factory.definition_processor.resolve_deferred_forward_declarations()
+ )
+ if kept_forwards:
+ logger.info(
+ "Registered {} forward-declared C/C++ types with no definition",
+ kept_forwards,
+ )
- logger.info(ls.FOUND_FUNCTIONS.format(count=len(self.function_registry)))
+ # After rehydration (an incremental run's class may live in an
+ # unchanged header) and after forward declarations (a kept
+ # forward-declared TYPE also proves the name is a class, not a
+ # macro).
+ orphan_ctors = (
+ self.factory.definition_processor.resolve_deferred_cpp_artifacts()
+ )
+ if orphan_ctors:
+ logger.info("Registered {} recovery-orphaned C++ ctors", orphan_ctors)
+
+ # After artifact resolution so a recovery-registered definition also
+ # counts; a prototype duplicating any bodied definition is dropped.
+ kept_prototypes = (
+ self.factory.definition_processor.resolve_deferred_cpp_prototypes()
+ )
+ if kept_prototypes:
+ logger.info(
+ "Registered {} C/C++ function prototypes with no definition",
+ kept_prototypes,
+ )
+
+ # After forward declarations so a base whose only representation is
+ # a kept forward declaration still resolves to a real node.
+ inherits = self.factory.definition_processor.resolve_deferred_cpp_inherits()
+ if inherits:
+ logger.info("Resolved {} deferred C++ inheritance bases", inherits)
+
+ # Same reasoning for every other language: parents resolve against
+ # the full registry (including rehydrated definitions), and an
+ # unresolvable parent emits no edge instead of a phantom.
+ generic_inherits = self.factory.definition_processor.resolve_deferred_inherits()
+ if generic_inherits:
+ logger.info(
+ "Resolved {} deferred inheritance/implements parents",
+ generic_inherits,
+ )
+
+ module_impls = (
+ self.factory.definition_processor.resolve_deferred_cpp_module_impls()
+ )
+ if module_impls:
+ logger.info("Resolved {} C++20 module implementation links", module_impls)
+
+ # IMPORTS edges verify against every module qn this run produced
+ # (files, inline modules, rehydrated unchanged files); an internal
+ # target that resolves nowhere emits no edge.
+ known_module_paths: dict[str, str] = {
+ str(qn): path.as_posix()
+ for qn, path in (
+ self.factory.definition_processor.module_qn_to_file_path.items()
+ )
+ }
+ for qn in self.factory.definition_processor.declared_module_qns:
+ known_module_paths.setdefault(qn, "")
+ for qn in self._rehydrated_module_qns:
+ known_module_paths.setdefault(qn, "")
+ imports_emitted = self.factory.import_processor.flush_deferred_import_edges(
+ known_module_paths
+ )
+ if imports_emitted:
+ logger.info("Emitted {} verified IMPORTS edges", imports_emitted)
+
+ # Last containment step: every node-registering pass above (deferred
+ # C++ methods, Go receivers, kept forward declarations) must finish
+ # before parent qns are verified against the registry.
+ linked = self.factory.definition_processor.resolve_deferred_parent_links()
+ if linked:
+ logger.info("Resolved {} deferred containment parents", linked)
+
+ logger.info(ls.FOUND_FUNCTIONS, count=len(self.function_registry))
logger.info(ls.PASS_3_CALLS)
self._process_function_calls()
+ # LINQ query-operator edges join AFTER Pass 3 with the complete
+ # function-location registry (both ends must be registered nodes).
+ self._emit_csharp_query_calls()
+
self.factory.definition_processor.process_all_method_overrides()
+ # Deferred endpoint emission: every module is parsed now, so router
+ # mount prefixes (possibly cross-module) can resolve (issue #877).
+ self._emit_pending_endpoints()
+
+ # Call-registered routes (Express, net/http, echo, gin) become
+ # endpoints too, so JS and Go servers are linkable (issue #886).
+ self._emit_route_call_endpoints()
+
+ # ast-grep findings post-pass (opt-in FINDINGS group). Links to the
+ # Modules the definition pass already emitted, so no dangling edges.
+ self.finding_analyzer.analyze(
+ self.factory.definition_processor.module_qn_to_file_path
+ )
+
logger.info(ls.ANALYSIS_COMPLETE)
self.ingestor.flush_all()
+ self._link_endpoint_resources()
+
+ self._prune_orphan_nodes()
+
self._generate_semantic_embeddings()
+ def _emit_pending_endpoints(self) -> None:
+ if not self.capture.rel_enabled(cs.RelationshipType.EXPOSES):
+ return
+ dp = self.factory.definition_processor
+ module_asts = self._python_module_asts()
+ entries = list(dp.pending_endpoints)
+ # A mount-only incremental change re-parses just the mounting module;
+ # the unchanged handlers must still re-emit under the new prefix, so
+ # they come back from the graph. A full build queued them all already.
+ if not self._is_full_build:
+ entries.extend(
+ self._rehydrated_route_handlers(
+ {qn for _label, qn, _decorators, _module in entries},
+ set(module_asts),
+ )
+ )
+ if not entries:
+ return
+ self._drop_stale_handler_exposes(
+ [qn for _label, qn, _decorators, _module in entries]
+ )
+ registry = build_router_registry(module_asts)
+ for label, qn, decorators, module_qn in entries:
+ emit_endpoints(
+ self._sink,
+ label,
+ qn,
+ decorators,
+ module_qn=module_qn,
+ prefix_resolver=registry.mount_prefixes,
+ )
+ dp.pending_endpoints.clear()
+
+ def _drop_stale_handler_exposes(self, handler_qns: list[str]) -> None:
+ # Re-emitted handlers drop their previous EXPOSES first, so a
+ # template that changed prefix loses its stale anchor and the orphan
+ # cleanup prunes the outdated endpoint node.
+ if not isinstance(self.ingestor, QueryProtocol):
+ return
+ try:
+ self.ingestor.execute_write(
+ CYPHER_DELETE_HANDLER_EXPOSES, {"qns": handler_qns}
+ )
+ except Exception:
+ logger.debug("Stale EXPOSES cleanup unavailable; emission continues")
+
+ def _emit_route_call_endpoints(self) -> None:
+ if not self.capture.rel_enabled(cs.RelationshipType.EXPOSES):
+ return
+ modules = self._route_module_asts()
+ if not modules:
+ return
+ # Cleanup keyed on every scanned module, BEFORE emission:
+ # a module whose last route disappeared (or whose attribution moved)
+ # still sheds its old EXPOSES edges even though it contributes no new
+ # registrations.
+ self._drop_stale_module_exposes(sorted(modules))
+ for module_qn, (root, language) in modules.items():
+ for registration in collect_route_registrations(root, language):
+ label, source_qn = self._route_source(module_qn, registration)
+ identity = f"{registration.method} {registration.path}"
+ _emit_endpoint(self._sink, label, source_qn, identity)
+
+ def _drop_stale_module_exposes(self, module_qns: list[str]) -> None:
+ # Ownership is the DEFINES containment closure from each Module
+ # node, so prefix-sharing sibling modules keep their endpoints.
+ if not isinstance(self.ingestor, QueryProtocol):
+ return
+ try:
+ self.ingestor.execute_write(
+ CYPHER_DELETE_MODULE_EXPOSES, {"module_qns": module_qns}
+ )
+ except Exception:
+ logger.debug("Stale EXPOSES cleanup unavailable; emission continues")
+
+ def _route_source(
+ self, module_qn: str, registration: RouteRegistration
+ ) -> tuple[cs.NodeLabel, str]:
+ # Attribution ladder: identifier handler, else the registering call's
+ # enclosing function, else the module, so the endpoint stays anchored
+ # and the trace lands at the wiring site.
+ registry = self.factory.definition_processor.function_registry
+ candidates = []
+ if registration.handler_name:
+ candidates.append(
+ f"{module_qn}{cs.SEPARATOR_DOT}{registration.handler_name}"
+ )
+ if registration.scope:
+ candidates.append(f"{module_qn}{cs.SEPARATOR_DOT}{registration.scope}")
+ for qn in candidates:
+ node_type = registry.get(qn)
+ if node_type is not None:
+ label = (
+ cs.NodeLabel.METHOD
+ if node_type == NodeType.METHOD
+ else cs.NodeLabel.FUNCTION
+ )
+ return label, qn
+ return cs.NodeLabel.MODULE, module_qn
+
+ def _route_module_asts(
+ self,
+ ) -> dict[str, tuple[Node, cs.SupportedLanguage]]:
+ dp = self.factory.definition_processor
+ files: dict[str, Path] = dict(dp.module_qn_to_file_path)
+ for qn, path in self._graph_route_module_paths():
+ files.setdefault(qn, path)
+ out: dict[str, tuple[Node, cs.SupportedLanguage]] = {}
+ for qn, path in files.items():
+ entry = self.ast_cache.load(path)
+ if entry is not None and entry[1] in ROUTE_CALL_LANGUAGES:
+ out[qn] = entry
+ return out
+
+ def _graph_route_module_paths(self) -> list[tuple[str, Path]]:
+ # Unchanged modules come back from the graph on an incremental run,
+ # already narrowed to route-capable extensions at the query.
+ if not isinstance(self.ingestor, QueryProtocol):
+ return []
+ params = {
+ cs.KEY_PROJECT_PREFIX: self.project_name + cs.SEPARATOR_DOT,
+ "extensions": list(ROUTE_MODULE_EXTENSIONS),
+ }
+ try:
+ rows = list(self.ingestor.fetch_all(CYPHER_PROJECT_MODULES, params))
+ except Exception:
+ return []
+ out: list[tuple[str, Path]] = []
+ for row in rows:
+ if not isinstance(row, dict):
+ continue
+ qn, rel_path = row.get(cs.KEY_QUALIFIED_NAME), row.get(cs.KEY_PATH)
+ if isinstance(qn, str) and isinstance(rel_path, str):
+ out.append((qn, self.repo_path / rel_path))
+ return out
+
+ def _rehydrated_route_handlers(
+ self, already_pending: set[str], module_qns: set[str]
+ ) -> list[tuple[cs.NodeLabel, str, list[str], str | None]]:
+ if not isinstance(self.ingestor, QueryProtocol):
+ return []
+ params = {cs.KEY_PROJECT_PREFIX: self.project_name + cs.SEPARATOR_DOT}
+ try:
+ rows = list(self.ingestor.fetch_all(CYPHER_PROJECT_ROUTE_HANDLERS, params))
+ except Exception:
+ return []
+ out: list[tuple[cs.NodeLabel, str, list[str], str | None]] = []
+ for row in rows:
+ if not isinstance(row, dict):
+ continue
+ entry = _route_handler_entry(row, already_pending, module_qns)
+ if entry is not None:
+ out.append(entry)
+ return out
+
+ def _python_module_asts(self) -> dict[str, Node]:
+ # Re-parsed files first; on an incremental run the unchanged modules
+ # holding the mounts come back from the graph's Module nodes, loaded
+ # through the disk-backed AST cache.
+ dp = self.factory.definition_processor
+ files: dict[str, Path] = {
+ qn: path
+ for qn, path in dp.module_qn_to_file_path.items()
+ if path.suffix == ".py"
+ }
+ for qn, path in self._graph_python_modules():
+ files.setdefault(qn, path)
+ asts: dict[str, Node] = {}
+ for qn, path in files.items():
+ entry = self.ast_cache.load(path)
+ if entry is not None and entry[1] == cs.SupportedLanguage.PYTHON:
+ asts[qn] = entry[0]
+ return asts
+
+ def _graph_python_modules(self) -> list[tuple[str, Path]]:
+ if not isinstance(self.ingestor, QueryProtocol):
+ return []
+ params = {cs.KEY_PROJECT_PREFIX: self.project_name + cs.SEPARATOR_DOT}
+ try:
+ rows = list(self.ingestor.fetch_all(CYPHER_PROJECT_PY_MODULES, params))
+ except Exception:
+ return []
+ out: list[tuple[str, Path]] = []
+ for row in rows:
+ if not isinstance(row, dict):
+ continue
+ qn, rel_path = row.get(cs.KEY_QUALIFIED_NAME), row.get(cs.KEY_PATH)
+ if isinstance(qn, str) and isinstance(rel_path, str):
+ out.append((qn, self.repo_path / rel_path))
+ return out
+
+ def _link_endpoint_resources(self) -> None:
+ # After flush_all so this run's Resource nodes are queryable; NETWORK
+ # resources of previously indexed projects join here too, which is
+ # what makes client-URL-to-endpoint edges cross-project (issue #425).
+ # The raw ingestor bypasses the capture filter, so gate explicitly.
+ if not self.capture.rel_enabled(cs.RelationshipType.RESOLVES_TO):
+ return
+ if not isinstance(self.ingestor, QueryProtocol):
+ return
+ created = link_endpoints(self.ingestor)
+ if created:
+ logger.info("Resolved {} client request URLs to endpoints", created)
+ self.ingestor.flush_all()
+
+ def _rehydrate_registry_from_graph(self) -> None:
+ # Incremental runs populate the function registry only from re-parsed
+ # files. Read every definition's qualified name back from the graph and
+ # re-register the ones missing locally, so calls and instantiations
+ # into files that were not re-parsed still resolve and their edges are
+ # re-emitted. Without this, editing one file drops cross-file CALLS /
+ # INSTANTIATES into any unchanged file (issue #532, outbound half).
+ if not isinstance(self.ingestor, QueryProtocol):
+ return
+ added = 0
+ project_params = {cs.KEY_PROJECT_PREFIX: self.project_name + "."}
+ try:
+ rows = self.ingestor.fetch_all(cs.CYPHER_ALL_DEFINITION_QNS, project_params)
+ except Exception:
+ # Rehydration completes cross-file resolution for files this run
+ # did not re-parse: a FULL build parsed them all, so it degrades
+ # to the freshly parsed registry; an incremental run would drop
+ # cross-file edges, so the outage aborts it.
+ if not self._is_full_build:
+ raise
+ logger.warning(ls.REHYDRATE_QUERY_FAILED)
+ return
+ for row in rows:
+ qn = row.get(cs.KEY_QUALIFIED_NAME)
+ label = row.get(cs.KEY_LABEL)
+ if not isinstance(qn, str) or not isinstance(label, str):
+ continue
+ if qn in self.function_registry:
+ continue
+ try:
+ node_type = NodeType(label)
+ except ValueError:
+ continue
+ self.function_registry[qn] = node_type
+ # Restore the property-name set for unchanged files: property-dispatch
+ # resolution (`obj.prop`) consults it, so a re-parsed file's call to a
+ # @property defined elsewhere would otherwise drop.
+ if row.get(cs.KEY_IS_PROPERTY):
+ self.function_registry.mark_property(qn)
+ # Restore the macro-namespace set for unchanged files: the Rust
+ # macro/fn gate consults it, so a re-parsed file's invocation of a
+ # macro defined elsewhere would otherwise drop.
+ if row.get(cs.KEY_IS_MACRO):
+ self.factory.definition_processor.macro_qns.add(qn)
+ # Record the defining file so _is_cpp_defined can language-check
+ # rehydrated candidates (deferred C++ INHERITS resolution runs
+ # after this and must reach bases in UNCHANGED headers).
+ if isinstance(path := row.get(cs.KEY_PATH), str):
+ self.factory.definition_processor.rehydrated_definition_paths[qn] = path
+ # Spans for hybrid expansion-call callee joins: only C/C++
+ # Function/Method rows carry a usable span.
+ start = row.get(cs.KEY_START_LINE)
+ end = row.get(cs.KEY_END_LINE)
+ if (
+ node_type in (NodeType.FUNCTION, NodeType.METHOD)
+ and isinstance(start, int)
+ and isinstance(end, int)
+ and os.path.splitext(path)[1].lower() in _CPP_SPAN_FILE_EXTENSIONS
+ ):
+ self._rehydrated_cpp_spans.setdefault(path, []).append(
+ CppDefinitionSpan(start, end, node_type.value, qn)
+ )
+ added += 1
+ if added:
+ logger.info(ls.REGISTRY_REHYDRATED, count=added)
+ # Module qns from unchanged files: deferred import verification and
+ # C++20 module-impl resolution must count them as real targets, or
+ # an incremental run would drop edges a clean index emits.
+ try:
+ module_rows = self.ingestor.fetch_all(
+ cs.CYPHER_ALL_MODULE_QNS, project_params
+ )
+ except Exception:
+ if not self._is_full_build:
+ raise
+ logger.warning(ls.REHYDRATE_QUERY_FAILED)
+ return
+ for row in module_rows:
+ qn = row.get(cs.KEY_QUALIFIED_NAME)
+ label = row.get(cs.KEY_LABEL)
+ if not isinstance(qn, str) or not isinstance(label, str):
+ continue
+ if label == cs.NodeLabel.MODULE_INTERFACE.value:
+ self.factory.definition_processor.cpp_module_interfaces.add(qn)
+ else:
+ self._rehydrated_module_qns.add(qn)
+ self._rehydrate_class_inheritance_from_graph()
+
+ def _seed_module_qns_from_graph(self, eligible_paths: set[str]) -> None:
+ # Cross-language module-qn disambiguation (definition_processor.
+ # _disambiguate_module_qn) only sees files processed this run. On an
+ # incremental ADD of a file whose basename collides with an already-
+ # indexed sibling of another language (shapes.rs owns proj.shapes,
+ # then shapes.cpp is added), the added file would re-claim the bare qn
+ # and overwrite the existing module under the qualified_name
+ # constraint. Seed the qn->file map from the graph BEFORE processing so
+ # the disambiguator sees taken qns; a re-parsed file keeps its bare qn.
+ if not isinstance(self.ingestor, QueryProtocol):
+ return
+ module_map = self.factory.definition_processor.module_qn_to_file_path
+ # Only incremental runs seed (full builds process every file), so a
+ # failed read here always aborts: a missing seed can silently let an
+ # added file overwrite a sibling module's qn.
+ for row in self.ingestor.fetch_all(cs.CYPHER_ALL_MODULE_PATHS_INTERNAL):
+ qn = row.get(cs.KEY_QUALIFIED_NAME)
+ path = row.get(cs.KEY_PATH)
+ if not isinstance(qn, str) or not isinstance(path, str) or not path:
+ continue
+ if path.startswith(cs.INLINE_MODULE_PATH_PREFIX):
+ continue
+ # Only seed modules whose file survives this run (still eligible).
+ # A file deleted OR newly excluded this cycle is gone from
+ # eligible_paths, so a same-basename ADD (delete shapes.rs + add
+ # shapes.cpp) takes the bare qn a clean index would give it.
+ if path not in eligible_paths:
+ continue
+ module_map.setdefault(qn, self.repo_path / path)
+
+ def _rehydrate_class_inheritance_from_graph(self) -> None:
+ # Incremental runs rebuild class_inheritance only from re-parsed files.
+ # Restore the child->bases map for classes defined in files that were
+ # not re-parsed, so protocol dispatch and inherited-method resolution
+ # work in Pass 3 (issue #532 residual). Only fill entries missing
+ # locally: a re-parsed class already has its fresh, correctly ordered
+ # bases, so we must not overwrite or duplicate them. CYPHER_ALL_INHERITS
+ # is ordered by base_index, so a rehydrated class's bases keep their
+ # original source order (multiple inheritance resolves the same base a
+ # clean index would).
+ if not isinstance(self.ingestor, QueryProtocol):
+ return
+ class_inheritance = self.factory.definition_processor.class_inheritance
+ try:
+ rows = self.ingestor.fetch_all(
+ cs.CYPHER_ALL_INHERITS,
+ {cs.KEY_PROJECT_PREFIX: self.project_name + "."},
+ )
+ except Exception:
+ if not self._is_full_build:
+ raise
+ logger.warning(ls.REHYDRATE_QUERY_FAILED)
+ return
+ for child, bases in self._rehydrated_bases_by_child(
+ rows, class_inheritance
+ ).items():
+ class_inheritance[child] = bases
+
+ @staticmethod
+ def _rehydrated_bases_by_child(
+ rows: list[ResultRow], existing: dict[str, list[str]]
+ ) -> dict[str, list[str]]:
+ # Group persisted INHERITS rows into child -> ordered bases, restoring
+ # source order from base_index. Skip children already present locally
+ # (freshly re-parsed). A class with more than one base needs a reliable
+ # order (method resolution / override attribution are first-match-wins
+ # over the base list); if any of its edges lacks a base_index (e.g. an
+ # INHERITS written by an older index before base_index existed) the
+ # order cannot be trusted, so that class is NOT rehydrated and falls
+ # back to name-based resolution rather than risk the wrong base.
+ # Single-base classes are order-independent and always safe.
+ collected: dict[str, list[tuple[int | None, str]]] = {}
+ for row in rows:
+ child = row.get(cs.KEY_CHILD_QN)
+ base = row.get(cs.KEY_BASE_QN)
+ if not isinstance(child, str) or not isinstance(base, str):
+ continue
+ if child in existing:
+ continue
+ raw_index = row.get(cs.KEY_BASE_INDEX)
+ index = raw_index if isinstance(raw_index, int) else None
+ collected.setdefault(child, []).append((index, base))
+ result: dict[str, list[str]] = {}
+ for child, pairs in collected.items():
+ if len(pairs) > 1 and any(index is None for index, _ in pairs):
+ continue
+ pairs.sort(key=lambda pair: (pair[0] is None, pair[0] or 0))
+ result[child] = [base for _index, base in pairs]
+ return result
+
+ def _capture_inbound_edges(self, reindexed_keys: list[str]) -> list[ResultRow]:
+ # Record the reference edges unchanged files point at the re-indexed
+ # files, BEFORE those files' subtrees (and thus the inbound edges) are
+ # deleted. Restoring the exact edges avoids re-resolving the callers,
+ # whose resolution would diverge from a clean index (cgr resolution is
+ # context-sensitive).
+ if not reindexed_keys or not isinstance(self.ingestor, QueryProtocol):
+ return []
+ try:
+ return self.ingestor.fetch_all(
+ cs.CYPHER_INBOUND_EDGES, {cs.CYPHER_PARAM_PATHS: reindexed_keys}
+ )
+ except Exception:
+ # A FULL build re-parses every caller, so nothing is lost and the
+ # sync may continue; an incremental run cannot re-resolve edges
+ # from files it will not parse, so the outage must abort it
+ # rather than silently drop them.
+ if not self._is_full_build:
+ raise
+ logger.warning(ls.INBOUND_CAPTURE_FAILED)
+ return []
+
+ def _restore_inbound_edges(self, captured: list[ResultRow]) -> None:
+ # Re-emit each captured inbound edge whose target still exists after the
+ # re-index. A target that was renamed or removed is correctly left
+ # without its stale inbound edge, matching a clean re-index.
+ if not captured:
+ return
+ module_label = cs.NodeLabel.MODULE.value
+ restored = 0
+ for row in captured:
+ caller_label = row.get(cs.KEY_CALLER_LABEL)
+ caller_qn = row.get(cs.KEY_CALLER_QN)
+ rel = row.get(cs.KEY_REL)
+ target_label = row.get(cs.KEY_TARGET_LABEL)
+ target_qn = row.get(cs.KEY_TARGET_QN)
+ if not (
+ isinstance(caller_label, str)
+ and isinstance(caller_qn, str)
+ and isinstance(rel, str)
+ and isinstance(target_label, str)
+ and isinstance(target_qn, str)
+ ):
+ continue
+ if target_label != module_label and target_qn not in self.function_registry:
+ continue
+ caller_key = cs.NODE_UNIQUE_CONSTRAINTS.get(caller_label)
+ target_key = cs.NODE_UNIQUE_CONSTRAINTS.get(target_label)
+ if caller_key is None or target_key is None:
+ continue
+ self._sink.ensure_relationship_batch(
+ (caller_label, caller_key, caller_qn),
+ rel,
+ (target_label, target_key, target_qn),
+ )
+ restored += 1
+ if restored:
+ logger.info(ls.INCREMENTAL_REBUILD_INBOUND, count=restored)
+
def remove_file_from_state(self, file_path: Path) -> None:
- logger.debug(ls.REMOVING_STATE.format(path=file_path))
+ logger.debug(ls.REMOVING_STATE, path=file_path)
if file_path in self.ast_cache:
del self.ast_cache[file_path]
logger.debug(ls.REMOVED_FROM_CACHE)
- relative_path = file_path.relative_to(self.repo_path)
+ relative_path = cached_relative_path(file_path, self.repo_path)
path_parts = (
relative_path.parent.parts
if file_path.name == cs.INIT_PY
@@ -307,53 +1261,725 @@ def remove_file_from_state(self, file_path: Path) -> None:
del self.function_registry[qn]
if qns_to_remove:
- logger.debug(ls.REMOVING_QNS.format(count=len(qns_to_remove)))
+ logger.debug(ls.REMOVING_QNS, count=len(qns_to_remove))
for simple_name, qn_set in self.simple_name_lookup.items():
original_count = len(qn_set)
new_qn_set = qn_set - qns_to_remove
if len(new_qn_set) < original_count:
self.simple_name_lookup[simple_name] = new_qn_set
- logger.debug(ls.CLEANED_SIMPLE_NAME.format(name=simple_name))
+ logger.debug(ls.CLEANED_SIMPLE_NAME, name=simple_name)
+
+ def _existing_module_paths(self) -> frozenset[str] | None:
+ """Paths of this project's Module nodes already in the graph.
+
+ Empty when the sink has no query surface (offline writers, unit
+ fakes) or answers with something that is not rows: such a sink holds
+ no readable state worth deleting first. None when the sink CLAIMS
+ readability but the query itself failed: the graph state is unknown,
+ and treating it as empty would skip every delete and recreate the
+ stale accumulation this probe exists to prevent.
+ """
+ fetch_all = getattr(self.ingestor, "fetch_all", None)
+ if fetch_all is None:
+ return frozenset()
+ try:
+ rows = fetch_all(
+ cs.CYPHER_PROJECT_MODULE_PATHS,
+ {
+ cs.KEY_PROJECT_NAME: self.project_name,
+ cs.KEY_PROJECT_PREFIX: self.project_name + ".",
+ },
+ )
+ except Exception:
+ return None
+ try:
+ return frozenset(
+ path
+ for row in rows
+ if isinstance(path := row.get(cs.KEY_PATH), str)
+ and not path.startswith(cs.INLINE_MODULE_PATH_PREFIX)
+ )
+ except (TypeError, AttributeError):
+ return frozenset()
+
+ def _delete_module_entities(self, file_key: str) -> None:
+ """Remove a changed/deleted file's Module subtree from the graph.
+
+ The incremental path re-parses a changed file and re-adds its
+ entities, but the entities the previous parse contributed (the
+ Module and everything it DEFINES, plus their IMPORTS/CALLS edges via
+ DETACH) must be removed first; otherwise renamed-away Function/Class/
+ Method nodes and their edges linger alongside the new ones.
+ """
+ if isinstance(self.ingestor, QueryProtocol):
+ self.ingestor.execute_write(
+ cs.CYPHER_DELETE_MODULE,
+ {
+ cs.KEY_PATH: file_key,
+ cs.KEY_PROJECT_NAME: self.project_name,
+ cs.KEY_PROJECT_PREFIX: self.project_name + ".",
+ },
+ )
- def _process_files(self) -> None:
- for filepath in self.repo_path.rglob("*"):
- if filepath.is_file() and not should_skip_path(
- filepath,
+ def _diff_dir_against_cache(
+ self,
+ dir_path_str: str,
+ dir_key: str,
+ old_hashes: FileHashCache,
+ old_dir_mtimes: DirMtimesCache,
+ ) -> tuple[str | None, str | None]:
+ prefix = "" if dir_key == cs.ROOT_DIR_KEY else f"{dir_key}/"
+ expected_files: set[str] = set()
+ expected_dirs: set[str] = set()
+ for fk in old_hashes:
+ if fk.startswith(prefix):
+ rest = fk[len(prefix) :]
+ if "/" not in rest:
+ expected_files.add(rest)
+ for dk in old_dir_mtimes:
+ if dk == cs.ROOT_DIR_KEY or not dk.startswith(prefix):
+ continue
+ rest = dk[len(prefix) :]
+ if "/" not in rest:
+ expected_dirs.add(rest)
+
+ actual_files: set[str] = set()
+ actual_dirs: set[str] = set()
+ try:
+ with os.scandir(dir_path_str) as it:
+ for entry in it:
+ name = entry.name
+ if name in cs.CGR_STATE_FILENAMES:
+ continue
+ try:
+ is_symlink = entry.is_symlink()
+ except OSError:
+ is_symlink = False
+ try:
+ is_dir_following = entry.is_dir()
+ except OSError:
+ is_dir_following = False
+ if is_symlink and is_dir_following:
+ continue
+ if is_dir_following:
+ actual_dirs.add(name)
+ else:
+ actual_files.add(name)
+ except OSError:
+ return None, dir_key
+
+ dir_parts: tuple[str, ...] = (
+ () if dir_key == cs.ROOT_DIR_KEY else tuple(dir_key.split("/"))
+ )
+ dir_prefix_for_keep = "" if dir_key == cs.ROOT_DIR_KEY else f"{dir_key}/"
+
+ for name in actual_dirs - expected_dirs:
+ if not self._should_keep_dir(name, dir_prefix_for_keep):
+ continue
+ return f"{prefix}{name}", None
+ for name in actual_files - expected_files:
+ dot = name.rfind(".")
+ suffix = name[dot:] if dot != -1 else ""
+ if should_skip_rel_file(
+ f"{prefix}{name}",
+ dir_parts,
+ suffix,
+ exclude_paths=self.exclude_paths,
+ unignore_paths=self.unignore_paths,
+ ):
+ continue
+ return f"{prefix}{name}", None
+
+ for name in expected_files - actual_files:
+ return None, f"{prefix}{name}"
+ for name in expected_dirs - actual_dirs:
+ return None, f"{prefix}{name}"
+
+ return None, None
+
+ def _should_keep_dir(self, dirname: str, dir_prefix: str) -> bool:
+ rel_dir = f"{dir_prefix}{dirname}"
+ # an explicit exclude can never be rescued by unignore (excludes win
+ # at the file level too), so prune the subtree outright.
+ if self.exclude_paths and matches_ignore_patterns(
+ f"{rel_dir}/", self.exclude_paths
+ ):
+ return False
+ if dirname not in cs.IGNORE_PATTERNS:
+ return True
+ # Cargo's src/bin/ holds first-party binaries, not build output;
+ # mirrors has_ignored_dir_part.
+ if (
+ dirname == cs.DIR_BIN
+ and dir_prefix.rstrip(cs.SEPARATOR_SLASH).rsplit(cs.SEPARATOR_SLASH, 1)[-1]
+ == cs.DIR_SRC
+ ):
+ return True
+ return bool(
+ self.unignore_paths
+ and any(
+ unignore_could_match_within(u, rel_dir) for u in self.unignore_paths
+ )
+ )
+
+ def _drop_cache_if_graph_lost(self) -> None:
+ """Discard the hash cache when the graph no longer holds this project.
+
+ The cache lives inside the repo, but the database is shared: cleaning
+ the database while indexing another repo, an MCP wipe_database, or a
+ fresh Memgraph instance voids the cache without touching it, and an
+ incremental sync that trusts it would skip every file and leave the
+ project silently empty.
+ """
+ cache_path = self.repo_path / cs.HASH_CACHE_FILENAME
+ if not cache_path.is_file():
+ return
+ fetch_all = getattr(self.ingestor, "fetch_all", None)
+ if fetch_all is None:
+ return
+ try:
+ rows = fetch_all(
+ cs.CYPHER_COUNT_PROJECT_MODULES,
+ {
+ cs.KEY_PROJECT_NAME: self.project_name,
+ cs.KEY_PROJECT_PREFIX: self.project_name + ".",
+ },
+ )
+ except Exception:
+ # A graph that cannot answer (connection refused, a sink that
+ # rejects reads) cannot invalidate the cache: fail open.
+ return
+ try:
+ # count() always yields exactly one row; anything else means the
+ # sink did not really answer and cannot invalidate the cache.
+ count = int(rows[0]["count"])
+ except (KeyError, IndexError, TypeError, ValueError):
+ return
+ if count:
+ return
+ logger.warning(ls.HASH_CACHE_ORPHANED.format(project=self.project_name))
+ cache_path.unlink(missing_ok=True)
+ (self.repo_path / cs.DIR_MTIMES_FILENAME).unlink(missing_ok=True)
+
+ def _warn_if_parser_changed(self) -> None:
+ # No hash cache means a full build is coming: nothing to compare.
+ if not (self.repo_path / cs.HASH_CACHE_FILENAME).is_file():
+ return
+ stored = _load_parser_fingerprint(
+ self.repo_path / cs.PARSER_FINGERPRINT_FILENAME
+ )
+ # A missing stamp on an existing graph means it was built by an
+ # unknown (pre-fingerprint) parser: treat it as stale too, without
+ # paying for a fingerprint computation that cannot match.
+ if stored is None or stored != compute_parser_fingerprint():
+ logger.warning(ls.PARSER_FINGERPRINT_MISMATCH)
+
+ def _is_already_in_sync(self) -> bool:
+ if self._single_file is not None:
+ return False
+ cache_path = self.repo_path / cs.HASH_CACHE_FILENAME
+ if not cache_path.is_file():
+ return False
+ cache_mtime = cache_path.stat().st_mtime
+ dir_mtimes_path = self.repo_path / cs.DIR_MTIMES_FILENAME
+ old_hashes = _load_hash_cache(cache_path)
+ old_dir_mtimes = _load_dir_mtimes(dir_mtimes_path)
+ if not old_hashes or not old_dir_mtimes:
+ return False
+
+ repo_str = str(self.repo_path)
+ for dir_key, cached_mtime in old_dir_mtimes.items():
+ dir_path_str = (
+ repo_str if dir_key == cs.ROOT_DIR_KEY else f"{repo_str}/{dir_key}"
+ )
+ try:
+ current_mtime = os.stat(dir_path_str).st_mtime
+ except OSError:
+ return False
+ if current_mtime != cached_mtime:
+ addition, removal = self._diff_dir_against_cache(
+ dir_path_str, dir_key, old_hashes, old_dir_mtimes
+ )
+ if addition is not None or removal is not None:
+ return False
+
+ for file_key, old_hash in old_hashes.items():
+ file_path_str = f"{repo_str}/{file_key}"
+ try:
+ stat = os.stat(file_path_str)
+ except OSError:
+ return False
+ if stat.st_mtime <= cache_mtime:
+ continue
+ if _hash_file(Path(file_path_str)) != old_hash:
+ return False
+ return True
+
+ def _collect_eligible_files(self) -> list[tuple[Path, str]]:
+ if self._single_file is not None:
+ if not should_skip_path(
+ self._single_file,
self.repo_path,
exclude_paths=self.exclude_paths,
unignore_paths=self.unignore_paths,
):
- lang_config = get_language_spec(filepath.suffix)
- if (
- lang_config
- and isinstance(lang_config.language, cs.SupportedLanguage)
- and lang_config.language in self.parsers
+ file_key = cached_relative_path(
+ self._single_file, self.repo_path
+ ).as_posix()
+ return [(self._single_file, file_key)]
+ return []
+
+ eligible: list[tuple[Path, str]] = []
+ state_filenames = cs.CGR_STATE_FILENAMES
+ repo_str = str(self.repo_path)
+ repo_prefix_len = len(repo_str) + 1
+ exclude_paths = self.exclude_paths
+ unignore_paths = self.unignore_paths
+ self._collected_dir_mtimes = {}
+ for dirpath, dirnames, filenames in os.walk(repo_str):
+ if len(dirpath) < repo_prefix_len:
+ rel_dir = ""
+ dir_parts: tuple[str, ...] = ()
+ dir_key = cs.ROOT_DIR_KEY
+ else:
+ rel_dir = dirpath[repo_prefix_len:].replace(os.sep, "/")
+ dir_parts = tuple(rel_dir.split("/")) if rel_dir else ()
+ dir_key = rel_dir or cs.ROOT_DIR_KEY
+ dir_prefix = f"{rel_dir}/" if rel_dir else ""
+ try:
+ self._collected_dir_mtimes[dir_key] = os.stat(dirpath).st_mtime
+ except OSError:
+ pass
+ dirnames[:] = sorted(
+ d for d in dirnames if self._should_keep_dir(d, dir_prefix)
+ )
+ for fname in sorted(filenames):
+ if fname in state_filenames:
+ continue
+ dot = fname.rfind(".")
+ suffix = fname[dot:] if dot != -1 else ""
+ rel_path_str = f"{dir_prefix}{fname}"
+ if not should_skip_rel_file(
+ rel_path_str,
+ dir_parts,
+ suffix,
+ exclude_paths=exclude_paths,
+ unignore_paths=unignore_paths,
):
- result = self.factory.definition_processor.process_file(
- filepath,
- lang_config.language,
- self.queries,
- self.factory.structure_processor.structural_elements,
+ eligible.append((Path(f"{dirpath}/{fname}"), rel_path_str))
+ return eligible
+
+ def _process_files(self, force: bool = False) -> None:
+ cache_path = self.repo_path / cs.HASH_CACHE_FILENAME
+ dir_mtimes_path = self.repo_path / cs.DIR_MTIMES_FILENAME
+ old_hashes = _load_hash_cache(cache_path) if not force else {}
+ is_full_build = (force or not old_hashes) and self._single_file is None
+ self._is_full_build = is_full_build
+ cache_mtime = cache_path.stat().st_mtime if cache_path.is_file() else 0.0
+ if force:
+ logger.info(ls.INCREMENTAL_FORCE)
+
+ _touch_empty_json(cache_path)
+ _touch_empty_json(dir_mtimes_path)
+
+ eligible_files = self._collect_eligible_files()
+
+ if not is_full_build:
+ self._seed_module_qns_from_graph({key for _fp, key in eligible_files})
+ # A full build can still land on a graph that already holds this
+ # project: the cache lives in the repo working tree, the graph does
+ # not, so a fresh clone of an indexed repo (or a force run) parses
+ # every file as "new" while the previous parse's state is still
+ # there. A file whose Module already exists must take the
+ # delete-before-reingest path or renamed-away symbols and their
+ # CALLS/REFERENCES edges accumulate alongside the fresh parse.
+ preexisting_paths = (
+ self._existing_module_paths() if is_full_build else frozenset()
+ )
+ new_hashes: FileHashCache = {}
+ skipped_count = 0
+ changed_count = 0
+ unreadable_count = 0
+ unreadable_keys: set[str] = set()
+
+ current_file_keys: set[str] = set()
+
+ processed_since_flush = 0
+
+ changed_entries: list[tuple[Path, str, bool, bytes]] = []
+ for filepath, file_key in eligible_files:
+ if not force and file_key in old_hashes:
+ try:
+ file_mtime = filepath.stat().st_mtime
+ except OSError:
+ unreadable_count += 1
+ unreadable_keys.add(file_key)
+ continue
+ if file_mtime <= cache_mtime:
+ new_hashes[file_key] = old_hashes[file_key]
+ current_file_keys.add(file_key)
+ skipped_count += 1
+ continue
+
+ hashed = _hash_file_with_bytes(filepath)
+ if hashed is None:
+ unreadable_count += 1
+ unreadable_keys.add(file_key)
+ continue
+ current_hash, file_bytes = hashed
+
+ current_file_keys.add(file_key)
+ new_hashes[file_key] = current_hash
+
+ if (
+ not force
+ and file_key in old_hashes
+ and old_hashes[file_key] == current_hash
+ ):
+ logger.debug(ls.FILE_HASH_UNCHANGED, path=file_key)
+ skipped_count += 1
+ continue
+
+ is_new = (
+ file_key not in old_hashes
+ and preexisting_paths is not None
+ and file_key not in preexisting_paths
+ )
+ if not is_new:
+ logger.debug(ls.FILE_HASH_CHANGED, path=file_key)
+ else:
+ logger.debug(ls.FILE_HASH_NEW, path=file_key)
+ changed_entries.append((filepath, file_key, is_new, file_bytes))
+
+ # Before deleting any changed file's subtree (which removes the inbound
+ # CALLS/IMPORTS/INSTANTIATES edges incident on it), capture those edges
+ # so they can be restored verbatim afterwards (issue #532, inbound
+ # half). New files have no prior inbound edges, so only re-indexed
+ # (changed, non-new) files matter.
+ reindexed_keys = sorted(
+ file_key for _fp, file_key, is_new, _b in changed_entries if not is_new
+ )
+ captured_inbound = self._capture_inbound_edges(reindexed_keys)
+ self._reparsed_file_keys = {
+ file_key for _fp, file_key, _new, _b in changed_entries
+ }
+
+ pre_parsed = self._pre_parse_changed_files(changed_entries)
+
+ with Progress(
+ SpinnerColumn(),
+ TextColumn(ls.PROGRESS_INDEXING_LABEL),
+ TextColumn("[progress.description]{task.description}"),
+ transient=True,
+ disable=not sys.stderr.isatty(),
+ ) as progress:
+ task = progress.add_task("", total=len(eligible_files))
+ if skipped_count or unreadable_count:
+ progress.advance(task, skipped_count + unreadable_count)
+
+ for filepath, file_key, is_new, file_bytes in changed_entries:
+ if not is_new:
+ self.remove_file_from_state(filepath)
+ self._delete_module_entities(file_key)
+
+ changed_count += 1
+ self._process_single_file(
+ filepath,
+ file_bytes=file_bytes,
+ pre_parsed=pre_parsed.get(filepath),
+ )
+
+ processed_since_flush += 1
+ if processed_since_flush >= settings.FILE_FLUSH_INTERVAL:
+ logger.info(ls.PERIODIC_FLUSH.format(count=processed_since_flush))
+ self.ingestor.flush_all()
+ processed_since_flush = 0
+
+ progress.update(
+ task,
+ advance=1,
+ description=ls.PROGRESS_FILES_PROCESSED.format(count=changed_count),
+ )
+
+ # On a cacheless rebuild old_hashes cannot name what disappeared, so
+ # the graph's own module paths join the reconciliation: a file that
+ # was deleted OR newly excluded since the previous index is absent
+ # from current_file_keys and its subtree must go. (Disk-deleted files
+ # are also swept by orphan pruning; excluded ones still exist on disk
+ # and only this reconciliation catches them.) A file that merely
+ # could not be READ this run still exists: sweeping it would erase
+ # live state over a transient error, so unreadable keys are exempt.
+ graph_paths = (
+ preexisting_paths if preexisting_paths is not None else frozenset()
+ )
+ deleted_keys = (
+ (set(old_hashes.keys()) | graph_paths) - current_file_keys - unreadable_keys
+ )
+ if deleted_keys:
+ logger.info(ls.INCREMENTAL_DELETED, count=len(deleted_keys))
+ for deleted_key in deleted_keys:
+ deleted_path = self.repo_path / deleted_key
+ self.remove_file_from_state(deleted_path)
+ self._delete_module_entities(deleted_key)
+ if isinstance(self.ingestor, QueryProtocol):
+ # Keyed on the absolute path: a sibling project's File
+ # node can share the relative path (issue #897).
+ self.ingestor.execute_write(
+ cs.CYPHER_DELETE_FILE,
+ {cs.KEY_PATH: deleted_path.resolve().as_posix()},
)
- if result:
- root_node, language = result
- self.ast_cache[filepath] = (root_node, language)
- elif self._is_dependency_file(filepath.name, filepath):
- self.factory.definition_processor.process_dependencies(filepath)
+ self._restore_inbound_edges(captured_inbound)
+
+ if skipped_count > 0:
+ logger.info(ls.INCREMENTAL_SKIPPED, count=skipped_count)
+ if changed_count > 0:
+ logger.info(ls.INCREMENTAL_CHANGED, count=changed_count)
+ if unreadable_count > 0:
+ logger.info(ls.INCREMENTAL_UNREADABLE, count=unreadable_count)
+
+ _save_hash_cache(cache_path, new_hashes)
+ _save_dir_mtimes(dir_mtimes_path, self._collected_dir_mtimes)
+ # Stamp only full builds: re-stamping an incremental run would
+ # silence the staleness warning while unchanged files still carry
+ # the old parser's edges.
+ if is_full_build:
+ _save_parser_fingerprint(
+ self.repo_path / cs.PARSER_FINGERPRINT_FILENAME,
+ compute_parser_fingerprint(),
+ )
+
+ def _pre_parse_changed_files(
+ self,
+ changed_entries: list[tuple[Path, str, bool, bytes]],
+ ) -> dict[Path, tuple[Node, dict[str, list] | None]]:
+ result: dict[Path, tuple[Node, dict[str, list] | None]] = {}
+ for filepath, _file_key, _is_new, file_bytes in changed_entries:
+ lang_config = get_language_spec(filepath.suffix)
+ if not (
+ lang_config
+ and isinstance(lang_config.language, cs.SupportedLanguage)
+ and lang_config.language in self.parsers
+ ):
+ continue
+ language = lang_config.language
+ parser = self.queries[language].get(cs.KEY_PARSER)
+ if not parser:
+ continue
+ tree = parse_with_preproc_recovery(parser, file_bytes, language)
+ root_node = tree.root_node
+ combined_query = COMBINED_FUNC_CLASS_IMPORT_QUERIES.get(language)
+ combined_captures: dict[str, list] | None = None
+ if combined_query:
+ cursor = QueryCursor(combined_query)
+ combined_captures = sorted_captures(cursor, root_node)
+ result[filepath] = (root_node, combined_captures)
+ return result
+
+ def _process_single_file(
+ self,
+ filepath: Path,
+ file_bytes: bytes | None = None,
+ pre_parsed: tuple[Node, dict[str, list] | None] | None = None,
+ ) -> None:
+ if self._cpp_frontend_covered:
+ rel = cached_relative_path(filepath, self.repo_path).as_posix()
+ if rel in self._cpp_frontend_covered:
+ # The libclang frontend already emitted this file's
+ # definitions; keep only the generic File node.
self.factory.structure_processor.process_generic_file(
filepath, filepath.name
)
+ return
+
+ lang_config = get_language_spec(filepath.suffix)
+ if (
+ lang_config
+ and isinstance(lang_config.language, cs.SupportedLanguage)
+ and lang_config.language in self.parsers
+ ):
+ result = self.factory.definition_processor.process_file(
+ filepath,
+ lang_config.language,
+ self.queries,
+ self.factory.structure_processor.structural_elements,
+ source_bytes=file_bytes,
+ pre_parsed=pre_parsed,
+ )
+ if result:
+ root_node, language = result
+ self.ast_cache[filepath] = (root_node, language)
+ self._parsed_files.append((filepath, language))
+ elif self._is_dependency_file(filepath.name, filepath):
+ self.factory.definition_processor.process_dependencies(filepath)
+ elif self.ast_grep_tier.handles(filepath.suffix):
+ self.ast_grep_tier.process_file(
+ filepath, self.factory.structure_processor.structural_elements
+ )
+
+ self.factory.structure_processor.process_generic_file(filepath, filepath.name)
+
+ def _ast_for(self, file_path: Path) -> Node | None:
+ entry = self.ast_cache.load(file_path)
+ return entry[0] if entry else None
+
+ def _load_ast_from_disk(
+ self, file_path: Path
+ ) -> tuple[Node, cs.SupportedLanguage] | None:
+ # BoundedASTCache loader: re-parse an evicted file. Evicted files carry
+ # stale captures (nodes from the discarded tree), so drop them:
+ # downstream recomputes captures from the fresh tree.
+ language = get_language_for_extension(file_path.suffix)
+ if language is None or language not in self.parsers:
+ return None
+ parser = self.queries[language].get(cs.KEY_PARSER)
+ if parser is None:
+ return None
+ try:
+ file_bytes = file_path.read_bytes()
+ except OSError as e:
+ logger.error(ls.CALL_PROCESSING_FAILED, path=file_path, error=e)
+ return None
+ root_node = parse_with_preproc_recovery(parser, file_bytes, language).root_node
+ self.factory._func_class_captures_cache.pop(file_path, None)
+ return (root_node, language)
def _process_function_calls(self) -> None:
- ast_cache_items = list(self.ast_cache.items())
- for file_path, (root_node, language) in ast_cache_items:
+ captures_cache = self.factory._func_class_captures_cache
+ # Iterate every file parsed this run, not the bounded AST cache: on a
+ # large repo the cache evicts most files, and iterating it drops their
+ # calls (a whole module ends up with zero CALLS edges).
+ for file_path, language in self._parsed_files:
+ root_node = self._ast_for(file_path)
+ if root_node is None:
+ continue
+ self.factory.call_processor.collect_callable_field_bindings(
+ file_path,
+ root_node,
+ language,
+ self.queries,
+ func_class_captures_cache=captures_cache,
+ )
+ # Bindings are pending until every file's ctor metadata (param order,
+ # param->attribute renames) is in: a construction site may be scanned
+ # before the file defining its class.
+ self.factory.call_processor.finalize_callable_field_bindings()
+ for file_path, language in self._parsed_files:
+ root_node = self._ast_for(file_path)
+ if root_node is None:
+ continue
+ if captures_cache is not None and file_path in captures_cache:
+ cached = captures_cache[file_path]
+ if not cached.get(cs.CAPTURE_CALL) and not cached.get(
+ cs.CAPTURE_FUNCTION
+ ):
+ continue
self.factory.call_processor.process_calls_in_file(
- file_path, root_node, language, self.queries
+ file_path,
+ root_node,
+ language,
+ self.queries,
+ func_class_captures_cache=captures_cache,
)
+ self.factory.call_processor.finalize_callable_param_flow()
+ self.factory.call_processor.finalize_flow()
+
+ def _prune_orphan_nodes(self) -> None:
+ """Remove graph nodes whose files/folders no longer exist on disk."""
+ if not isinstance(self.ingestor, QueryProtocol):
+ return
+
+ logger.info(ls.PRUNE_START)
+ total_pruned = 0
+
+ project_prefix = self.project_name + "."
+ repo_abs = self.repo_path.resolve().as_posix()
+ prune_specs: list[tuple[str, str, str]] = [
+ (cs.CYPHER_ALL_FILE_PATHS, cs.CYPHER_DELETE_FILE, "File"),
+ (
+ cs.CYPHER_ALL_MODULE_PATHS_INTERNAL,
+ cs.CYPHER_DELETE_MODULE,
+ "Module",
+ ),
+ (cs.CYPHER_ALL_FOLDER_PATHS, cs.CYPHER_DELETE_FOLDER, "Folder"),
+ ]
+
+ read_failed = False
+ for query_all, delete_query, label in prune_specs:
+ try:
+ rows = self.ingestor.fetch_all(query_all)
+ except Exception:
+ # A graph that cannot be read cannot be pruned safely; the
+ # next healthy run sweeps whatever this one left behind.
+ logger.warning(ls.PRUNE_QUERY_FAILED, label=label)
+ read_failed = True
+ continue
+ orphans = []
+ for r in rows:
+ path = r.get("path")
+ if not isinstance(path, str) or not path:
+ continue
+ if path.startswith(cs.INLINE_MODULE_PATH_PREFIX):
+ continue
+ abs_path = r.get("absolute_path")
+ qn = r.get("qualified_name", "")
+ # Component-aware containment: a bare prefix test would also
+ # match a sibling root such as -old (issue #897).
+ if isinstance(abs_path, str) and not (
+ abs_path == repo_abs or abs_path.startswith(repo_abs + "/")
+ ):
+ continue
+ if isinstance(qn, str) and qn and not qn.startswith(project_prefix):
+ continue
+ if not (self.repo_path / path).exists():
+ # File/Folder deletes key on the absolute path: a sibling
+ # project's node can share the relative path (issue #897).
+ key = (
+ abs_path
+ if isinstance(abs_path, str) and abs_path
+ else (self.repo_path / path).resolve().as_posix()
+ )
+ orphans.append((path, key))
+
+ if orphans:
+ logger.info(ls.PRUNE_FOUND, count=len(orphans), label=label)
+ for orphan_path, orphan_key in orphans:
+ logger.debug(ls.PRUNE_DELETING, label=label, path=orphan_path)
+ if delete_query == cs.CYPHER_DELETE_MODULE:
+ # Module deletes are project-scoped; a sibling
+ # project's module can share the relative path.
+ self._delete_module_entities(orphan_path)
+ else:
+ self.ingestor.execute_write(
+ delete_query, {cs.KEY_PATH: orphan_key}
+ )
+ total_pruned += len(orphans)
+
+ if read_failed:
+ # The same outage that broke the path reads would break (or act
+ # on stale state through) the cleanup below; leave everything
+ # for the next healthy run.
+ return
+
+ # Drop external import-target modules that no module imports anymore,
+ # e.g. an imported name renamed/removed on an incremental rebuild.
+ self.ingestor.execute_write(cs.CYPHER_DELETE_ORPHAN_EXTERNAL_MODULES)
+
+ # Drop shared Resource nodes whose component no longer reaches any
+ # code node, e.g. an endpoint whose route changed on a rebuild.
+ prune_unanchored_resources(self.ingestor)
+
+ if total_pruned:
+ logger.info(ls.PRUNE_COMPLETE, count=total_pruned)
+ else:
+ logger.info(ls.PRUNE_SKIP)
def _generate_semantic_embeddings(self) -> None:
+ if self.skip_embeddings:
+ logger.info(ls.EMBEDDINGS_SKIPPED)
+ return
+
if not has_semantic_dependencies():
logger.info(ls.SEMANTIC_NOT_AVAILABLE)
return
@@ -363,22 +1989,55 @@ def _generate_semantic_embeddings(self) -> None:
return
try:
- from .embedder import embed_code
- from .vector_store import store_embedding
+ from .embedder import embed_code_batch, get_embedding_cache
+ from .vector_store import (
+ close_qdrant_client,
+ store_embedding_batch,
+ verify_stored_ids,
+ )
logger.info(ls.PASS_4_EMBEDDINGS)
results = self.ingestor.fetch_all(
- cs.CYPHER_QUERY_EMBEDDINGS, {"project_name": self.project_name + "."}
+ cs.CYPHER_QUERY_EMBEDDINGS, {"project_name": self.project_name}
)
if not results:
logger.info(ls.NO_FUNCTIONS_FOR_EMBEDDING)
return
- logger.info(ls.GENERATING_EMBEDDINGS.format(count=len(results)))
+ logger.info(ls.GENERATING_EMBEDDINGS, count=len(results))
embedded_count = 0
+ expected_ids: set[int] = set()
+ pending: list[tuple[int, str, str]] = []
+ flush_at = settings.QDRANT_BATCH_SIZE
+
+ def flush() -> int:
+ nonlocal pending
+ if not pending:
+ return 0
+ snippets = [item[2] for item in pending]
+ try:
+ embeddings = embed_code_batch(snippets)
+ except Exception as e:
+ logger.warning(
+ ls.EMBEDDING_BATCH_COMPUTE_FAILED,
+ count=len(pending),
+ error=e,
+ )
+ pending = []
+ return 0
+ points: list[tuple[int, list[float], str]] = [
+ (node_id, emb, qname)
+ for (node_id, qname, _), emb in zip(pending, embeddings)
+ ]
+ for node_id, _qname, _src in pending:
+ expected_ids.add(node_id)
+ stored = store_embedding_batch(points)
+ pending = []
+ return stored
+
for row in results:
parsed = self._parse_embedding_result(row)
if parsed is None:
@@ -391,33 +2050,62 @@ def _generate_semantic_embeddings(self) -> None:
file_path = parsed.get(cs.KEY_PATH)
if start_line is None or end_line is None or file_path is None:
- logger.debug(ls.NO_SOURCE_FOR.format(name=qualified_name))
+ logger.debug(ls.NO_SOURCE_FOR, name=qualified_name)
+ continue
- elif source_code := self._extract_source_code(
+ if source_code := self._extract_source_code(
qualified_name, file_path, start_line, end_line
):
- try:
- embedding = embed_code(source_code)
- store_embedding(node_id, embedding, qualified_name)
- embedded_count += 1
-
- if embedded_count % settings.EMBEDDING_PROGRESS_INTERVAL == 0:
+ pending.append((node_id, qualified_name, source_code))
+ if len(pending) >= flush_at:
+ embedded_count += flush()
+ if (
+ embedded_count % settings.EMBEDDING_PROGRESS_INTERVAL == 0
+ and embedded_count > 0
+ ):
logger.debug(
- ls.EMBEDDING_PROGRESS.format(
- done=embedded_count, total=len(results)
- )
+ ls.EMBEDDING_PROGRESS,
+ done=embedded_count,
+ total=len(results),
)
-
- except Exception as e:
- logger.warning(
- ls.EMBEDDING_FAILED.format(name=qualified_name, error=e)
- )
else:
- logger.debug(ls.NO_SOURCE_FOR.format(name=qualified_name))
- logger.info(ls.EMBEDDINGS_COMPLETE.format(count=embedded_count))
+ logger.debug(ls.NO_SOURCE_FOR, name=qualified_name)
+
+ embedded_count += flush()
+
+ logger.info(ls.EMBEDDINGS_COMPLETE, count=embedded_count)
+ self._reconcile_embeddings(expected_ids, verify_stored_ids)
+
+ get_embedding_cache().save()
+ close_qdrant_client()
+
+ except Exception as e:
+ logger.warning(ls.EMBEDDING_GENERATION_FAILED, error=e)
+
+ def _reconcile_embeddings(
+ self,
+ expected_ids: set[int],
+ verify_fn: Callable[[set[int]], set[int]],
+ ) -> None:
+ if not expected_ids:
+ return
+ try:
+ stored_ids = verify_fn(expected_ids)
+ missing = expected_ids - stored_ids
+ if missing:
+ sample = sorted(missing)[:10]
+ logger.warning(
+ ls.EMBEDDING_RECONCILE_MISSING.format(
+ missing=len(missing),
+ expected=len(expected_ids),
+ sample_ids=sample,
+ )
+ )
+ else:
+ logger.info(ls.EMBEDDING_RECONCILE_OK.format(count=len(expected_ids)))
except Exception as e:
- logger.warning(ls.EMBEDDING_GENERATION_FAILED.format(error=e))
+ logger.warning(ls.EMBEDDING_RECONCILE_FAILED.format(error=e))
def _extract_source_code(
self, qualified_name: str, file_path: str, start_line: int, end_line: int
diff --git a/codebase_rag/language_spec.py b/codebase_rag/language_spec.py
index cf550ab08..2b9cdb43f 100644
--- a/codebase_rag/language_spec.py
+++ b/codebase_rag/language_spec.py
@@ -82,6 +82,14 @@ def _rust_get_name(node: Node) -> str | None:
name_node = node.child_by_field_name(cs.FIELD_NAME)
if name_node and name_node.type == cs.TS_IDENTIFIER and name_node.text:
return name_node.text.decode(cs.ENCODING_UTF8)
+ elif node.type == cs.TS_IMPL_ITEM:
+ # An `impl Foo` block is an FQN scope but has no `name` field; its
+ # target type anchors its methods' qns (owner_module.Foo.method).
+ # Without this the scope walk drops `Foo`, so a nested fn in an impl
+ # method binds to a phantom parent qn.
+ from .parsers.rs import utils as rs_utils
+
+ return rs_utils.extract_impl_target(node)
return _generic_get_name(node)
@@ -97,7 +105,47 @@ def _rust_file_to_module(file_path: Path, repo_root: Path) -> list[str]:
return []
+def _php_file_to_module(file_path: Path, repo_root: Path) -> list[str]:
+ try:
+ rel = file_path.relative_to(repo_root)
+ parts = list(rel.with_suffix("").parts)
+ if parts and parts[0] in ("src", "app", "lib"):
+ parts = parts[1:]
+ return parts
+ except ValueError:
+ return []
+
+
+def _c_unwrap_declarator(declarator: Node | None) -> Node | None:
+ while declarator and declarator.type == cs.CppNodeType.POINTER_DECLARATOR:
+ declarator = declarator.child_by_field_name(cs.FIELD_DECLARATOR)
+ return declarator
+
+
+def _c_get_name(node: Node) -> str | None:
+ if node.type in cs.C_NAME_NODE_TYPES:
+ name_node = node.child_by_field_name(cs.FIELD_NAME)
+ if name_node and name_node.text:
+ return name_node.text.decode(cs.ENCODING_UTF8)
+ elif node.type == cs.TS_CPP_FUNCTION_DEFINITION:
+ declarator = node.child_by_field_name(cs.FIELD_DECLARATOR)
+ declarator = _c_unwrap_declarator(declarator)
+ if declarator and declarator.type == cs.TS_CPP_FUNCTION_DECLARATOR:
+ name_node = declarator.child_by_field_name(cs.FIELD_DECLARATOR)
+ if name_node and name_node.type == cs.TS_IDENTIFIER and name_node.text:
+ return name_node.text.decode(cs.ENCODING_UTF8)
+ return _generic_get_name(node)
+
+
def _cpp_get_name(node: Node) -> str | None:
+ # C++17 `namespace a::b {` is ONE node named `a::b`; render it as
+ # dotted segments so both nesting spellings, the namespace walk in
+ # cpp/utils, and the libclang frontend agree on qns.
+ if node.type == cs.CppNodeType.NAMESPACE_DEFINITION:
+ name = _generic_get_name(node)
+ if name:
+ return name.replace(cs.SEPARATOR_DOUBLE_COLON, cs.SEPARATOR_DOT)
+ return name
if node.type in cs.CPP_NAME_NODE_TYPES:
name_node = node.child_by_field_name(cs.FIELD_NAME)
if name_node and name_node.text:
@@ -112,6 +160,59 @@ def _cpp_get_name(node: Node) -> str | None:
return _generic_get_name(node)
+def _csharp_get_name(node: Node) -> str | None:
+ # A file-scoped `namespace N;` is a SIBLING of the declarations it
+ # governs, not their ancestor, so it never appears in a type's ancestor
+ # walk. compilation_unit IS every top-level type's ancestor, so fold the
+ # file-scoped namespace in here. Block `namespace N { }` is an ordinary
+ # ancestor and needs no shim.
+ if node.type == cs.TS_CSHARP_COMPILATION_UNIT:
+ for child in node.children:
+ if child.type == cs.TS_CSHARP_FILE_SCOPED_NAMESPACE_DECLARATION:
+ name_node = child.child_by_field_name(cs.TS_CSHARP_FIELD_NAME)
+ if name_node and name_node.text:
+ return name_node.text.decode(cs.ENCODING_UTF8)
+ return None
+ # Operators expose no `name` field and a destructor's `name` collides
+ # with the constructor; delegate to the shared synthesizer so the FQN
+ # scope walk and the registered node qn agree. Local import avoids a
+ # module-load cycle (csharp.utils -> parsers.utils).
+ if node.type in cs.CSHARP_SYNTHESIZED_NAME_TYPES:
+ from .parsers.csharp import utils as csharp_utils
+
+ return csharp_utils.synthesize_method_name(node)
+ name = _generic_get_name(node)
+ # A reserved keyword as the name means tree-sitter parse-recovered a broken
+ # construct into a declaration node (e.g. a `#if` splitting an if/else chain
+ # makes the trailing `else if` parse as a local_function named `if`). Such a
+ # node is never a real definition, so drop it.
+ if name in cs.CSHARP_RESERVED_KEYWORDS:
+ return None
+ return name
+
+
+def _dart_get_name(node: Node) -> str | None:
+ # Most Dart declarations expose a `name` field (functions, getters,
+ # setters, classes, enums, extensions). Constructors/factories and mixins
+ # do not: their LAST bare `identifier` child is the declared name
+ # (`C.named` -> `named`, `mixin Swimmer` -> `Swimmer`, a default
+ # constructor `C(...)` -> `C`). The constructor check comes FIRST: the
+ # grammar's `name` field on constructor_signature is the CLASS identifier,
+ # which would collapse every named constructor into a duplicate default.
+ if node.type in cs.DART_CONSTRUCTOR_SIGNATURE_TYPES:
+ ids = [c for c in node.named_children if c.type == cs.TS_IDENTIFIER and c.text]
+ if ids:
+ return ids[-1].text.decode(cs.ENCODING_UTF8)
+ return None
+ name_node = node.child_by_field_name(cs.FIELD_NAME)
+ if name_node and name_node.text:
+ return name_node.text.decode(cs.ENCODING_UTF8)
+ ids = [c for c in node.named_children if c.type == cs.TS_IDENTIFIER and c.text]
+ if ids:
+ return ids[-1].text.decode(cs.ENCODING_UTF8)
+ return None
+
+
PYTHON_FQN_SPEC = FQNSpec(
scope_node_types=frozenset(cs.FQN_PY_SCOPE_TYPES),
function_node_types=frozenset(cs.FQN_PY_FUNCTION_TYPES),
@@ -154,6 +255,13 @@ def _cpp_get_name(node: Node) -> str | None:
file_to_module_parts=_generic_file_to_module,
)
+C_FQN_SPEC = FQNSpec(
+ scope_node_types=frozenset(cs.FQN_C_SCOPE_TYPES),
+ function_node_types=frozenset(cs.FQN_C_FUNCTION_TYPES),
+ get_name=_c_get_name,
+ file_to_module_parts=_generic_file_to_module,
+)
+
LUA_FQN_SPEC = FQNSpec(
scope_node_types=frozenset(cs.FQN_LUA_SCOPE_TYPES),
function_node_types=frozenset(cs.FQN_LUA_FUNCTION_TYPES),
@@ -175,17 +283,24 @@ def _cpp_get_name(node: Node) -> str | None:
file_to_module_parts=_generic_file_to_module,
)
-CSHARP_FQN_SPEC = FQNSpec(
- scope_node_types=frozenset(cs.FQN_CS_SCOPE_TYPES),
- function_node_types=frozenset(cs.FQN_CS_FUNCTION_TYPES),
- get_name=_generic_get_name,
- file_to_module_parts=_generic_file_to_module,
-)
-
PHP_FQN_SPEC = FQNSpec(
scope_node_types=frozenset(cs.FQN_PHP_SCOPE_TYPES),
function_node_types=frozenset(cs.FQN_PHP_FUNCTION_TYPES),
get_name=_generic_get_name,
+ file_to_module_parts=_php_file_to_module,
+)
+
+CSHARP_FQN_SPEC = FQNSpec(
+ scope_node_types=frozenset(cs.FQN_CSHARP_SCOPE_TYPES),
+ function_node_types=frozenset(cs.FQN_CSHARP_FUNCTION_TYPES),
+ get_name=_csharp_get_name,
+ file_to_module_parts=_generic_file_to_module,
+)
+
+DART_FQN_SPEC = FQNSpec(
+ scope_node_types=frozenset(cs.FQN_DART_SCOPE_TYPES),
+ function_node_types=frozenset(cs.FQN_DART_FUNCTION_TYPES),
+ get_name=_dart_get_name,
file_to_module_parts=_generic_file_to_module,
)
@@ -193,17 +308,30 @@ def _cpp_get_name(node: Node) -> str | None:
cs.SupportedLanguage.PYTHON: PYTHON_FQN_SPEC,
cs.SupportedLanguage.JS: JS_FQN_SPEC,
cs.SupportedLanguage.TS: TS_FQN_SPEC,
+ cs.SupportedLanguage.TSX: TS_FQN_SPEC,
cs.SupportedLanguage.RUST: RUST_FQN_SPEC,
cs.SupportedLanguage.JAVA: JAVA_FQN_SPEC,
+ cs.SupportedLanguage.C: C_FQN_SPEC,
cs.SupportedLanguage.CPP: CPP_FQN_SPEC,
cs.SupportedLanguage.LUA: LUA_FQN_SPEC,
cs.SupportedLanguage.GO: GO_FQN_SPEC,
cs.SupportedLanguage.SCALA: SCALA_FQN_SPEC,
- cs.SupportedLanguage.CSHARP: CSHARP_FQN_SPEC,
cs.SupportedLanguage.PHP: PHP_FQN_SPEC,
+ cs.SupportedLanguage.CSHARP: CSHARP_FQN_SPEC,
+ cs.SupportedLanguage.DART: DART_FQN_SPEC,
}
+# Node-type sets shared by the typescript and tsx grammar variants.
+_TS_FUNCTION_NODE_TYPES = cs.JS_TS_FUNCTION_NODES + (cs.TS_FUNCTION_SIGNATURE,)
+_TS_CLASS_NODE_TYPES = cs.JS_TS_CLASS_NODES + (
+ cs.TS_ABSTRACT_CLASS_DECLARATION,
+ cs.TS_ENUM_DECLARATION,
+ cs.TS_INTERFACE_DECLARATION,
+ cs.TS_TYPE_ALIAS_DECLARATION,
+ cs.TS_INTERNAL_MODULE,
+)
+
LANGUAGE_SPECS: dict[cs.SupportedLanguage, LanguageSpec] = {
cs.SupportedLanguage.PYTHON: LanguageSpec(
language=cs.SupportedLanguage.PYTHON,
@@ -229,15 +357,22 @@ def _cpp_get_name(node: Node) -> str | None:
cs.SupportedLanguage.TS: LanguageSpec(
language=cs.SupportedLanguage.TS,
file_extensions=cs.TS_EXTENSIONS,
- function_node_types=cs.JS_TS_FUNCTION_NODES + (cs.TS_FUNCTION_SIGNATURE,),
- class_node_types=cs.JS_TS_CLASS_NODES
- + (
- cs.TS_ABSTRACT_CLASS_DECLARATION,
- cs.TS_ENUM_DECLARATION,
- cs.TS_INTERFACE_DECLARATION,
- cs.TS_TYPE_ALIAS_DECLARATION,
- cs.TS_INTERNAL_MODULE,
- ),
+ function_node_types=_TS_FUNCTION_NODE_TYPES,
+ class_node_types=_TS_CLASS_NODE_TYPES,
+ module_node_types=cs.SPEC_JS_MODULE_TYPES,
+ call_node_types=cs.SPEC_JS_CALL_TYPES,
+ import_node_types=cs.JS_TS_IMPORT_NODES,
+ import_from_node_types=cs.JS_TS_IMPORT_NODES,
+ ),
+ # .tsx needs the SEPARATE tsx grammar: the plain typescript grammar turns
+ # JSX into an ERROR forest (dropping every call inside a component), and
+ # the tsx grammar misparses bare generic arrows (`(x) => x`) that are
+ # legal .ts, so each extension keeps its own grammar with a shared spec.
+ cs.SupportedLanguage.TSX: LanguageSpec(
+ language=cs.SupportedLanguage.TSX,
+ file_extensions=cs.TSX_EXTENSIONS,
+ function_node_types=_TS_FUNCTION_NODE_TYPES,
+ class_node_types=_TS_CLASS_NODE_TYPES,
module_node_types=cs.SPEC_JS_MODULE_TYPES,
call_node_types=cs.SPEC_JS_CALL_TYPES,
import_node_types=cs.JS_TS_IMPORT_NODES,
@@ -259,6 +394,8 @@ def _cpp_get_name(node: Node) -> str | None:
(function_signature_item
name: (identifier) @name) @function
(closure_expression) @function
+ (macro_definition
+ name: (identifier) @name) @function
""",
class_query="""
(struct_item
@@ -285,8 +422,14 @@ def _cpp_get_name(node: Node) -> str | None:
function: (scoped_identifier
"::"
name: (identifier) @name)) @call
+ (call_expression
+ function: (generic_function) @name) @call
(macro_invocation
macro: (identifier) @name) @call
+ (token_tree
+ (identifier) @name @call
+ .
+ (token_tree . "("))
""",
),
cs.SupportedLanguage.GO: LanguageSpec(
@@ -340,9 +483,31 @@ def _cpp_get_name(node: Node) -> str | None:
(method_invocation
name: (identifier) @name) @call
(object_creation_expression
- type: (type_identifier) @name) @call
+ type: (_) @name) @call
""",
),
+ cs.SupportedLanguage.C: LanguageSpec(
+ language=cs.SupportedLanguage.C,
+ file_extensions=cs.C_EXTENSIONS,
+ function_node_types=cs.SPEC_C_FUNCTION_TYPES,
+ class_node_types=cs.SPEC_C_CLASS_TYPES,
+ module_node_types=cs.SPEC_C_MODULE_TYPES,
+ call_node_types=cs.SPEC_C_CALL_TYPES,
+ import_node_types=cs.IMPORT_NODES_INCLUDE,
+ import_from_node_types=cs.IMPORT_NODES_INCLUDE,
+ package_indicators=cs.SPEC_C_PACKAGE_INDICATORS,
+ function_query="""
+ (function_definition) @function
+ """,
+ class_query="""
+ (struct_specifier) @class
+ (union_specifier) @class
+ (enum_specifier) @class
+ """,
+ call_query="""
+ (call_expression) @call
+ """,
+ ),
cs.SupportedLanguage.CPP: LanguageSpec(
language=cs.SupportedLanguage.CPP,
file_extensions=cs.CPP_EXTENSIONS,
@@ -381,16 +546,6 @@ def _cpp_get_name(node: Node) -> str | None:
(delete_expression) @call
""",
),
- cs.SupportedLanguage.CSHARP: LanguageSpec(
- language=cs.SupportedLanguage.CSHARP,
- file_extensions=cs.CS_EXTENSIONS,
- function_node_types=cs.SPEC_CS_FUNCTION_TYPES,
- class_node_types=cs.SPEC_CS_CLASS_TYPES,
- module_node_types=cs.SPEC_CS_MODULE_TYPES,
- call_node_types=cs.SPEC_CS_CALL_TYPES,
- import_node_types=cs.IMPORT_NODES_USING,
- import_from_node_types=cs.IMPORT_NODES_USING,
- ),
cs.SupportedLanguage.PHP: LanguageSpec(
language=cs.SupportedLanguage.PHP,
file_extensions=cs.PHP_EXTENSIONS,
@@ -398,6 +553,42 @@ def _cpp_get_name(node: Node) -> str | None:
class_node_types=cs.SPEC_PHP_CLASS_TYPES,
module_node_types=cs.SPEC_PHP_MODULE_TYPES,
call_node_types=cs.SPEC_PHP_CALL_TYPES,
+ import_node_types=cs.SPEC_PHP_IMPORT_TYPES,
+ import_from_node_types=cs.SPEC_PHP_IMPORT_FROM_TYPES,
+ function_query="""
+ (function_definition
+ name: (name) @name) @function
+ (method_declaration
+ name: (name) @name) @function
+ (anonymous_function) @function
+ (arrow_function) @function
+ """,
+ class_query="""
+ (class_declaration
+ name: (name) @name) @class
+ (interface_declaration
+ name: (name) @name) @class
+ (trait_declaration
+ name: (name) @name) @class
+ (enum_declaration
+ name: (name) @name) @class
+ """,
+ call_query="""
+ (function_call_expression
+ function: (name) @name) @call
+ (function_call_expression
+ function: (qualified_name) @name) @call
+ (member_call_expression
+ name: (name) @name) @call
+ (scoped_call_expression
+ name: (name) @name) @call
+ (nullsafe_member_call_expression
+ name: (name) @name) @call
+ (object_creation_expression
+ (name) @name) @call
+ (object_creation_expression
+ (qualified_name) @name) @call
+ """,
),
cs.SupportedLanguage.LUA: LanguageSpec(
language=cs.SupportedLanguage.LUA,
@@ -408,6 +599,56 @@ def _cpp_get_name(node: Node) -> str | None:
call_node_types=cs.SPEC_LUA_CALL_TYPES,
import_node_types=cs.SPEC_LUA_IMPORT_TYPES,
),
+ cs.SupportedLanguage.CSHARP: LanguageSpec(
+ language=cs.SupportedLanguage.CSHARP,
+ file_extensions=cs.CS_EXTENSIONS,
+ function_node_types=cs.SPEC_CSHARP_FUNCTION_TYPES,
+ class_node_types=cs.SPEC_CSHARP_CLASS_TYPES,
+ module_node_types=cs.SPEC_CSHARP_MODULE_TYPES,
+ call_node_types=cs.SPEC_CSHARP_CALL_TYPES,
+ import_node_types=cs.SPEC_CSHARP_IMPORT_TYPES,
+ import_from_node_types=cs.SPEC_CSHARP_IMPORT_TYPES,
+ # Bare captures (like C/C++): names come from _csharp_get_name, since
+ # operators/ctors/dtors have no uniform `name` field.
+ function_query="""
+ (method_declaration) @function
+ (constructor_declaration) @function
+ (destructor_declaration) @function
+ (local_function_statement) @function
+ (operator_declaration) @function
+ (conversion_operator_declaration) @function
+ (property_declaration) @function
+ """,
+ class_query="""
+ (class_declaration) @class
+ (struct_declaration) @class
+ (record_declaration) @class
+ (interface_declaration) @class
+ (enum_declaration) @class
+ """,
+ call_query="""
+ (invocation_expression) @call
+ (object_creation_expression) @call
+ (implicit_object_creation_expression) @call
+ """,
+ ),
+ cs.SupportedLanguage.DART: LanguageSpec(
+ language=cs.SupportedLanguage.DART,
+ file_extensions=cs.DART_EXTENSIONS,
+ function_node_types=cs.SPEC_DART_FUNCTION_TYPES,
+ class_node_types=cs.SPEC_DART_CLASS_TYPES,
+ module_node_types=cs.SPEC_DART_MODULE_TYPES,
+ # The grammar has no call-expression node: a call site is a
+ # `selector` (or `cascade_section`) holding an `argument_part`,
+ # invoking whatever the preceding sibling chain names; the call
+ # name is reassembled by dart_call_name. Names come from
+ # _dart_get_name (bare captures); the signature/body split is
+ # repaired by dart_definition_end_point at ingestion.
+ call_node_types=cs.SPEC_DART_CALL_TYPES,
+ call_query=cs.DART_CALL_QUERY,
+ import_node_types=cs.SPEC_DART_IMPORT_TYPES,
+ import_from_node_types=cs.SPEC_DART_IMPORT_TYPES,
+ ),
}
_EXTENSION_TO_SPEC: dict[str, LanguageSpec] = {}
diff --git a/codebase_rag/logs.py b/codebase_rag/logs.py
index 3e075c877..f676b6974 100644
--- a/codebase_rag/logs.py
+++ b/codebase_rag/logs.py
@@ -1,34 +1,88 @@
-# (H) Provider logs
+# Provider logs
PROVIDER_REGISTERED = "Registered provider: {name}"
-# (H) Graph loading logs
+# Graph loading logs
LOADING_GRAPH = "Loading graph from {path}"
LOADED_GRAPH = "Loaded {nodes} nodes and {relationships} relationships with indexes"
ENSURING_PROJECT = "Ensuring Project: {name}"
-# (H) Pass logs
+# Pass logs
PASS_1_STRUCTURE = "--- Pass 1: Identifying Packages and Folders ---"
PASS_2_FILES = (
"\n--- Pass 2: Processing Files, Caching ASTs, and Collecting Definitions ---"
)
PASS_3_CALLS = "--- Pass 3: Processing Function Calls from AST Cache ---"
PASS_4_EMBEDDINGS = "--- Pass 4: Generating semantic embeddings ---"
+CPP_FRONTEND_RUNNING = "--- C/C++ libclang frontend: {path} ---"
+CPP_FRONTEND_UNAVAILABLE = (
+ "C/C++ libclang frontend enabled but libclang is unavailable; using tree-sitter"
+)
+CPP_FRONTEND_NO_COMPDB = (
+ "C/C++ libclang frontend enabled but no compile_commands.json found; "
+ "using tree-sitter"
+)
+CPP_FRONTEND_COVERED = "C/C++ libclang frontend covered {count} file(s)"
+CPP_FRONTEND_HYBRID_PENDING = (
+ "C/C++ hybrid frontend queued {count} macro use(s) for span attribution"
+)
+CPP_FRONTEND_MACRO_CALLS = "Resolved {count} hybrid macro CALLS edge(s)"
+CPP_FRONTEND_EXPANSION_CALLS = "Resolved {count} hybrid expansion CALLS edge(s)"
+CSHARP_FRONTEND_RUNNING = "--- C# Roslyn frontend: {path} ---"
+CSHARP_FRONTEND_UNAVAILABLE = (
+ "C# Roslyn frontend enabled but dotnet is unavailable; using tree-sitter"
+)
+CSHARP_FRONTEND_AUTO_FALLBACK = (
+ "C# frontend AUTO: dotnet not found; using tree-sitter only"
+)
+CSHARP_FRONTEND_NO_PROJECT = (
+ "C# Roslyn frontend enabled but no .csproj/.sln found; using tree-sitter"
+)
+CSHARP_FRONTEND_BUILD_FAILED = (
+ "C# Roslyn frontend tool failed to build; using tree-sitter"
+)
+CSHARP_FRONTEND_TYPES = "C# Roslyn frontend classified {count} type base list(s)"
+CSHARP_FRONTEND_FACTS = (
+ "C# Roslyn frontend facts: {calls} call site(s), {partials} partial group(s), "
+ "{queries} query call(s), {externals} external site(s)"
+)
+CSHARP_FRONTEND_PARTIALS_JOINED = (
+ "C# Roslyn frontend merged {count} partial-type group(s)"
+)
+CSHARP_FRONTEND_QUERY_EDGES = (
+ "C# Roslyn frontend emitted {count} LINQ query CALLS edge(s)"
+)
+CSHARP_FRONTEND_PARSE_FAILED = (
+ "C# Roslyn frontend produced no parseable JSON; using tree-sitter.\n"
+ "stdout: {stdout}\nstderr: {stderr}"
+)
+CSHARP_FRONTEND_RUN_FAILED = (
+ "C# Roslyn frontend tool did not finish ({error}); using tree-sitter"
+)
+CSHARP_FRONTEND_NO_FACTS = (
+ "C# Roslyn frontend produced no facts; every join falls back to "
+ "tree-sitter. Tool diagnostics:\n{stderr}"
+)
+GRAPH_ALREADY_IN_SYNC = (
+ "Knowledge graph already in sync (hash cache matches every file). Skipping passes."
+)
-# (H) Analysis logs
+# Analysis logs
FOUND_FUNCTIONS = "\n--- Found {count} functions/methods in codebase ---"
+REGISTRY_REHYDRATED = "Rehydrated {count} definitions from the graph for resolution"
+INCREMENTAL_REBUILD_INBOUND = "Rebuilding inbound edges from {count} dependent files"
ANALYSIS_COMPLETE = "\n--- Analysis complete. Flushing all data to database... ---"
REMOVING_STATE = "Removing in-memory state for: {path}"
REMOVED_FROM_CACHE = " - Removed from ast_cache"
REMOVING_QNS = " - Removing {count} QNs from function_registry"
CLEANED_SIMPLE_NAME = " - Cleaned simple_name '{name}'"
-# (H) Function ingest logs
+# Function ingest logs
FUNC_FOUND = " Found Function: {name} (qn: {qn})"
FUNC_EXPECTED_NODE = "Expected Node but got {actual_type}: {value}"
METHOD_FOUND = " Found Method: {name} (qn: {qn})"
EXPORT_FOUND = " Found {export_type}: {name} (qn: {qn})"
-# (H) Definition processor logs
+# Definition processor logs
DEF_PARSING_AST = "Parsing and Caching AST for {language}: {path}"
DEF_UNSUPPORTED_LANGUAGE = "Unsupported language '{language}' for {path}"
DEF_NO_PARSER = "No parser available for {language}"
@@ -36,25 +90,75 @@
DEF_PARSING_DEPENDENCY = " Parsing dependency file: {path}"
DEF_FOUND_DEPENDENCY = " Found dependency: {name} (spec: {spec})"
-# (H) Semantic/embedding logs
+# Semantic/embedding logs
SEMANTIC_NOT_AVAILABLE = (
"Semantic search dependencies not available, skipping embedding generation"
)
INGESTOR_NO_QUERY = "Ingestor does not support querying, skipping embedding generation"
+EMBEDDINGS_SKIPPED = (
+ "Skipping semantic embedding generation (--no-embeddings / CGR_SKIP_EMBEDDINGS)"
+)
+EMBEDDING_DEVICE_UNAVAILABLE = (
+ "Requested embedding device '{device}' is unavailable, falling back to"
+ " automatic selection"
+)
NO_FUNCTIONS_FOR_EMBEDDING = "No functions or methods found for embedding generation"
GENERATING_EMBEDDINGS = "Generating embeddings for {count} functions/methods"
EMBEDDING_PROGRESS = "Generated {done}/{total} embeddings"
EMBEDDING_FAILED = "Failed to embed {name}: {error}"
+EMBEDDING_SNIPPET_FAILED = (
+ "Failed to generate embedding for code snippet of length {length}"
+)
+EMBEDDING_BATCH_COMPUTE_FAILED = "Failed to embed batch of {count}: {error}"
+CONTEXT_TOKEN_COUNT_FAILED = "Context token count failed: {error}"
NO_SOURCE_FOR = "No source code found for {name}"
EMBEDDINGS_COMPLETE = "Successfully generated {count} semantic embeddings"
EMBEDDING_GENERATION_FAILED = "Failed to generate semantic embeddings: {error}"
EMBEDDING_STORE_FAILED = "Failed to store embedding for {name}: {error}"
+EMBEDDING_STORE_RETRY = "Vector store upsert failed (attempt {attempt}/{max_attempts}), retrying in {delay:.1f}s: {error}"
+EMBEDDING_BATCH_STORED = "Stored batch of {count} embeddings in vector store"
+EMBEDDING_BATCH_FAILED = "Failed to store embedding batch: {error}"
EMBEDDING_SEARCH_FAILED = "Failed to search embeddings: {error}"
-
-# (H) Image logs
-IMAGE_COPIED = "Copied image to temporary path: {path}"
-
-# (H) Protobuf service logs
+EMBEDDING_RECONCILE_OK = (
+ "Vector store reconciliation: all {count} expected embeddings found"
+)
+EMBEDDING_RECONCILE_MISSING = "Vector store reconciliation: {missing} of {expected} embeddings missing (IDs: {sample_ids})"
+EMBEDDING_RECONCILE_FAILED = "Vector store reconciliation check failed: {error}"
+VECTOR_STORE_BACKEND_UNAVAILABLE = (
+ "Vector store backend '{backend}' dependencies are not available"
+)
+VECTOR_STORE_BACKEND_UNKNOWN = "Unknown vector store backend '{backend}'"
+VECTOR_STORE_DELETE_PROJECT = (
+ "Deleting {count} {backend} vectors for project '{project}'"
+)
+VECTOR_STORE_DELETE_PROJECT_DONE = "Deleted {backend} vectors for project '{project}'"
+VECTOR_STORE_DELETE_PROJECT_FAILED = (
+ "Failed to delete {backend} vectors for project '{project}': {error}"
+)
+VECTOR_STORE_CLEARED = "Cleared all {backend} vectors"
+QDRANT_DELETE_PROJECT = "Deleting {count} Qdrant vectors for project '{project}'"
+QDRANT_DELETE_PROJECT_DONE = "Deleted Qdrant vectors for project '{project}'"
+QDRANT_DELETE_PROJECT_FAILED = (
+ "Failed to delete Qdrant vectors for project '{project}': {error}"
+)
+QDRANT_LOCK_ERROR = (
+ "Failed to open embedded Qdrant at '{path}': {error}. The storage folder is "
+ "locked by another process; look for the '.lock' sentinel inside it. Embedded "
+ "Qdrant allows only one process at a time, so a running MCP server and a CLI "
+ "indexing run cannot share it. Set QDRANT_URL to point at a shared Qdrant "
+ "server for concurrent access."
+)
+EMBEDDING_CACHE_HIT = "Embedding cache hit for {count} snippets"
+EMBEDDING_CACHE_LOADED = "Loaded embedding cache with {count} entries from {path}"
+EMBEDDING_CACHE_SAVE_FAILED = "Failed to save embedding cache to {path}: {error}"
+EMBEDDING_CACHE_LOAD_FAILED = "Failed to load embedding cache from {path}: {error}"
+
+# Multimodal attachment logs
+MULTIMODAL_ATTACHED = "Attached multimodal content: {path}"
+MULTIMODAL_NOT_FOUND = "Multimodal path referenced but not found: {path}"
+MULTIMODAL_READ_FAILED = "Failed to read multimodal file '{path}': {error}"
+
+# Protobuf service logs
PROTOBUF_INIT = "ProtobufFileIngestor initialized to write to: {path}"
PROTOBUF_NO_MESSAGE_CLASS = (
"No Protobuf message class found for label '{label}'. Skipping node."
@@ -71,7 +175,7 @@
PROTOBUF_FLUSH_SUCCESS = "Successfully flushed {nodes} unique nodes and {rels} unique relationships to {path}"
PROTOBUF_FLUSHING = "Flushing data to {path}..."
-# (H) Parser loader logs
+# Parser loader logs
BUILDING_BINDINGS = "Building Python bindings for {lang}..."
BUILD_FAILED = "Failed to build {lang} bindings: stdout={stdout}, stderr={stderr}"
BUILD_SUCCESS = "Successfully built {lang} bindings"
@@ -87,18 +191,38 @@
LOCALS_QUERY_FAILED = "Failed to create locals query for {lang}: {error}"
GRAMMAR_LOADED = "Successfully loaded {lang} grammar."
GRAMMAR_LOAD_FAILED = "Failed to load {lang} grammar: {error}"
-INITIALIZED_PARSERS = "Initialized parsers for: {languages}"
+PARSERS_LAZY_READY = "Parser registry ready; grammars load on first use."
-# (H) Ignore pattern logs
+# Ignore pattern logs
CGRIGNORE_LOADED = (
"Loaded {exclude_count} exclude and {unignore_count} unignore patterns from {path}"
)
CGRIGNORE_READ_FAILED = "Failed to read {path}: {error}"
-# (H) File watcher logs
+CGR_INSTRUCTIONS_LOADED = "Loaded project instructions from {path} ({chars} chars)"
+CGR_INSTRUCTIONS_READ_FAILED = "Failed to read project instructions {path}: {error}"
+
+# File watcher logs
WATCHER_ACTIVE = "File watcher is now active."
+WATCHER_DEBOUNCE_ACTIVE = (
+ "File watcher active with debouncing (debounce={debounce}s, max_wait={max_wait}s)"
+)
WATCHER_SKIP_NO_QUERY = "Ingestor does not support querying, skipping real-time update."
CHANGE_DETECTED = "Change detected: {event_type} on {path}. Updating graph."
+CHANGE_DEBOUNCING = (
+ "Change detected: {event_type} on {name} (debouncing for {debounce}s)"
+)
+DEBOUNCE_RESET = "Reset debounce timer for {path}"
+DEBOUNCE_MAX_WAIT = "Max wait ({max_wait}s) exceeded for {path}, processing now"
+DEBOUNCE_SCHEDULED = (
+ "Scheduled update for {path} in {debounce}s (max wait: {remaining}s remaining)"
+)
+DEBOUNCE_PROCESSING = "Processing debounced change: {path}"
+DEBOUNCE_NO_EVENT = "No pending event for {path}, skipping"
+DEBOUNCE_MAX_WAIT_ADJUSTED = (
+ "max_wait ({max_wait}s) is less than debounce ({debounce}s). "
+ "Setting max_wait to debounce value."
+)
DELETION_QUERY = "Ran deletion query for path: {path}"
RECALC_CALLS = "Recalculating all function call relationships for consistency..."
GRAPH_UPDATED = "Graph updated successfully for change in: {name}"
@@ -107,7 +231,7 @@
WATCHING = "Watching for changes in: {path}"
LOGGER_CONFIGURED = "Logger configured for Real-Time Updater."
-# (H) Build logs
+# Build logs
BUILD_BINARY = "Building binary: {name}"
BUILD_PROGRESS = "This may take a few minutes..."
BUILD_READY = "Binary is ready for distribution!"
@@ -116,13 +240,7 @@
BUILD_STDOUT = "STDOUT: {stdout}"
BUILD_STDERR = "STDERR: {stderr}"
-# (H) No-docs check logs
-NO_DOCS_VIOLATIONS_FOUND = (
- "No-docs violations found (module docstrings or inline comments):"
-)
-NO_DOCS_ERROR = " {error}"
-
-# (H) Graph summary logs
+# Graph summary logs
GRAPH_SUMMARY = "Graph Summary:"
GRAPH_TOTAL_NODES = " Total nodes: {count:,}"
GRAPH_TOTAL_RELS = " Total relationships: {count:,}"
@@ -140,22 +258,23 @@
GRAPH_ANALYSIS_ERROR = "Error analyzing graph: {error}"
GRAPH_FILE_NOT_FOUND = "Graph file not found: {path}"
-# (H) FQN logs
+# FQN logs
FQN_RESOLVE_FAILED = "Failed to resolve FQN for node at {path}: {error}"
FQN_FIND_FAILED = "Failed to find function by FQN {fqn} in {path}: {error}"
FQN_EXTRACT_FAILED = "Failed to extract function FQNs from {path}: {error}"
-# (H) Source extraction logs
+# Source extraction logs
SOURCE_FILE_NOT_FOUND = "Source file not found: {path}"
SOURCE_INVALID_RANGE = "Invalid line range: {start}-{end}"
SOURCE_RANGE_EXCEEDS = "Line range {start}-{end} exceeds file length {length} in {path}"
SOURCE_EXTRACT_FAILED = "Failed to extract source from {path}: {error}"
SOURCE_AST_FAILED = "AST extraction failed for {name}: {error}"
-# (H) Memgraph logs
+# Memgraph logs
MG_CONNECTING = "Connecting to Memgraph at {host}:{port}..."
MG_CONNECTED = "Successfully connected to Memgraph."
-MG_EXCEPTION = "An exception occurred: {error}. Flushing remaining items..."
+MG_EXCEPTION = "An exception occurred: {error}. Attempting best-effort flush..."
+MG_FLUSH_ERROR = "Failed to flush during cleanup: {error}"
MG_DISCONNECTED = "\nDisconnected from Memgraph."
MG_CYPHER_ERROR = "!!! Cypher Error: {error}"
MG_CYPHER_QUERY = " Query: {query}"
@@ -168,6 +287,11 @@
MG_PROJECT_DELETED = "--- Project {project_name} deleted. ---"
MG_ENSURING_CONSTRAINTS = "Ensuring constraints..."
MG_CONSTRAINTS_DONE = "Constraints checked/created."
+MG_LEGACY_PURGE = (
+ "Purged {count} Folder/File node(s) written by the superseded "
+ "relative-path key (issue #897); re-run with --update-graph for "
+ "affected projects to rebuild their containment."
+)
MG_ENSURING_INDEXES = "Ensuring label-property indexes for MERGE performance..."
MG_INDEXES_DONE = "Indexes checked/created."
MG_NODE_BUFFER_FLUSH = (
@@ -177,7 +301,9 @@
"Relationship buffer reached batch size ({size}). Performing incremental flush."
)
MG_NO_CONSTRAINT = "No unique constraint defined for label '{label}'. Skipping flush."
-MG_MISSING_PROP = "Skipping {label} node missing required '{key}' property: {props}"
+MG_MISSING_PROP = (
+ "Skipping {label} node missing required '{key}' property (keys: {prop_keys})"
+)
MG_NODES_FLUSHED = "Flushed {flushed} of {total} buffered nodes."
MG_NODES_SKIPPED = (
"Skipped {count} buffered nodes due to missing identifiers or constraints."
@@ -189,17 +315,29 @@
)
MG_FLUSH_START = "--- Flushing all pending writes to database... ---"
MG_FLUSH_COMPLETE = "--- Flushing complete. ---"
+MG_PARALLEL_FLUSH_NODES = (
+ "Parallel flushing {count} label groups with {workers} workers"
+)
+MG_PARALLEL_FLUSH_RELS = (
+ "Parallel flushing {count} relationship groups with {workers} workers"
+)
+MG_LABEL_FLUSH_ERROR = "Error flushing label group '{label}': {error}"
+MG_REL_FLUSH_ERROR = "Error flushing relationship group '{pattern}': {error}"
+MG_NO_CONN_NODES = "No database connection for label '{label}', skipping flush."
+MG_NO_CONN_RELS = (
+ "No database connection for relationship group '{pattern}', skipping flush."
+)
MG_FETCH_QUERY = "Executing fetch query: {query} with params: {params}"
MG_WRITE_QUERY = "Executing write query: {query} with params: {params}"
MG_EXPORTING = "Exporting graph data..."
MG_EXPORTED = "Exported {nodes} nodes and {rels} relationships"
-# (H) LLM/Cypher logs
+# LLM/Cypher logs
CYPHER_GENERATING = " [CypherGenerator] Generating query for: '{query}'"
CYPHER_GENERATED = " [CypherGenerator] Generated Cypher: {query}"
CYPHER_ERROR = " [CypherGenerator] Error: {error}"
-# (H) Tool file logs
+# Tool file logs
TOOL_FILE_READ = "[FileReader] Attempting to read file: {path}"
TOOL_FILE_READ_SUCCESS = "[FileReader] Successfully read text from {path}"
TOOL_FILE_BINARY = "[FileReader] {message}"
@@ -215,6 +353,13 @@
)
TOOL_QUERY_RECEIVED = "[Tool:QueryGraph] Received NL query: '{query}'"
TOOL_QUERY_ERROR = "[Tool:QueryGraph] Error during query execution: {error}"
+TOOL_QUERY_TIMEOUT = (
+ "[Tool:QueryGraph] Query exceeded {timeout:.1f}s and was cancelled: {query}"
+)
+QUERY_RESULTS_TRUNCATED = (
+ "[Tool:QueryGraph] Results truncated: showing {kept} of {total} rows "
+ "({tokens} tokens, limit {max_tokens})"
+)
TOOL_SHELL_EXEC = "Executing shell command: {cmd}"
TOOL_SHELL_RETURN = "Return code: {code}"
TOOL_SHELL_STDOUT = "Stdout: {stdout}"
@@ -224,15 +369,14 @@
"Process already terminated when timeout kill was attempted."
)
TOOL_SHELL_ERROR = "An error occurred while executing command: {error}"
-TOOL_DOC_ANALYZE = "[DocumentAnalyzer] Analyzing '{path}' with question: '{question}'"
-# (H) Shell timing log
+# Shell timing log
SHELL_TIMING = "'{func}' executed in {time:.2f}ms"
-# (H) Generic function timing log
+# Generic function timing log
FUNC_TIMING = "{func} completed in {time:.2f}ms"
-# (H) File editor logs
+# File editor logs
EDITOR_NO_PARSER = "No parser available for {path}"
EDITOR_NO_LANG_CONFIG = "No language config found for extension {ext}"
EDITOR_FUNC_NOT_FOUND_AT_LINE = "No function '{name}' found at line {line}"
@@ -262,11 +406,11 @@
EDITOR_SURGICAL_FAILED = "Surgical patches failed to apply cleanly"
EDITOR_SURGICAL_ERROR = "Error during surgical block replacement: {error}"
-# (H) Directory lister logs
+# Directory lister logs
DIR_LISTING = "Listing contents of directory: {path}"
DIR_LIST_ERROR = "Error listing directory {path}: {error}"
-# (H) Semantic search logs
+# Semantic search logs
SEMANTIC_NO_MATCH = "No semantic matches found for query: {query}"
SEMANTIC_FOUND = "Found {count} semantic matches for: {query}"
SEMANTIC_FAILED = "Semantic search failed for query '{query}': {error}"
@@ -276,54 +420,45 @@
SEMANTIC_TOOL_SEARCH = "[Tool:SemanticSearch] Searching for: '{query}'"
SEMANTIC_TOOL_SOURCE = "[Tool:GetFunctionSource] Retrieving source for node ID: {id}"
-# (H) Document analyzer logs
-DOC_COPIED = "Copied external file to: {path}"
-DOC_SUCCESS = "Successfully received analysis for '{path}'."
-DOC_NO_TEXT = "No text found in response: {response}"
-DOC_API_ERROR = "Google GenAI API error for '{path}': {error}"
-DOC_FAILED = "Failed to analyze document '{path}': {error}"
-DOC_RESULT = "[analyze_document] Result type: {type}, content: {preview}..."
-DOC_EXCEPTION = "[analyze_document] Exception during analysis: {error}"
-
-# (H) Code retrieval logs
+# Code retrieval logs
CODE_RETRIEVER_INIT = "CodeRetriever initialized with root: {root}"
CODE_RETRIEVER_SEARCH = "[CodeRetriever] Searching for: {name}"
CODE_RETRIEVER_ERROR = "[CodeRetriever] Error: {error}"
CODE_TOOL_RETRIEVE = "[Tool:GetCode] Retrieving code for: {name}"
-# (H) Tool init logs
+# Tool init logs
FILE_EDITOR_INIT = "FileEditor initialized with root: {root}"
FILE_READER_INIT = "FileReader initialized with root: {root}"
SHELL_COMMANDER_INIT = "ShellCommander initialized with root: {root}"
-DOC_ANALYZER_INIT = "DocumentAnalyzer initialized with root: {root}"
-# (H) Tool error logs
+# Tool error logs
FILE_EDITOR_WARN = "[FileEditor] {msg}"
FILE_EDITOR_ERR = "[FileEditor] {msg}"
FILE_EDITOR_ERR_EDIT = "[FileEditor] Error editing file {path}: {error}"
FILE_READER_ERR = "Error reading file {path}: {error}"
-DOC_ANALYZER_API_ERR = "[DocumentAnalyzer] API validation error: {error}"
-# (H) File writer logs
+# File writer logs
FILE_WRITER_INIT = "FileWriter initialized with root: {root}"
FILE_WRITER_CREATE = "[FileWriter] Creating file: {path}"
FILE_WRITER_SUCCESS = "[FileWriter] Successfully wrote {chars} characters to {path}"
-# (H) Error logs (used with logger.error/warning)
+# Error logs (used with logger.error/warning)
UNEXPECTED = "An unexpected error occurred: {error}"
EXPORT_ERROR = "Export error: {error}"
+STATS_ERROR = "Stats error: {error}"
+DEADCODE_SCANNING = "Scanning project '{project_name}' for dead code"
+DEADCODE_ERROR = "Dead code scan error: {error}"
INDEXING_FAILED = "Indexing failed"
PATH_NOT_IN_QUESTION = (
- "Could not find original path in question for replacement: {path}"
+ "Could not locate path token in user message for attachment: {path}"
)
-IMAGE_NOT_FOUND = "Image path found, but does not exist: {path}"
-IMAGE_COPY_FAILED = "Failed to copy image to temporary directory: {error}"
FILE_OUTSIDE_ROOT = "Security risk: Attempted to {action} file outside of project root."
-# (H) Call processor logs
+# Call processor logs
CALL_PROCESSING_FILE = "Processing calls in cached AST for: {path}"
CALL_PROCESSING_FAILED = "Failed to process calls in {path}: {error}"
CALL_FOUND_NODES = "Found {count} call nodes in {language} for {caller}"
+CALL_SKIP_CLASS = "Skipping CALLS edge from {caller} to {call_name} (callee is Class node: {callee_qn})"
CALL_FOUND = (
"Found call from {caller} to {call_name} (resolved as {callee_type}:{callee_qn})"
)
@@ -350,6 +485,7 @@
CALL_WILDCARD = "Wildcard-resolved call: {call_name} -> {qn}"
CALL_SAME_MODULE = "Same-module resolution: {call_name} -> {qn}"
CALL_TRIE_FALLBACK = "Trie-based fallback resolution: {call_name} -> {qn}"
+CALL_PACKAGE_MEMBER = "Package-member resolved call: {member} -> {qn}"
CALL_UNRESOLVED = "Could not resolve call: {call_name}"
CALL_CHAINED = (
"Resolved chained call: {call_name} -> {method_qn} (via {obj_expr}:{obj_type})"
@@ -367,7 +503,7 @@
"Unexpected parent type for node {node}: {parent_type}. Skipping."
)
-# (H) Dependency parser logs
+# Dependency parser logs
DEP_PARSE_ERROR_PYPROJECT = "Error parsing pyproject.toml {path}: {error}"
DEP_PARSE_ERROR_REQUIREMENTS = "Error parsing requirements.txt {path}: {error}"
DEP_PARSE_ERROR_PACKAGE_JSON = "Error parsing package.json {path}: {error}"
@@ -376,8 +512,9 @@
DEP_PARSE_ERROR_GEMFILE = "Error parsing Gemfile {path}: {error}"
DEP_PARSE_ERROR_COMPOSER = "Error parsing composer.json {path}: {error}"
DEP_PARSE_ERROR_CSPROJ = "Error parsing .csproj {path}: {error}"
+DEP_PARSE_ERROR_PUBSPEC = "Error parsing pubspec.yaml {path}: {error}"
-# (H) Import processor logs
+# Import processor logs
IMP_TOOL_NOT_AVAILABLE = "External tool '{tool}' not available for stdlib introspection"
IMP_CACHE_LOADED = "Loaded stdlib cache from {path}"
IMP_CACHE_LOAD_ERROR = "Could not load stdlib cache: {error}"
@@ -390,6 +527,10 @@
" Created IMPORTS relationship: {from_module} -> {to_module} (from {full_name})"
)
IMP_PARSE_FAILED = "Failed to parse imports in {module}: {error}"
+IMP_DROPPED_PHANTOM_TARGET = (
+ " Dropped IMPORTS edge to unverifiable internal target: "
+ "{from_module} -> {to_module}"
+)
IMP_IMPORT = " Import: {local} -> {full}"
IMP_ALIASED_IMPORT = " Aliased import: {alias} -> {full}"
IMP_WILDCARD_IMPORT = " Wildcard import: * -> {module}"
@@ -404,19 +545,22 @@
IMP_JAVA_STATIC = "Java static import: {name} -> {path}"
IMP_JAVA_IMPORT = "Java import: {name} -> {path}"
IMP_RUST = "Rust import: {name} -> {path}"
+IMP_CSHARP = "C# using: {name} -> {path}"
IMP_GO = "Go import: {package} -> {path}"
+IMP_GO_MODULE_PATH = "Go module path: {module} -> {path}"
IMP_CPP_INCLUDE = "C++ include: {local} -> {full} (system: {system})"
IMP_CPP_MODULE = "C++20 module import: {local} -> {full}"
IMP_CPP_MODULE_IMPL = "C++20 module implementation: {name}"
IMP_CPP_MODULE_IFACE = "C++20 module interface: {name}"
IMP_CPP_PARTITION = "C++20 module partition import: {partition} -> {full}"
IMP_GENERIC = "Generic import parsing for {language}: {node_type}"
+IMP_PY_SOURCE_ROOT = "Python source root: {name} -> {path}"
-# (H) Structure processor logs
+# Structure processor logs
STRUCT_IDENTIFIED_PACKAGE = " Identified Package: {package_qn}"
STRUCT_IDENTIFIED_FOLDER = " Identified Folder: '{relative_root}'"
-# (H) Class ingest logs
+# Class ingest logs
CLASS_CPP_MODULE_INTERFACE = " Found C++ Module Interface: {qn}"
CLASS_CPP_MODULE_IMPL = " Found C++ Module Implementation: {qn}"
CLASS_FOUND_INTERFACE = " Found Interface: {name} (qn: {qn})"
@@ -435,7 +579,7 @@
CLASS_METHOD_OVERRIDE = "Method override: {method_qn} OVERRIDES {parent_method_qn}"
CLASS_CPP_INHERITANCE = "Found C++ inheritance: {parent_name} -> {parent_qn}"
-# (H) Java type inference logs
+# Java type inference logs
JAVA_VAR_TYPE_MAP_BUILT = "Built Java variable type map with {count} entries"
JAVA_VAR_TYPE_MAP_FAILED = "Failed to build Java variable type map: {error}"
JAVA_PARAM = "Parameter: {name} -> {type}"
@@ -457,7 +601,7 @@
JAVA_ENHANCED_FOR_VAR = "Enhanced for loop variable: {name} -> {type}"
JAVA_ENHANCED_FOR_VAR_ALT = "Enhanced for loop variable (alt): {name} -> {type}"
-# (H) JS type inference logs
+# JS type inference logs
JS_VAR_DECLARATOR_FOUND = "Found variable declarator: {var_name} in {module_qn}"
JS_VAR_INFERRED = "Inferred JS variable: {var_name} -> {var_type}"
JS_VAR_INFER_FAILED = "Could not infer type for variable: {var_name}"
@@ -484,14 +628,14 @@
"Error inferring JS method return type for {method_call}: {error}"
)
-# (H) Lua type inference logs
+# Lua type inference logs
LUA_VAR_TYPE_MAP_BUILT = "Built Lua variable type map with {count} variables"
LUA_VAR_INFERRED = "Inferred Lua variable: {var_name} -> {var_type}"
LUA_TYPE_INFERENCE_RETURN = (
"Lua type inference: {class_name}:{method_name}() returns {class_qn}"
)
-# (H) Python type inference logs
+# Python type inference logs
PY_BUILD_VAR_MAP_FAILED = "Failed to build local variable type map: {error}"
PY_PARAM_TYPE_INFERRED = "Inferred parameter type: {param} -> {type}"
PY_TYPE_INFER_ATTEMPT = (
@@ -526,7 +670,7 @@
PY_FOUND_INIT_METHOD = " Found __init__ method!"
PY_INIT_NOT_FOUND = " No __init__ method found in class body"
-# (H) JS/TS ingest logs
+# JS/TS ingest logs
JS_PROTOTYPE_INHERITANCE = "Prototype inheritance: {child_qn} INHERITS {parent_qn}"
JS_PROTOTYPE_INHERITANCE_FAILED = "Failed to detect prototype inheritance: {error}"
JS_PROTOTYPE_METHOD_FOUND = " Found Prototype Method: {method_name} (qn: {method_qn})"
@@ -551,7 +695,7 @@
"Failed to detect assignment arrow functions: {error}"
)
-# (H) JS/TS module system logs
+# JS/TS module system logs
JS_COMMONJS_DESTRUCTURE_FAILED = (
"Failed to process CommonJS destructuring pattern: {error}"
)
@@ -568,7 +712,7 @@
JS_ES6_EXPORTS_QUERY_FAILED = "Failed to process ES6 exports query: {error}"
JS_ES6_EXPORTS_DETECT_FAILED = "Failed to detect ES6 exports: {error}"
-# (H) MCP tool logs
+# MCP tool logs
MCP_INDEXING_REPO = "[MCP] Indexing repository at: {path}"
MCP_CLEARING_DB = "[MCP] Clearing existing database to avoid conflicts..."
MCP_DB_CLEARED = "[MCP] Database cleared. Starting fresh indexing..."
@@ -593,8 +737,16 @@
MCP_ERROR_WRITE = "[MCP] Error writing file: {error}"
MCP_LIST_DIR = "[MCP] list_directory: {path}"
MCP_ERROR_LIST_DIR = "[MCP] Error listing directory: {error}"
+MCP_SEMANTIC_NOT_AVAILABLE = (
+ "[MCP] Semantic search not available. Install with: uv sync --extra semantic"
+)
+MCP_UPDATING_REPO = "[MCP] Updating repository at: {path}"
+MCP_ERROR_UPDATING = "[MCP] Error updating repository: {error}"
+MCP_SEMANTIC_SEARCH = "[MCP] semantic_search: {query}"
+MCP_ASK_AGENT = "[MCP] ask_agent: {question}"
+MCP_ASK_AGENT_ERROR = "[MCP] Error running ask_agent: {error}"
-# (H) MCP server logs
+# MCP server logs
MCP_SERVER_INFERRED_ROOT = "[GraphCode MCP] Using inferred project root: {path}"
MCP_SERVER_NO_ROOT = (
"[GraphCode MCP] No project root configured, using current directory: {path}"
@@ -612,12 +764,79 @@
MCP_SERVER_CONNECTED = "[GraphCode MCP] Connected to Memgraph at {host}:{port}"
MCP_SERVER_FATAL_ERROR = "[GraphCode MCP] Fatal error: {error}"
MCP_SERVER_SHUTDOWN = "[GraphCode MCP] Shutting down server..."
-
-# (H) Exclude prompt logs
+MCP_HTTP_SERVER_STARTING = "[GraphCode MCP] Starting HTTP server on {host}:{port}..."
+MCP_HTTP_EXPOSURE_REFUSED = (
+ "Refusing to bind the HTTP MCP server to {host}: the endpoint has no "
+ "authentication unless MCP_HTTP_AUTH_TOKEN is set. Configure a token to "
+ "expose it beyond loopback, or bind to 127.0.0.1."
+)
+MCP_HTTP_SERVER_READY = (
+ "[GraphCode MCP] HTTP server ready. MCP endpoint: http://{host}:{port}/mcp"
+)
+
+# Incremental update logs
+HASH_CACHE_LOADED = "Loaded hash cache with {count} entries from {path}"
+HASH_CACHE_LOAD_FAILED = "Failed to load hash cache from {path}: {error}"
+HASH_CACHE_SAVED = "Saved hash cache with {count} entries to {path}"
+HASH_CACHE_SAVE_FAILED = "Failed to save hash cache to {path}: {error}"
+PERIODIC_FLUSH = "Periodic flush after {count} files processed"
+INCREMENTAL_SKIPPED = "Skipped {count} unchanged files"
+INCREMENTAL_CHANGED = "Re-indexing {count} changed files"
+INCREMENTAL_DELETED = "Removed state for {count} deleted files"
+INCREMENTAL_FORCE = "Force mode enabled, bypassing hash cache"
+HASH_CACHE_ORPHANED = (
+ "Hash cache exists but project '{project}' has no modules in the graph; "
+ "the database was likely wiped since the last sync. Discarding the cache "
+ "and rebuilding fully."
+)
+PARSER_FINGERPRINT_SAVE_FAILED = "Failed to save parser fingerprint to {path}: {error}"
+PARSER_FINGERPRINT_MISMATCH = (
+ "Parser code changed since this graph was built. Incremental sync keeps "
+ "results from the old parser for unchanged files, so the graph may be "
+ "stale. Run 'cgr start --clean' to rebuild it from scratch."
+)
+
+REHYDRATE_QUERY_FAILED = (
+ "Could not read persisted definitions from the graph; continuing with "
+ "only this run's freshly parsed registry."
+)
+INBOUND_CAPTURE_FAILED = (
+ "Could not read inbound edges from the graph; this full rebuild "
+ "re-parses every caller, so the edges are re-resolved from source."
+)
+
+# Orphan pruning logs
+PRUNE_START = "--- Pruning orphan nodes from graph ---"
+PRUNE_QUERY_FAILED = "Could not read {label} paths from the graph; skipping its prune."
+PRUNE_FOUND = "Found {count} orphan {label} nodes to remove"
+PRUNE_DELETING = "Pruning orphan {label}: {path}"
+PRUNE_COMPLETE = "Pruning complete. Removed {count} orphan nodes."
+PRUNE_SKIP = "No orphan nodes found. Graph is clean."
+FILE_HASH_UNCHANGED = "File unchanged (hash match): {path}"
+FILE_HASH_CHANGED = "File changed (hash mismatch): {path}"
+FILE_HASH_NEW = "New file detected: {path}"
+FILE_UNREADABLE = (
+ "Skipping unreadable file (broken symlink or removed): {path} ({error})"
+)
+INCREMENTAL_UNREADABLE = "Skipped {count} unreadable files (broken symlinks or removed)"
+
+# Exclude prompt logs
EXCLUDE_INVALID_INDEX = "Invalid index: {index} (out of range)"
EXCLUDE_INVALID_INPUT = "Invalid input: '{input}' (expected number)"
-# (H) Model switching logs
+# Model switching logs
MODEL_SWITCHED = "Model switched to: {model}"
MODEL_SWITCH_FAILED = "Failed to switch model: {error}"
MODEL_CURRENT = "Current model: {model}"
+
+# Progress bar logs
+PROGRESS_INDEXING_LABEL = "[bold blue]Indexing files..."
+PROGRESS_FILES_PROCESSED = "{count} processed"
+
+# Capture selection logs
+CAPTURE_UNKNOWN_TOKEN = "Ignoring unknown capture token: {token}"
+CAPTURE_DEPENDENCY_GAP = (
+ "Capture selection keeps {rel} but its usual companion {missing} is disabled; "
+ "obeying as requested (edges may be incomplete)"
+)
+CAPTURE_RESOLVED = "Capture enabled: {rels}"
diff --git a/codebase_rag/main.py b/codebase_rag/main.py
index af58a84a4..ce8884844 100644
--- a/codebase_rag/main.py
+++ b/codebase_rag/main.py
@@ -1,45 +1,66 @@
+"""Interactive agent loop: turn natural-language questions into graph queries."""
+
from __future__ import annotations
import asyncio
import difflib
import json
+import mimetypes
import os
import shlex
import shutil
+import subprocess
import sys
import uuid
from collections import deque
-from collections.abc import Coroutine
+from collections.abc import Callable, Coroutine
+from contextlib import contextmanager
from dataclasses import replace
+from decimal import Decimal
+from html import escape as html_escape
from pathlib import Path
from typing import TYPE_CHECKING
from loguru import logger
-from prompt_toolkit import prompt
+from prompt_toolkit import PromptSession, prompt
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.key_binding import KeyBindings
from prompt_toolkit.shortcuts import print_formatted_text
-from pydantic_ai import DeferredToolRequests, DeferredToolResults, ToolDenied
-from rich.markdown import Markdown
+from pydantic_ai import (
+ BinaryContent,
+ DeferredToolRequests,
+ DeferredToolResults,
+ ToolDenied,
+)
+from pydantic_ai.messages import (
+ ModelRequest,
+ ModelResponse,
+ ToolCallPart,
+ ToolReturnPart,
+ UserContent,
+)
+from rich.console import Group
+from rich.live import Live
from rich.panel import Panel
-from rich.prompt import Confirm, Prompt
+from rich.prompt import Prompt
+from rich.spinner import Spinner
from rich.table import Table
from rich.text import Text
from . import constants as cs
from . import exceptions as ex
from . import logs as ls
-from .config import ModelConfig, load_cgrignore_patterns, settings
+from .config import ModelConfig, load_ignore_patterns, settings
from .models import AppContext
from .prompts import OPTIMIZATION_PROMPT, OPTIMIZATION_PROMPT_WITH_REFERENCE
from .providers.base import get_provider_from_config
from .services import QueryProtocol
from .services.graph_service import MemgraphIngestor
from .services.llm import CypherGenerator, create_rag_orchestrator
+from .tools.ast_grep_service import AstGrepService
from .tools.code_retrieval import CodeRetriever, create_code_retrieval_tool
from .tools.codebase_query import create_query_tool
from .tools.directory_lister import DirectoryLister, create_directory_lister_tool
-from .tools.document_analyzer import DocumentAnalyzer, create_document_analyzer_tool
from .tools.file_editor import FileEditor, create_file_editor_tool
from .tools.file_reader import FileReader, create_file_reader_tool
from .tools.file_writer import FileWriter, create_file_writer_tool
@@ -48,6 +69,8 @@
create_semantic_search_tool,
)
from .tools.shell_command import ShellCommander, create_shell_command_tool
+from .tools.structural_editor import create_structural_editor_tool
+from .tools.structural_search import create_structural_search_tool
from .types_defs import (
CHAT_LOOP_UI,
OPTIMIZATION_LOOP_UI,
@@ -57,17 +80,21 @@
ConfirmationToolNames,
CreateFileArgs,
GraphData,
+ QueryJsonOutput,
RawToolArgs,
ReplaceCodeArgs,
ShellCommandArgs,
+ StructuralReplaceArgs,
ToolArgs,
)
+from .utils.rich_markdown import LeftAlignedMarkdown
if TYPE_CHECKING:
from prompt_toolkit.key_binding import KeyPressEvent
from pydantic_ai import Agent
from pydantic_ai.messages import ModelMessage
from pydantic_ai.models import Model
+ from pydantic_ai.usage import RunUsage
def style(
@@ -109,6 +136,50 @@ def get_session_context() -> str:
return ""
+def _autowrap_diff_blocks(text: str) -> str:
+ if cs.DIFF_GIT_HEADER not in text:
+ return text
+ lines = text.split("\n")
+ out: list[str] = []
+ in_fence = False
+ in_diff = False
+
+ def is_diff_continuation(line: str) -> bool:
+ if line == "":
+ return True
+ return line.startswith(cs.DIFF_CONTINUATION_PREFIXES)
+
+ for line in lines:
+ if line.startswith(cs.MARKDOWN_FENCE):
+ if in_diff:
+ out.append(cs.MARKDOWN_FENCE)
+ in_diff = False
+ in_fence = not in_fence
+ out.append(line)
+ continue
+ if in_fence:
+ out.append(line)
+ continue
+ if not in_diff and line.startswith(cs.DIFF_GIT_HEADER):
+ out.append(cs.MARKDOWN_FENCE_DIFF)
+ in_diff = True
+ out.append(line)
+ continue
+ if in_diff:
+ if is_diff_continuation(line):
+ out.append(line)
+ else:
+ out.append(cs.MARKDOWN_FENCE)
+ in_diff = False
+ out.append(line)
+ continue
+ out.append(line)
+
+ if in_diff:
+ out.append(cs.MARKDOWN_FENCE)
+ return "\n".join(out)
+
+
def _print_unified_diff(target: str, replacement: str, path: str) -> None:
separator = dim(cs.HORIZONTAL_SEPARATOR)
app_context.console.print(f"\n{cs.UI_DIFF_FILE_HEADER.format(path=path)}")
@@ -177,6 +248,13 @@ def _to_tool_args(
)
case tool_names.shell_command:
return ShellCommandArgs(command=raw_args.command)
+ case tool_names.structural_replace:
+ return StructuralReplaceArgs(
+ pattern=raw_args.pattern,
+ rewrite=raw_args.rewrite,
+ language=raw_args.language,
+ dry_run=raw_args.dry_run,
+ )
case _:
return ShellCommandArgs()
@@ -208,6 +286,33 @@ def _display_tool_call_diff(
style(f"$ {command}", cs.Color.YELLOW, cs.StyleModifier.NONE)
)
+ case tool_names.structural_replace:
+ pattern = str(tool_args.get(cs.ARG_PATTERN, ""))
+ rewrite = str(tool_args.get(cs.ARG_REWRITE, ""))
+ dry_run = tool_args.get(cs.ARG_DRY_RUN, True)
+ app_context.console.print(f"\n{cs.AST_GREP_APPROVAL_HEADER}")
+ app_context.console.print(
+ style(
+ cs.AST_GREP_APPROVAL_PATTERN.format(pattern=pattern),
+ cs.Color.YELLOW,
+ cs.StyleModifier.NONE,
+ )
+ )
+ app_context.console.print(
+ style(
+ cs.AST_GREP_APPROVAL_REWRITE.format(rewrite=rewrite),
+ cs.Color.YELLOW,
+ cs.StyleModifier.NONE,
+ )
+ )
+ app_context.console.print(
+ style(
+ cs.AST_GREP_APPROVAL_DRY_RUN.format(dry_run=dry_run),
+ cs.Color.YELLOW,
+ cs.StyleModifier.NONE,
+ )
+ )
+
case _:
app_context.console.print(
cs.UI_TOOL_ARGS_FORMAT.format(
@@ -216,7 +321,7 @@ def _display_tool_call_diff(
)
-def _process_tool_approvals(
+async def _process_tool_approvals(
requests: DeferredToolRequests,
approval_prompt: str,
denial_default: str,
@@ -228,30 +333,102 @@ def _process_tool_approvals(
tool_args = _to_tool_args(
call.tool_name, RawToolArgs(**call.args_as_dict()), tool_names
)
- app_context.console.print(
- f"\n{cs.UI_TOOL_APPROVAL.format(tool_name=call.tool_name)}"
+ will_prompt = (
+ app_context.session.confirm_edits and not app_context.session.is_yolo()
)
+
+ if will_prompt:
+ app_context.console.print(
+ f"\n{cs.UI_TOOL_APPROVAL.format(tool_name=call.tool_name)}"
+ )
_display_tool_call_diff(call.tool_name, tool_args, tool_names)
- if app_context.session.confirm_edits:
- if Confirm.ask(style(approval_prompt, cs.Color.CYAN)):
- deferred_results.approvals[call.tool_call_id] = True
- else:
- feedback = Prompt.ask(
- cs.UI_FEEDBACK_PROMPT,
- default="",
- )
- denial_msg = feedback.strip() or denial_default
- deferred_results.approvals[call.tool_call_id] = ToolDenied(denial_msg)
- else:
+ if not will_prompt:
+ deferred_results.approvals[call.tool_call_id] = True
+ continue
+
+ if await _confirm_with_toggle(approval_prompt):
+ deferred_results.approvals[call.tool_call_id] = True
+ elif app_context.session.is_yolo():
deferred_results.approvals[call.tool_call_id] = True
+ else:
+ feedback = await _prompt_with_toggle(cs.UI_FEEDBACK_PROMPT)
+ denial_msg = feedback.strip() or denial_default
+ deferred_results.approvals[call.tool_call_id] = ToolDenied(denial_msg)
return deferred_results
+def _approval_keybindings() -> KeyBindings:
+ bindings = KeyBindings()
+
+ @bindings.add(cs.KeyBinding.SHIFT_TAB)
+ def _toggle(event: KeyPressEvent) -> None:
+ app_context.session.cycle_permission_mode()
+ if app_context.session.is_yolo():
+ event.app.exit(result=cs.YES_ANSWER)
+ else:
+ event.app.invalidate()
+
+ @bindings.add(cs.KeyBinding.CTRL_C)
+ def _interrupt(event: KeyPressEvent) -> None:
+ event.app.exit(exception=KeyboardInterrupt)
+
+ return bindings
+
+
+async def _confirm_with_toggle(question: str) -> bool:
+ bindings = _approval_keybindings()
+ prompt_text = HTML(
+ f' [y/n] (Y): '
+ )
+ session: PromptSession[str] = PromptSession()
+ while True:
+ try:
+ answer = await session.prompt_async(
+ prompt_text,
+ key_bindings=bindings,
+ style=ORANGE_STYLE,
+ bottom_toolbar=_status_bar_label,
+ refresh_interval=0.5,
+ )
+ except (KeyboardInterrupt, EOFError):
+ return False
+ if app_context.session.is_yolo():
+ return True
+ normalized = (answer or "").strip().lower()
+ if normalized in cs.YES_ANSWERS:
+ return True
+ if normalized in cs.NO_ANSWERS:
+ return False
+
+
+async def _prompt_with_toggle(question: str) -> str:
+ bindings = _approval_keybindings()
+ prompt_text = HTML(
+ f': '
+ )
+ session: PromptSession[str] = PromptSession()
+ try:
+ answer = await session.prompt_async(
+ prompt_text,
+ key_bindings=bindings,
+ style=ORANGE_STYLE,
+ bottom_toolbar=_status_bar_label,
+ refresh_interval=0.5,
+ )
+ except (KeyboardInterrupt, EOFError):
+ return ""
+ return answer or ""
+
+
+def _rich_log_sink(message: object) -> None:
+ app_context.console.print(str(message), end="", markup=False, highlight=False)
+
+
def _setup_common_initialization(repo_path: str) -> Path:
logger.remove()
- logger.add(sys.stdout, format=cs.LOG_FORMAT)
+ logger.add(_rich_log_sink, format=cs.LOG_FORMAT, colorize=False)
project_root = Path(repo_path).resolve()
tmp_dir = project_root / cs.TMP_DIR
@@ -262,6 +439,7 @@ def _setup_common_initialization(repo_path: str) -> Path:
tmp_dir.unlink()
tmp_dir.mkdir()
+ app_context.session.target_repo = project_root
return project_root
@@ -385,46 +563,125 @@ async def run_with_cancellation[T](
return CancelledResult(cancelled=True)
+def _cancel_orphaned_tool_calls(message_history: list[ModelMessage]) -> None:
+ if not message_history:
+ return
+ last = message_history[-1]
+ if not isinstance(last, ModelResponse):
+ return
+ tool_calls = [p for p in last.parts if isinstance(p, ToolCallPart)]
+ if not tool_calls:
+ return
+ message_history.append(
+ ModelRequest(
+ parts=[
+ ToolReturnPart(
+ tool_name=p.tool_name,
+ content=cs.MSG_TOOL_CALL_CANCELLED,
+ tool_call_id=p.tool_call_id,
+ )
+ for p in tool_calls
+ ]
+ )
+ )
+
+
+def _price_current_run(
+ usage: RunUsage, model_config: ModelConfig | None
+) -> Decimal | None:
+ if model_config is None:
+ try:
+ model_config = settings.active_orchestrator_config
+ except Exception: # noqa: BLE001 - pricing is display-only, never fatal
+ return None
+ from .services.usage_cost import price_run
+
+ return price_run(usage, model_config.provider, model_config.model_id)
+
+
+def _record_and_print_turn_usage(
+ turn_input: int, turn_output: int, turn_cost: Decimal, turn_priced: bool
+) -> None:
+ session = app_context.session
+ session.total_input_tokens += turn_input
+ session.total_output_tokens += turn_output
+ session.total_cost_usd += turn_cost
+ if not turn_priced:
+ session.cost_incomplete = True
+ line = cs.UI_TURN_USAGE_TOKENS.format(
+ ti=turn_input,
+ to=turn_output,
+ si=session.total_input_tokens,
+ so=session.total_output_tokens,
+ )
+ if turn_priced:
+ template = (
+ cs.UI_TURN_USAGE_COST_PARTIAL
+ if session.cost_incomplete
+ else cs.UI_TURN_USAGE_COST
+ )
+ line += template.format(tc=turn_cost, sc=session.total_cost_usd)
+ app_context.console.print(dim(line))
+
+
async def _run_agent_response_loop(
rag_agent: Agent[None, str | DeferredToolRequests],
message_history: list[ModelMessage],
- question_with_context: str,
+ question_with_context: str | list[UserContent],
config: AgentLoopUI,
tool_names: ConfirmationToolNames,
model_override: Model | None = None,
+ model_override_config: ModelConfig | None = None,
) -> None:
deferred_results: DeferredToolResults | None = None
+ pending_prompt: str | list[UserContent] | None = question_with_context
+ turn_input = turn_output = 0
+ turn_cost = Decimal(0)
+ turn_priced = False
while True:
- with app_context.console.status(config.status_message):
+ with _thinking_with_status_bar(config.status_message):
response = await run_with_cancellation(
rag_agent.run(
- question_with_context,
+ pending_prompt,
message_history=message_history,
deferred_tool_results=deferred_results,
model=model_override,
),
)
+ pending_prompt = None
if isinstance(response, CancelledResult):
log_session_event(config.cancelled_log)
app_context.session.cancelled = True
+ _cancel_orphaned_tool_calls(message_history)
break
+ message_history.extend(response.new_messages())
+
+ run_usage = response.usage()
+ turn_input += run_usage.input_tokens
+ turn_output += run_usage.output_tokens
+ run_cost = _price_current_run(run_usage, model_override_config)
+ if run_cost is not None:
+ turn_cost += run_cost
+ turn_priced = True
+
if isinstance(response.output, DeferredToolRequests):
- deferred_results = _process_tool_approvals(
+ deferred_results = await _process_tool_approvals(
response.output,
config.approval_prompt,
config.denial_default,
tool_names,
)
- message_history.extend(response.new_messages())
continue
+ asyncio.create_task(_refresh_context_tokens(list(message_history)))
+
output_text = response.output
if not isinstance(output_text, str):
continue
- markdown_response = Markdown(output_text)
+ markdown_response = LeftAlignedMarkdown(_autowrap_diff_blocks(output_text))
app_context.console.print(
Panel(
markdown_response,
@@ -434,35 +691,30 @@ async def _run_agent_response_loop(
)
log_session_event(f"{cs.SESSION_PREFIX_ASSISTANT}{output_text}")
- message_history.extend(response.new_messages())
+ _record_and_print_turn_usage(turn_input, turn_output, turn_cost, turn_priced)
break
-def _find_image_paths(question: str) -> list[Path]:
+def _find_multimodal_paths(question: str) -> list[Path]:
try:
if os.name == "nt":
- # (H) On Windows, shlex.split with posix=False to preserve backslashes
tokens = shlex.split(question, posix=False)
else:
tokens = shlex.split(question)
except ValueError:
tokens = question.split()
- image_paths: list[Path] = []
+ paths: list[Path] = []
for token in tokens:
- # (H) Strip quotes if they remain (shlex with posix=False might keep some)
token = token.strip("'\"")
- # (H) Check if it looks like an image path
- if token.lower().endswith(cs.IMAGE_EXTENSIONS):
- # (H) On Windows, could be C:\... or \...
- # (H) On POSIX, starts with /
+ if token.lower().endswith(cs.MULTIMODAL_EXTENSIONS):
p = Path(token)
if p.is_absolute() or token.startswith("/") or token.startswith("\\"):
- image_paths.append(p)
- return image_paths
+ paths.append(p)
+ return paths
-def _get_path_variants(path_str: str) -> tuple[str, ...]:
+def _path_variants(path_str: str) -> tuple[str, ...]:
return (
path_str.replace(" ", r"\ "),
f"'{path_str}'",
@@ -471,40 +723,421 @@ def _get_path_variants(path_str: str) -> tuple[str, ...]:
)
-def _replace_path_in_question(question: str, old_path: str, new_path: str) -> str:
- for variant in _get_path_variants(old_path):
- if variant in question:
- return question.replace(variant, new_path)
- logger.warning(ls.PATH_NOT_IN_QUESTION.format(path=old_path))
- return question
+def _guess_media_type(path: Path) -> str:
+ mime, _ = mimetypes.guess_type(str(path))
+ return mime or cs.MIME_TYPE_FALLBACK
-def _handle_chat_images(question: str, project_root: Path) -> str:
- image_files = _find_image_paths(question)
- if not image_files:
+def _build_user_prompt(question: str) -> str | list[UserContent]:
+ paths = _find_multimodal_paths(question)
+ if not paths:
return question
- tmp_dir = project_root / cs.TMP_DIR
- tmp_dir.mkdir(exist_ok=True)
- updated_question = question
-
- for original_path in image_files:
- if not original_path.exists() or not original_path.is_file():
- logger.warning(ls.IMAGE_NOT_FOUND.format(path=original_path))
+ content: list[UserContent] = []
+ remaining = question
+ for path in paths:
+ if not path.exists() or not path.is_file():
+ logger.warning(ls.MULTIMODAL_NOT_FOUND.format(path=path))
continue
-
+ match_token = next(
+ (v for v in _path_variants(str(path)) if v in remaining), None
+ )
+ if match_token is None:
+ logger.warning(ls.PATH_NOT_IN_QUESTION.format(path=path))
+ continue
+ before, _, after = remaining.partition(match_token)
+ if before.strip():
+ content.append(before.rstrip())
try:
- new_path = tmp_dir / f"{uuid.uuid4()}-{original_path.name}"
- shutil.copy(original_path, new_path)
- new_relative = str(new_path.relative_to(project_root))
- updated_question = _replace_path_in_question(
- updated_question, str(original_path), new_relative
+ content.append(
+ BinaryContent(
+ data=path.read_bytes(), media_type=_guess_media_type(path)
+ )
)
- logger.info(ls.IMAGE_COPIED.format(path=new_relative))
+ logger.info(ls.MULTIMODAL_ATTACHED.format(path=path))
except Exception as e:
- logger.error(ls.IMAGE_COPY_FAILED.format(error=e))
+ logger.error(ls.MULTIMODAL_READ_FAILED.format(path=path, error=e))
+ content.append(match_token)
+ remaining = after
+
+ if remaining.strip():
+ content.append(remaining.lstrip())
+
+ return content or question
- return updated_question
+
+def _permission_mode_label() -> str:
+ return (
+ cs.PERMISSION_MODE_YOLO_LABEL
+ if app_context.session.is_yolo()
+ else cs.PERMISSION_MODE_NORMAL_LABEL
+ )
+
+
+def _git_state() -> tuple[str, bool] | None:
+ repo = app_context.session.target_repo
+ if repo is None or not repo.exists():
+ return None
+ try:
+ result = subprocess.run(
+ ["git", "status", "--porcelain", "--branch"],
+ capture_output=True,
+ text=True,
+ timeout=1.0,
+ check=True,
+ cwd=repo,
+ )
+ except (subprocess.SubprocessError, OSError):
+ return None
+ lines = result.stdout.splitlines()
+ if not lines or not lines[0].startswith("## "):
+ return None
+ header = lines[0][3:].split("...", 1)[0].split(" ", 1)[0]
+ if header in ("HEAD", "No"):
+ return None
+ is_dirty = any(line for line in lines[1:])
+ return header, is_dirty
+
+
+def _terminal_columns() -> int:
+ return shutil.get_terminal_size((80, 24)).columns
+
+
+def _format_tokens(n: int) -> str:
+ if n >= 1_000_000:
+ return f"{n / 1_000_000:.1f}M"
+ if n >= 1_000:
+ return f"{n / 1_000:.1f}k"
+ return str(n)
+
+
+def _token_color(pct: float) -> str:
+ if pct >= cs.TOKEN_THRESHOLD_CRITICAL:
+ return cs.TOKEN_COLOR_CRITICAL
+ if pct >= cs.TOKEN_THRESHOLD_WARNING:
+ return cs.TOKEN_COLOR_WARNING
+ return cs.TOKEN_COLOR_OK
+
+
+def _token_usage() -> tuple[int, int, float]:
+ try:
+ used = int(app_context.session.context_tokens)
+ except (TypeError, ValueError):
+ used = 0
+ try:
+ model_id = settings.active_orchestrator_config.model_id or ""
+ except Exception:
+ model_id = ""
+ bare = model_id.split(":", 1)[-1]
+ max_ctx = cs.MODEL_CONTEXT_WINDOWS.get(bare, cs.DEFAULT_CONTEXT_WINDOW)
+ pct = (used / max_ctx * 100) if max_ctx > 0 else 0.0
+ return used, max_ctx, pct
+
+
+async def _refresh_context_tokens(messages: list[ModelMessage]) -> None:
+ try:
+ config = settings.active_orchestrator_config
+ except Exception:
+ return
+ if config.provider != cs.Provider.ANTHROPIC or not config.api_key:
+ return
+ try:
+ from .services.anthropic_token_counter import count_anthropic_context
+
+ count = await count_anthropic_context(config.api_key, config.model_id, messages)
+ app_context.session.context_tokens = count
+ except Exception as e:
+ logger.debug(ls.CONTEXT_TOKEN_COUNT_FAILED.format(error=e))
+
+
+def _prime_context_token_counter(system_prompt: str) -> None:
+ if not system_prompt:
+ return
+ from pydantic_ai.messages import ModelRequest, SystemPromptPart
+
+ baseline_messages: list[ModelMessage] = [
+ ModelRequest(parts=[SystemPromptPart(content=system_prompt)])
+ ]
+ asyncio.create_task(_refresh_context_tokens(baseline_messages))
+
+
+def _short_model_id() -> tuple[str, str]:
+ try:
+ orch = settings.active_orchestrator_config.model_id or ""
+ except Exception:
+ orch = ""
+ try:
+ cyph = settings.active_cypher_config.model_id or ""
+ except Exception:
+ cyph = ""
+ return orch.split(":", 1)[-1], cyph.split(":", 1)[-1]
+
+
+def _abbreviated_repo(p: Path | None) -> str:
+ if p is None:
+ return ""
+ try:
+ home = Path.home()
+ return (
+ f"~/{p.relative_to(home).as_posix()}"
+ if p.is_relative_to(home)
+ else p.as_posix()
+ )
+ except (ValueError, OSError, RuntimeError):
+ return p.as_posix()
+
+
+def _config_segments() -> list[tuple[str, str]]:
+ orch, cyph = _short_model_id()
+ segments: list[tuple[str, str]] = []
+ if orch:
+ segments.append((cs.STATUS_BAR_CONFIG_LABEL_O, orch))
+ if cyph:
+ segments.append((cs.STATUS_BAR_CONFIG_LABEL_C, cyph))
+ segments.append(
+ (
+ cs.STATUS_BAR_CONFIG_LABEL_EDIT,
+ cs.STATUS_BAR_EDIT_ON
+ if app_context.session.confirm_edits
+ else cs.STATUS_BAR_EDIT_OFF,
+ )
+ )
+ segments.append(
+ (
+ cs.STATUS_BAR_CONFIG_LABEL_INSTRUCTIONS,
+ cs.STATUS_BAR_EDIT_ON
+ if app_context.session.load_cgr_instructions
+ else cs.STATUS_BAR_EDIT_OFF,
+ )
+ )
+ repo = _abbreviated_repo(app_context.session.target_repo)
+ if repo:
+ segments.append((cs.STATUS_BAR_CONFIG_LABEL_REPO, repo))
+ return segments
+
+
+def _config_status_html() -> str:
+ parts = [
+ f''
+ f''
+ for label, value in _config_segments()
+ ]
+ return cs.STATUS_BAR_CONFIG_SEPARATOR.join(parts)
+
+
+def _config_status_plain() -> str:
+ parts = [f"{label}:{value}" for label, value in _config_segments()]
+ return cs.STATUS_BAR_CONFIG_SEPARATOR.join(parts)
+
+
+def _config_status_rich() -> Text:
+ line = Text()
+ segments = _config_segments()
+ for i, (label, value) in enumerate(segments):
+ if i > 0:
+ line.append(cs.STATUS_BAR_CONFIG_SEPARATOR, style="dim")
+ line.append(f"{label}:", style=f"bold {cs.STATUS_BAR_CONFIG_LABEL_COLOR}")
+ line.append(value, style=cs.STATUS_BAR_CONFIG_COLOR)
+ return line
+
+
+def _branch_chip_html_and_plain(state: tuple[str, bool] | None) -> tuple[str, str]:
+ if state is None:
+ return "", ""
+ branch, is_dirty = state
+ html_template = (
+ cs.STATUS_BAR_BRANCH_DIRTY_HTML if is_dirty else cs.STATUS_BAR_BRANCH_CLEAN_HTML
+ )
+ plain_template = (
+ cs.STATUS_BAR_BRANCH_DIRTY_PLAIN
+ if is_dirty
+ else cs.STATUS_BAR_BRANCH_CLEAN_PLAIN
+ )
+ return (
+ html_template.format(branch=html_escape(branch)),
+ plain_template.format(branch=branch),
+ )
+
+
+def _branch_chip_rich(state: tuple[str, bool] | None) -> Text:
+ if state is None:
+ return Text()
+ branch, is_dirty = state
+ marker = cs.STATUS_BAR_DIRTY_MARKER if is_dirty else ""
+ chip_style = cs.STATUS_BAR_DIRTY_STYLE if is_dirty else cs.STATUS_BAR_CLEAN_STYLE
+ chip = Text()
+ chip.append(
+ cs.STATUS_BAR_BRANCH_RICH_TEXT.format(branch=branch, marker=marker),
+ style=chip_style,
+ )
+ return chip
+
+
+def _status_bar_label() -> HTML | str:
+ mode = _permission_mode_label()
+ state = _git_state()
+ columns = _terminal_columns()
+ sep_html = (
+ f'"
+ )
+
+ used, max_ctx, pct = _token_usage()
+ used_str = _format_tokens(used)
+ max_str = _format_tokens(max_ctx)
+ pct_str = f"{pct:.1f}%"
+ token_html = cs.STATUS_BAR_TOKEN_HTML.format(
+ color=_token_color(pct),
+ used=used_str,
+ max_ctx=max_str,
+ pct=pct_str,
+ )
+ token_plain = f" {used_str} / {max_str} ({pct_str})"
+ body_html = html_escape(mode) + token_html
+ body_plain = mode + token_plain
+
+ config_html = _config_status_html()
+ config_plain = _config_status_plain()
+ branch_html, branch_plain = _branch_chip_html_and_plain(state)
+
+ config_with_branch_html = config_html
+ config_with_branch_plain = config_plain
+ if branch_html:
+ if config_html:
+ config_with_branch_html = f"{config_html} {branch_html}"
+ config_with_branch_plain = f"{config_plain} {branch_plain}"
+ else:
+ config_with_branch_html = branch_html
+ config_with_branch_plain = branch_plain
+
+ if not config_with_branch_plain:
+ return HTML(f"{sep_html}\n{body_html}")
+ inline_sep = " "
+ if len(body_plain) + len(inline_sep) + len(config_with_branch_plain) <= columns:
+ return HTML(f"{sep_html}\n{body_html}{inline_sep}{config_with_branch_html}")
+ return HTML(f"{sep_html}\n{config_with_branch_html}\n{body_html}")
+
+
+def _rich_status_bar() -> Text:
+ body = Text()
+ body.append(_permission_mode_label(), style="dim")
+ used, max_ctx, pct = _token_usage()
+ body.append(" ")
+ body.append(
+ f"{_format_tokens(used)} / {_format_tokens(max_ctx)} ({pct:.1f}%)",
+ style=_token_color(pct),
+ )
+
+ config_line = _config_status_rich()
+ branch_chip = _branch_chip_rich(_git_state())
+ if config_line.plain and branch_chip.plain:
+ config_line.append(" ")
+ config_line.append_text(branch_chip)
+ elif branch_chip.plain:
+ config_line = branch_chip
+
+ if not config_line.plain:
+ return body
+
+ inline_sep = " "
+ if (
+ len(body.plain) + len(inline_sep) + len(config_line.plain)
+ <= _terminal_columns()
+ ):
+ body.append(inline_sep)
+ body.append_text(config_line)
+ return body
+ return Text("\n").join([config_line, body])
+
+
+@contextmanager
+def _shift_tab_listener():
+ if sys.platform == "win32" or not sys.stdin.isatty():
+ yield
+ return
+ try:
+ import termios
+ except ImportError:
+ yield
+ return
+ fd = sys.stdin.fileno()
+ try:
+ original = termios.tcgetattr(fd)
+ except (termios.error, OSError):
+ yield
+ return
+ try:
+ new_attrs = termios.tcgetattr(fd)
+ new_attrs[3] &= ~(termios.ICANON | termios.ECHO)
+ new_attrs[6][termios.VMIN] = 0
+ new_attrs[6][termios.VTIME] = 0
+ termios.tcsetattr(fd, termios.TCSANOW, new_attrs)
+ loop = asyncio.get_running_loop()
+ buffer = bytearray()
+
+ def on_input() -> None:
+ try:
+ data = os.read(fd, 1024)
+ except OSError:
+ return
+ if not data:
+ return
+ buffer.extend(data)
+ while cs.SHIFT_TAB_ESCAPE in buffer:
+ idx = buffer.index(cs.SHIFT_TAB_ESCAPE)
+ del buffer[idx : idx + len(cs.SHIFT_TAB_ESCAPE)]
+ app_context.session.cycle_permission_mode()
+
+ loop.add_reader(fd, on_input)
+ try:
+ yield
+ finally:
+ try:
+ loop.remove_reader(fd)
+ except Exception:
+ pass
+ finally:
+ try:
+ termios.tcsetattr(fd, termios.TCSADRAIN, original)
+ except (termios.error, OSError):
+ pass
+
+
+@contextmanager
+def _thinking_with_status_bar(message: str):
+ spinner = Spinner(cs.STATUS_BAR_SPINNER, text=Text.from_markup(message))
+ separator = Text(
+ cs.STATUS_BAR_SEPARATOR_CHAR * _terminal_columns(),
+ style=cs.STATUS_BAR_SEPARATOR_COLOR,
+ )
+
+ def render() -> Group:
+ return Group(separator, spinner, _rich_status_bar())
+
+ with (
+ Live(
+ render(),
+ console=app_context.console,
+ refresh_per_second=4,
+ transient=True,
+ ) as live,
+ _shift_tab_listener(),
+ ):
+
+ async def _refresh_bar() -> None:
+ while True:
+ try:
+ live.update(render())
+ await asyncio.sleep(0.25)
+ except asyncio.CancelledError:
+ return
+
+ refresh_task = asyncio.get_running_loop().create_task(_refresh_bar())
+ try:
+ yield live
+ finally:
+ refresh_task.cancel()
def get_multiline_input(prompt_text: str = cs.PROMPT_ASK_QUESTION) -> str:
@@ -514,6 +1147,10 @@ def get_multiline_input(prompt_text: str = cs.PROMPT_ASK_QUESTION) -> str:
def submit(event: KeyPressEvent) -> None:
event.app.exit(result=event.app.current_buffer.text)
+ @bindings.add(cs.KeyBinding.CTRL_E)
+ def submit_ctrl_e(event: KeyPressEvent) -> None:
+ event.app.exit(result=event.app.current_buffer.text)
+
@bindings.add(cs.KeyBinding.ENTER)
def new_line(event: KeyPressEvent) -> None:
event.current_buffer.insert_text("\n")
@@ -522,6 +1159,11 @@ def new_line(event: KeyPressEvent) -> None:
def keyboard_interrupt(event: KeyPressEvent) -> None:
event.app.exit(exception=KeyboardInterrupt)
+ @bindings.add(cs.KeyBinding.SHIFT_TAB)
+ def toggle_permission_mode(event: KeyPressEvent) -> None:
+ app_context.session.cycle_permission_mode()
+ event.app.invalidate()
+
clean_prompt = Text.from_markup(prompt_text).plain
print_formatted_text(
@@ -538,6 +1180,8 @@ def keyboard_interrupt(event: KeyPressEvent) -> None:
key_bindings=bindings,
wrap_lines=True,
style=ORANGE_STYLE,
+ bottom_toolbar=_status_bar_label,
+ refresh_interval=0.5,
)
if result is None:
raise EOFError
@@ -664,22 +1308,21 @@ async def _run_interactive_loop(
log_session_event(f"{cs.SESSION_PREFIX_USER}{question}")
if app_context.session.cancelled:
- question_with_context = question + get_session_context()
+ question_text = question + get_session_context()
app_context.session.reset_cancelled()
else:
- question_with_context = question
+ question_text = question
- question_with_context = _handle_chat_images(
- question_with_context, project_root
- )
+ user_prompt: str | list[UserContent] = _build_user_prompt(question_text)
await _run_agent_response_loop(
rag_agent,
message_history,
- question_with_context,
+ user_prompt,
config,
tool_names,
model_override,
+ model_override_config,
)
initial_question = None
@@ -752,6 +1395,8 @@ def connect_memgraph(batch_size: int) -> MemgraphIngestor:
host=settings.MEMGRAPH_HOST,
port=settings.MEMGRAPH_PORT,
batch_size=batch_size,
+ username=settings.MEMGRAPH_USERNAME,
+ password=settings.MEMGRAPH_PASSWORD,
)
@@ -902,7 +1547,7 @@ def prompt_for_unignored_directories(
cli_excludes: list[str] | None = None,
) -> frozenset[str]:
detected = detect_excludable_directories(repo_path)
- cgrignore = load_cgrignore_patterns(repo_path)
+ cgrignore = load_ignore_patterns(repo_path)
cli_patterns = frozenset(cli_excludes) if cli_excludes else frozenset()
pre_excluded = cli_patterns | cgrignore.exclude
@@ -969,23 +1614,27 @@ def _validate_provider_config(role: cs.ModelRole, config: ModelConfig) -> None:
def _initialize_services_and_agent(
- repo_path: str, ingestor: QueryProtocol
-) -> tuple[Agent[None, str | DeferredToolRequests], ConfirmationToolNames]:
+ repo_path: str,
+ ingestor: QueryProtocol,
+ active_projects: list[str] | None = None,
+) -> tuple[Agent[None, str | DeferredToolRequests], ConfirmationToolNames, str]:
_validate_provider_config(
cs.ModelRole.ORCHESTRATOR, settings.active_orchestrator_config
)
_validate_provider_config(cs.ModelRole.CYPHER, settings.active_cypher_config)
- cypher_generator = CypherGenerator()
+ cypher_generator = CypherGenerator(active_projects=active_projects)
code_retriever = CodeRetriever(project_root=repo_path, ingestor=ingestor)
file_reader = FileReader(project_root=repo_path)
file_writer = FileWriter(project_root=repo_path)
file_editor = FileEditor(project_root=repo_path)
shell_commander = ShellCommander(
- project_root=repo_path, timeout=settings.SHELL_COMMAND_TIMEOUT
+ project_root=repo_path,
+ timeout=settings.SHELL_COMMAND_TIMEOUT,
+ is_yolo=app_context.session.is_yolo,
)
directory_lister = DirectoryLister(project_root=repo_path)
- document_analyzer = DocumentAnalyzer(project_root=repo_path)
+ ast_grep_service = AstGrepService(project_root=repo_path)
query_tool = create_query_tool(ingestor, cypher_generator, app_context.console)
code_tool = create_code_retrieval_tool(code_retriever)
@@ -994,17 +1643,19 @@ def _initialize_services_and_agent(
file_editor_tool = create_file_editor_tool(file_editor)
shell_command_tool = create_shell_command_tool(shell_commander)
directory_lister_tool = create_directory_lister_tool(directory_lister)
- document_analyzer_tool = create_document_analyzer_tool(document_analyzer)
- semantic_search_tool = create_semantic_search_tool()
- function_source_tool = create_get_function_source_tool()
+ semantic_search_tool = create_semantic_search_tool(ingestor)
+ function_source_tool = create_get_function_source_tool(ingestor)
+ structural_search_tool = create_structural_search_tool(ast_grep_service)
+ structural_editor_tool = create_structural_editor_tool(ast_grep_service)
confirmation_tool_names = ConfirmationToolNames(
replace_code=file_editor_tool.name,
create_file=file_writer_tool.name,
shell_command=shell_command_tool.name,
+ structural_replace=structural_editor_tool.name,
)
- rag_agent = create_rag_orchestrator(
+ rag_agent, system_prompt = create_rag_orchestrator(
tools=[
query_tool,
code_tool,
@@ -1013,21 +1664,57 @@ def _initialize_services_and_agent(
file_editor_tool,
shell_command_tool,
directory_lister_tool,
- document_analyzer_tool,
semantic_search_tool,
function_source_tool,
- ]
+ structural_search_tool,
+ structural_editor_tool,
+ ],
+ project_root=Path(repo_path),
+ load_instructions=app_context.session.load_cgr_instructions,
+ active_projects=active_projects,
)
- return rag_agent, confirmation_tool_names
+ return rag_agent, confirmation_tool_names, system_prompt
+
+
+def main_single_query(
+ repo_path: str,
+ batch_size: int,
+ question: str,
+ active_projects: list[str] | None = None,
+ output_format: cs.QueryFormat = cs.QueryFormat.TABLE,
+) -> None:
+ _setup_common_initialization(repo_path)
+ # Override logger to stderr so stdout is clean for scripted output
+ logger.remove()
+ logger.add(sys.stderr, level=cs.LOG_LEVEL_ERROR, format=cs.LOG_FORMAT)
+
+ with connect_memgraph(batch_size) as ingestor:
+ rag_agent, _, _ = _initialize_services_and_agent(
+ repo_path, ingestor, active_projects=active_projects
+ )
+ response = asyncio.run(rag_agent.run(question, message_history=[]))
+ if output_format == cs.QueryFormat.JSON:
+ payload = QueryJsonOutput(query=question, response=str(response.output))
+ print(json.dumps(payload, ensure_ascii=False)) # noqa: T201
+ else:
+ print(response.output) # noqa: T201
-async def main_async(repo_path: str, batch_size: int) -> None:
+async def main_async(
+ repo_path: str,
+ batch_size: int,
+ active_projects: list[str] | None = None,
+ show_config_table: bool = True,
+ pre_chat_sync: Callable[[], None] | None = None,
+ pre_chat_sync_message: str = cs.MSG_SYNCING_KNOWLEDGE_GRAPH,
+) -> None:
project_root = _setup_common_initialization(repo_path)
- table = _create_configuration_table(repo_path)
- app_context.console.print(table)
+ if show_config_table:
+ table = _create_configuration_table(repo_path)
+ app_context.console.print(table)
- with connect_memgraph(batch_size) as ingestor:
+ async with connect_memgraph(batch_size) as ingestor:
app_context.console.print(style(cs.MSG_CONNECTED_MEMGRAPH, cs.Color.GREEN))
app_context.console.print(
Panel(
@@ -1036,10 +1723,26 @@ async def main_async(repo_path: str, batch_size: int) -> None:
)
)
- rag_agent, tool_names = _initialize_services_and_agent(repo_path, ingestor)
+ rag_agent, tool_names, system_prompt = _initialize_services_and_agent(
+ repo_path, ingestor, active_projects=active_projects
+ )
+ _prime_context_token_counter(system_prompt)
+
+ if pre_chat_sync is not None:
+ await _run_pre_chat_sync(pre_chat_sync, pre_chat_sync_message)
+
await run_chat_loop(rag_agent, [], project_root, tool_names)
+async def _run_pre_chat_sync(task: Callable[[], None], message: str) -> None:
+ logger.disable("codebase_rag")
+ try:
+ with _thinking_with_status_bar(message):
+ await asyncio.to_thread(task)
+ finally:
+ logger.enable("codebase_rag")
+
+
async def main_optimize_async(
language: str,
target_repo_path: str,
@@ -1063,12 +1766,13 @@ async def main_optimize_async(
effective_batch_size = settings.resolve_batch_size(batch_size)
- with connect_memgraph(effective_batch_size) as ingestor:
+ async with connect_memgraph(effective_batch_size) as ingestor:
app_context.console.print(style(cs.MSG_CONNECTED_MEMGRAPH, cs.Color.GREEN))
- rag_agent, tool_names = _initialize_services_and_agent(
+ rag_agent, tool_names, system_prompt = _initialize_services_and_agent(
target_repo_path, ingestor
)
+ _prime_context_token_counter(system_prompt)
await run_optimization_loop(
rag_agent, [], project_root, language, tool_names, reference_document
)
diff --git a/codebase_rag/mcp/__init__.py b/codebase_rag/mcp/__init__.py
index 77c80d78a..f3a26b0b7 100644
--- a/codebase_rag/mcp/__init__.py
+++ b/codebase_rag/mcp/__init__.py
@@ -1 +1,2 @@
-from codebase_rag.mcp.server import main as main
+from codebase_rag.mcp.server import serve_http as serve_http
+from codebase_rag.mcp.server import serve_stdio as serve_stdio
diff --git a/codebase_rag/mcp/client.py b/codebase_rag/mcp/client.py
new file mode 100644
index 000000000..b6abb205d
--- /dev/null
+++ b/codebase_rag/mcp/client.py
@@ -0,0 +1,65 @@
+import asyncio
+import io
+import json
+import os
+import sys
+
+import typer
+from mcp import ClientSession
+from mcp.client.stdio import StdioServerParameters, stdio_client
+
+from codebase_rag import constants as cs
+
+app = typer.Typer()
+
+
+async def _query_with_errlog(question: str, errlog: io.TextIOWrapper) -> dict[str, str]:
+ server_params = StdioServerParameters(
+ command=sys.executable,
+ args=["-m", "codebase_rag.cli", "mcp-server"],
+ )
+
+ async with stdio_client(server=server_params, errlog=errlog) as (read, write):
+ async with ClientSession(read, write) as session:
+ await session.initialize()
+ result = await session.call_tool(
+ cs.MCPToolName.ASK_AGENT,
+ {cs.MCPParamName.QUESTION: question},
+ )
+
+ if result.content:
+ response_text = result.content[0].text
+ try:
+ parsed = json.loads(response_text)
+ if isinstance(parsed, dict):
+ return parsed
+ return {"output": str(parsed)}
+ except json.JSONDecodeError:
+ return {"output": response_text}
+ return {"output": "No response from server"}
+
+
+def query_mcp_server(question: str) -> dict[str, str]:
+ with open(os.devnull, "w") as devnull: # noqa: SIM115
+ return asyncio.run(_query_with_errlog(question, devnull))
+
+
+@app.command()
+def main(
+ question: str = typer.Option(
+ ..., "--ask-agent", "-a", help="Question to ask about the codebase"
+ ),
+) -> None:
+ try:
+ result = query_mcp_server(question)
+ if isinstance(result, dict) and "output" in result:
+ print(result["output"]) # noqa: T201
+ else:
+ print(json.dumps(result)) # noqa: T201
+ except Exception as e:
+ print(f"Error: {e}", file=sys.stderr) # noqa: T201
+ sys.exit(1)
+
+
+if __name__ == "__main__":
+ app()
diff --git a/codebase_rag/mcp/server.py b/codebase_rag/mcp/server.py
index 9218a2d93..99aeb37c5 100644
--- a/codebase_rag/mcp/server.py
+++ b/codebase_rag/mcp/server.py
@@ -1,7 +1,12 @@
+from __future__ import annotations
+
+import contextlib
import json
import os
import sys
+from collections.abc import Iterator
from pathlib import Path
+from typing import TYPE_CHECKING
from loguru import logger
from mcp.server import Server
@@ -16,6 +21,13 @@
from codebase_rag.services.graph_service import MemgraphIngestor
from codebase_rag.services.llm import CypherGenerator
from codebase_rag.types_defs import MCPToolArguments
+from codebase_rag.utils.path_utils import derive_project_name
+from codebase_rag.vector_store import close_qdrant_client
+
+if TYPE_CHECKING:
+ # starlette is a lazy dependency of the HTTP path only; its ASGI
+ # types are needed solely for annotations
+ from starlette.types import ASGIApp, Receive, Scope, Send
def setup_logging() -> None:
@@ -71,9 +83,16 @@ def create_server() -> tuple[Server, MemgraphIngestor]:
host=settings.MEMGRAPH_HOST,
port=settings.MEMGRAPH_PORT,
batch_size=settings.MEMGRAPH_BATCH_SIZE,
+ username=settings.MEMGRAPH_USERNAME,
+ password=settings.MEMGRAPH_PASSWORD,
)
- cypher_generator = CypherGenerator()
+ # Scope Cypher generation to this server's project (named exactly as
+ # indexing names it) so queries don't bleed into other projects sharing
+ # the database (issue #425).
+ cypher_generator = CypherGenerator(
+ active_projects=[derive_project_name(project_root)]
+ )
tools = create_mcp_tools_registry(
project_root=str(project_root),
@@ -135,18 +154,33 @@ async def call_tool(name: str, arguments: MCPToolArguments) -> list[TextContent]
return server, ingestor
-async def main() -> None:
+@contextlib.contextmanager
+def _service_lifecycle(ingestor: MemgraphIngestor) -> Iterator[None]:
+ """Manage shared service lifetimes for the MCP server.
+
+ Opens the Memgraph ingestor connection and releases the vector store client
+ on shutdown, so a CLI indexing run can reuse local resources once the server
+ stops.
+ """
+ try:
+ with ingestor:
+ logger.info(
+ lg.MCP_SERVER_CONNECTED.format(
+ host=settings.MEMGRAPH_HOST, port=settings.MEMGRAPH_PORT
+ )
+ )
+ yield
+ finally:
+ close_qdrant_client()
+
+
+async def serve_stdio() -> None:
logger.info(lg.MCP_SERVER_STARTING)
server, ingestor = create_server()
logger.info(lg.MCP_SERVER_CREATED)
- with ingestor:
- logger.info(
- lg.MCP_SERVER_CONNECTED.format(
- host=settings.MEMGRAPH_HOST, port=settings.MEMGRAPH_PORT
- )
- )
+ with _service_lifecycle(ingestor):
try:
async with stdio_server() as (read_stream, write_stream):
await server.run(
@@ -159,7 +193,106 @@ async def main() -> None:
logger.info(lg.MCP_SERVER_SHUTDOWN)
+_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "::1", "localhost"})
+
+
+def _validate_http_exposure(host: str, auth_token: str | None) -> None:
+ # The StreamableHTTP endpoint's only protection is the bearer token;
+ # refusing a non-loopback bind without one makes accidental network
+ # exposure of the unauthenticated transport impossible (follow-up to
+ # #808: the loopback default protects default deployments, this
+ # protects intentional remote ones).
+ if host not in _LOOPBACK_HOSTS and not auth_token:
+ raise ValueError(lg.MCP_HTTP_EXPOSURE_REFUSED.format(host=host))
+
+
+def _require_bearer_auth(app: ASGIApp, auth_token: str) -> ASGIApp:
+ import secrets
+
+ token_bytes = auth_token.encode(cs.ENCODING_UTF8)
+
+ async def guarded(scope: Scope, receive: Receive, send: Send) -> None:
+ if scope["type"] != "http":
+ await app(scope, receive, send)
+ return
+ provided: bytes | None = None
+ for key, value in scope.get("headers", []):
+ if key.lower() == b"authorization":
+ provided = value
+ break
+ # RFC 7235: the SCHEME compares case-insensitively (not secret, so
+ # an ordinary compare is fine); only the token value needs
+ # constant-time compare_digest
+ authorized = False
+ if provided is not None:
+ scheme, _, credential = provided.partition(b" ")
+ authorized = scheme.lower() == b"bearer" and secrets.compare_digest(
+ credential, token_bytes
+ )
+ if not authorized:
+ await send(
+ {
+ "type": "http.response.start",
+ "status": 401,
+ "headers": [(b"www-authenticate", b"Bearer")],
+ }
+ )
+ await send({"type": "http.response.body", "body": b""})
+ return
+ await app(scope, receive, send)
+
+ return guarded
+
+
+async def serve_http(
+ host: str = settings.MCP_HTTP_HOST,
+ port: int = settings.MCP_HTTP_PORT,
+) -> None:
+ import uvicorn
+ from mcp.server.streamable_http_manager import StreamableHTTPSessionManager
+ from starlette.applications import Starlette
+ from starlette.routing import Mount
+
+ auth_token = settings.MCP_HTTP_AUTH_TOKEN
+ _validate_http_exposure(host, auth_token)
+
+ logger.info(lg.MCP_HTTP_SERVER_STARTING.format(host=host, port=port))
+
+ server, ingestor = create_server()
+
+ session_manager = StreamableHTTPSessionManager(
+ app=server,
+ json_response=False,
+ stateless=False,
+ )
+
+ @contextlib.asynccontextmanager
+ async def lifespan(app: Starlette):
+ with _service_lifecycle(ingestor):
+ async with session_manager.run():
+ logger.info(lg.MCP_HTTP_SERVER_READY.format(host=host, port=port))
+ yield
+
+ # With a token, bearer auth fronts the mount even on loopback
+ # (defense in depth for shared hosts); without one the exposure
+ # guard above already confined the bind.
+ endpoint = session_manager.handle_request
+ if auth_token:
+ endpoint = _require_bearer_auth(endpoint, auth_token)
+
+ starlette_app = Starlette(
+ routes=[
+ Mount(settings.MCP_HTTP_ENDPOINT_PATH, app=endpoint),
+ ],
+ lifespan=lifespan,
+ )
+
+ config = uvicorn.Config(starlette_app, host=host, port=port, log_level="info")
+ uvicorn_server = uvicorn.Server(config)
+ await uvicorn_server.serve()
+
+
if __name__ == "__main__":
import asyncio
- asyncio.run(main())
+ asyncio.run(serve_stdio())
diff --git a/codebase_rag/mcp/tools.py b/codebase_rag/mcp/tools.py
index 5d1d2f7f5..98b0c6ed0 100644
--- a/codebase_rag/mcp/tools.py
+++ b/codebase_rag/mcp/tools.py
@@ -1,7 +1,11 @@
+import asyncio
import itertools
+import sys
from pathlib import Path
from loguru import logger
+from pydantic_ai import Agent
+from rich.console import Console
from codebase_rag import constants as cs
from codebase_rag import logs as lg
@@ -10,9 +14,13 @@
from codebase_rag.models import ToolMetadata
from codebase_rag.parser_loader import load_parsers
from codebase_rag.services.graph_service import MemgraphIngestor
-from codebase_rag.services.llm import CypherGenerator
+from codebase_rag.services.llm import CypherGenerator, create_rag_orchestrator
from codebase_rag.tools import tool_descriptions as td
-from codebase_rag.tools.code_retrieval import CodeRetriever, create_code_retrieval_tool
+from codebase_rag.tools.ast_grep_service import AstGrepService
+from codebase_rag.tools.code_retrieval import (
+ CodeRetriever,
+ create_code_retrieval_tool,
+)
from codebase_rag.tools.codebase_query import create_query_tool
from codebase_rag.tools.directory_lister import (
DirectoryLister,
@@ -21,6 +29,9 @@
from codebase_rag.tools.file_editor import FileEditor, create_file_editor_tool
from codebase_rag.tools.file_reader import FileReader, create_file_reader_tool
from codebase_rag.tools.file_writer import FileWriter, create_file_writer_tool
+from codebase_rag.tools.shell_command import ShellCommander, create_shell_command_tool
+from codebase_rag.tools.structural_editor import create_structural_editor_tool
+from codebase_rag.tools.structural_search import create_structural_search_tool
from codebase_rag.types_defs import (
CodeSnippetResultDict,
DeleteProjectErrorResult,
@@ -35,6 +46,9 @@
MCPToolSchema,
QueryResultDict,
)
+from codebase_rag.utils.dependencies import has_ast_grep, has_semantic_dependencies
+from codebase_rag.utils.path_utils import derive_project_name
+from codebase_rag.vector_store import clear_all_embeddings, delete_project_embeddings
class MCPToolsRegistry:
@@ -47,6 +61,7 @@ def __init__(
self.project_root = project_root
self.ingestor = ingestor
self.cypher_gen = cypher_gen
+ self._ingestor_lock = asyncio.Lock()
self.parsers, self.queries = load_parsers()
@@ -55,9 +70,12 @@ def __init__(
self.file_reader = FileReader(project_root=project_root)
self.file_writer = FileWriter(project_root=project_root)
self.directory_lister = DirectoryLister(project_root=project_root)
+ self.shell_commander = ShellCommander(project_root=project_root)
+ self.ast_grep_service = AstGrepService(project_root=project_root)
+ stderr_console = Console(file=sys.stderr, width=None, force_terminal=True)
self._query_tool = create_query_tool(
- ingestor=ingestor, cypher_gen=cypher_gen, console=None
+ ingestor=ingestor, cypher_gen=cypher_gen, console=stderr_console
)
self._code_tool = create_code_retrieval_tool(code_retriever=self.code_retriever)
self._file_editor_tool = create_file_editor_tool(file_editor=self.file_editor)
@@ -66,6 +84,31 @@ def __init__(
self._directory_lister_tool = create_directory_lister_tool(
directory_lister=self.directory_lister
)
+ self._shell_command_tool = create_shell_command_tool(
+ shell_commander=self.shell_commander
+ )
+ self._structural_search_tool = create_structural_search_tool(
+ service=self.ast_grep_service
+ )
+ self._structural_editor_tool = create_structural_editor_tool(
+ service=self.ast_grep_service
+ )
+ self._structural_available = has_ast_grep()
+
+ self._rag_agent: Agent | None = None
+
+ self._semantic_search_tool = None
+ self._semantic_search_available = False
+
+ if has_semantic_dependencies():
+ from codebase_rag.tools.semantic_search import (
+ create_semantic_search_tool,
+ )
+
+ self._semantic_search_tool = create_semantic_search_tool(self.ingestor)
+ self._semantic_search_available = True
+ else:
+ logger.info(lg.MCP_SEMANTIC_NOT_AVAILABLE)
self._tools: dict[str, ToolMetadata] = {
cs.MCPToolName.LIST_PROJECTS: ToolMetadata(
@@ -122,6 +165,17 @@ def __init__(
handler=self.index_repository,
returns_json=False,
),
+ cs.MCPToolName.UPDATE_REPOSITORY: ToolMetadata(
+ name=cs.MCPToolName.UPDATE_REPOSITORY,
+ description=td.MCP_TOOLS[cs.MCPToolName.UPDATE_REPOSITORY],
+ input_schema=MCPInputSchema(
+ type=cs.MCPSchemaType.OBJECT,
+ properties={},
+ required=[],
+ ),
+ handler=self.update_repository,
+ returns_json=False,
+ ),
cs.MCPToolName.QUERY_CODE_GRAPH: ToolMetadata(
name=cs.MCPToolName.QUERY_CODE_GRAPH,
description=td.MCP_TOOLS[cs.MCPToolName.QUERY_CODE_GRAPH],
@@ -247,33 +301,176 @@ def __init__(
returns_json=False,
),
}
+ if self._semantic_search_available:
+ self._tools[cs.MCPToolName.SEMANTIC_SEARCH] = ToolMetadata(
+ name=cs.MCPToolName.SEMANTIC_SEARCH,
+ description=td.MCP_TOOLS[cs.MCPToolName.SEMANTIC_SEARCH],
+ input_schema=MCPInputSchema(
+ type=cs.MCPSchemaType.OBJECT,
+ properties={
+ cs.MCPParamName.NATURAL_LANGUAGE_QUERY: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.STRING,
+ description=td.MCP_PARAM_NATURAL_LANGUAGE_QUERY,
+ ),
+ cs.MCPParamName.TOP_K: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.INTEGER,
+ description=td.MCP_PARAM_TOP_K,
+ default=5,
+ ),
+ },
+ required=[cs.MCPParamName.NATURAL_LANGUAGE_QUERY],
+ ),
+ handler=self.semantic_search,
+ returns_json=False,
+ )
+
+ if self._structural_available:
+ self._tools[cs.MCPToolName.STRUCTURAL_SEARCH] = ToolMetadata(
+ name=cs.MCPToolName.STRUCTURAL_SEARCH,
+ description=td.MCP_TOOLS[cs.MCPToolName.STRUCTURAL_SEARCH],
+ input_schema=MCPInputSchema(
+ type=cs.MCPSchemaType.OBJECT,
+ properties={
+ cs.MCPParamName.PATTERN: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.STRING,
+ description=td.MCP_PARAM_PATTERN,
+ ),
+ cs.MCPParamName.LANGUAGE: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.STRING,
+ description=td.MCP_PARAM_LANGUAGE,
+ ),
+ },
+ required=[cs.MCPParamName.PATTERN],
+ ),
+ handler=self.structural_search,
+ returns_json=False,
+ )
+ self._tools[cs.MCPToolName.STRUCTURAL_REPLACE] = ToolMetadata(
+ name=cs.MCPToolName.STRUCTURAL_REPLACE,
+ description=td.MCP_TOOLS[cs.MCPToolName.STRUCTURAL_REPLACE],
+ input_schema=MCPInputSchema(
+ type=cs.MCPSchemaType.OBJECT,
+ properties={
+ cs.MCPParamName.PATTERN: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.STRING,
+ description=td.MCP_PARAM_PATTERN,
+ ),
+ cs.MCPParamName.REWRITE: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.STRING,
+ description=td.MCP_PARAM_REWRITE,
+ ),
+ cs.MCPParamName.LANGUAGE: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.STRING,
+ description=td.MCP_PARAM_LANGUAGE,
+ ),
+ cs.MCPParamName.DRY_RUN: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.BOOLEAN,
+ description=td.MCP_PARAM_DRY_RUN,
+ default=True,
+ ),
+ },
+ required=[cs.MCPParamName.PATTERN, cs.MCPParamName.REWRITE],
+ ),
+ handler=self.structural_replace,
+ returns_json=False,
+ )
+
+ self._tools[cs.MCPToolName.ASK_AGENT] = ToolMetadata(
+ name=cs.MCPToolName.ASK_AGENT,
+ description=td.MCP_TOOLS[cs.MCPToolName.ASK_AGENT],
+ input_schema=MCPInputSchema(
+ type=cs.MCPSchemaType.OBJECT,
+ properties={
+ cs.MCPParamName.QUESTION: MCPInputSchemaProperty(
+ type=cs.MCPSchemaType.STRING,
+ description=td.MCP_PARAM_QUESTION,
+ )
+ },
+ required=[cs.MCPParamName.QUESTION],
+ ),
+ handler=self.ask_agent,
+ returns_json=True,
+ )
+
+ @property
+ def rag_agent(self) -> Agent:
+ if self._rag_agent is None:
+ from codebase_rag.tools.semantic_search import (
+ create_get_function_source_tool,
+ )
+
+ tools = [
+ self._query_tool,
+ self._code_tool,
+ self._file_reader_tool,
+ self._file_writer_tool,
+ self._file_editor_tool,
+ self._shell_command_tool,
+ self._directory_lister_tool,
+ create_get_function_source_tool(self.ingestor),
+ ]
+ if self._semantic_search_tool is not None:
+ tools.append(self._semantic_search_tool)
+ if self._structural_available:
+ tools.append(self._structural_search_tool)
+ tools.append(self._structural_editor_tool)
+ self._rag_agent, _ = create_rag_orchestrator(
+ tools=tools, project_root=Path(self.project_root)
+ )
+ return self._rag_agent
+
+ # Setter lets tests inject a mock agent without triggering LLM init
+ @rag_agent.setter
+ def rag_agent(self, value: Agent) -> None:
+ self._rag_agent = value
async def list_projects(self) -> ListProjectsResult:
logger.info(lg.MCP_LISTING_PROJECTS)
try:
- projects = self.ingestor.list_projects()
+ projects = await asyncio.to_thread(self.ingestor.list_projects)
return ListProjectsSuccessResult(projects=projects, count=len(projects))
except Exception as e:
logger.error(lg.MCP_ERROR_LIST_PROJECTS.format(error=e))
return ListProjectsErrorResult(error=str(e), projects=[], count=0)
+ def _get_project_node_ids(self, project_name: str) -> list[int]:
+ rows = self.ingestor.fetch_all(
+ cs.CYPHER_QUERY_PROJECT_NODE_IDS,
+ {cs.KEY_PROJECT_NAME: project_name},
+ )
+ result: list[int] = []
+ for row in rows:
+ node_id = row.get(cs.KEY_NODE_ID)
+ if isinstance(node_id, int):
+ result.append(node_id)
+ return result
+
+ def _cleanup_project_embeddings(self, project_name: str) -> None:
+ node_ids = self._get_project_node_ids(project_name)
+ delete_project_embeddings(project_name, node_ids)
+
+ def _delete_project_sync(self, project_name: str) -> DeleteProjectResult:
+ projects = self.ingestor.list_projects()
+ if project_name not in projects:
+ return DeleteProjectErrorResult(
+ success=False,
+ error=te.MCP_PROJECT_NOT_FOUND.format(
+ project_name=project_name, projects=projects
+ ),
+ )
+ self._cleanup_project_embeddings(project_name)
+ self.ingestor.delete_project(project_name)
+ return DeleteProjectSuccessResult(
+ success=True,
+ project=project_name,
+ message=cs.MCP_PROJECT_DELETED.format(project_name=project_name),
+ )
+
async def delete_project(self, project_name: str) -> DeleteProjectResult:
logger.info(lg.MCP_DELETING_PROJECT.format(project_name=project_name))
try:
- projects = self.ingestor.list_projects()
- if project_name not in projects:
- return DeleteProjectErrorResult(
- success=False,
- error=te.MCP_PROJECT_NOT_FOUND.format(
- project_name=project_name, projects=projects
- ),
- )
- self.ingestor.delete_project(project_name)
- return DeleteProjectSuccessResult(
- success=True,
- project=project_name,
- message=cs.MCP_PROJECT_DELETED.format(project_name=project_name),
- )
+ async with self._ingestor_lock:
+ return await asyncio.to_thread(self._delete_project_sync, project_name)
except Exception as e:
logger.error(lg.MCP_ERROR_DELETE_PROJECT.format(error=e))
return DeleteProjectErrorResult(success=False, error=str(e))
@@ -283,34 +480,109 @@ async def wipe_database(self, confirm: bool) -> str:
return cs.MCP_WIPE_CANCELLED
logger.warning(lg.MCP_WIPING_DATABASE)
try:
- self.ingestor.clean_database()
+ async with self._ingestor_lock:
+ await asyncio.to_thread(self.ingestor.clean_database)
+ await asyncio.to_thread(clear_all_embeddings)
return cs.MCP_WIPE_SUCCESS
except Exception as e:
logger.error(lg.MCP_ERROR_WIPE.format(error=e))
return cs.MCP_WIPE_ERROR.format(error=e)
+ def _index_repository_sync(self) -> str:
+ # Same collision-resistant derivation as the CLI: a bare directory
+ # name would let two repos named alike delete each other's graphs.
+ project_name = derive_project_name(Path(self.project_root))
+ logger.info(lg.MCP_CLEARING_PROJECT.format(project_name=project_name))
+ self._cleanup_project_embeddings(project_name)
+ self.ingestor.delete_project(project_name)
+
+ self.ingestor.ensure_constraints()
+ self.ingestor.flush_all()
+
+ updater = GraphUpdater(
+ ingestor=self.ingestor,
+ repo_path=Path(self.project_root),
+ parsers=self.parsers,
+ queries=self.queries,
+ project_name=project_name,
+ )
+ updater.run()
+ self.ingestor.flush_all()
+
+ return cs.MCP_INDEX_SUCCESS_PROJECT.format(
+ path=self.project_root, project_name=project_name
+ )
+
async def index_repository(self) -> str:
logger.info(lg.MCP_INDEXING_REPO.format(path=self.project_root))
- project_name = Path(self.project_root).resolve().name
try:
- logger.info(lg.MCP_CLEARING_PROJECT.format(project_name=project_name))
- self.ingestor.delete_project(project_name)
-
- updater = GraphUpdater(
- ingestor=self.ingestor,
- repo_path=Path(self.project_root),
- parsers=self.parsers,
- queries=self.queries,
- )
- updater.run()
-
- return cs.MCP_INDEX_SUCCESS_PROJECT.format(
- path=self.project_root, project_name=project_name
- )
+ async with self._ingestor_lock:
+ return await asyncio.to_thread(self._index_repository_sync)
except Exception as e:
logger.error(lg.MCP_ERROR_INDEXING.format(error=e))
return cs.MCP_INDEX_ERROR.format(error=e)
+ def _update_repository_sync(self) -> str:
+ project_name = derive_project_name(Path(self.project_root))
+
+ self.ingestor.ensure_constraints()
+ self.ingestor.flush_all()
+
+ updater = GraphUpdater(
+ ingestor=self.ingestor,
+ repo_path=Path(self.project_root),
+ parsers=self.parsers,
+ queries=self.queries,
+ project_name=project_name,
+ )
+ updater.run()
+ self.ingestor.flush_all()
+ return cs.MCP_UPDATE_SUCCESS.format(path=self.project_root)
+
+ async def update_repository(self) -> str:
+ logger.info(lg.MCP_UPDATING_REPO.format(path=self.project_root))
+ try:
+ async with self._ingestor_lock:
+ return await asyncio.to_thread(self._update_repository_sync)
+ except Exception as e:
+ logger.error(lg.MCP_ERROR_UPDATING.format(error=e))
+ return cs.MCP_UPDATE_ERROR.format(error=e)
+
+ async def semantic_search(self, natural_language_query: str, top_k: int = 5) -> str:
+ assert self._semantic_search_tool is not None
+ logger.info(lg.MCP_SEMANTIC_SEARCH.format(query=natural_language_query))
+ result = await self._semantic_search_tool.function(
+ query=natural_language_query, top_k=top_k
+ )
+ return str(result)
+
+ async def structural_search(self, pattern: str, language: str | None = None) -> str:
+ result = await self._structural_search_tool.function(
+ pattern=pattern, language=language
+ )
+ return str(result)
+
+ async def structural_replace(
+ self,
+ pattern: str,
+ rewrite: str,
+ language: str | None = None,
+ dry_run: bool = True,
+ ) -> str:
+ result = await self._structural_editor_tool.function(
+ pattern=pattern, rewrite=rewrite, language=language, dry_run=dry_run
+ )
+ return str(result)
+
+ async def ask_agent(self, question: str) -> dict[str, str]:
+ logger.info(lg.MCP_ASK_AGENT.format(question=question))
+ try:
+ response = await self.rag_agent.run(question, message_history=[])
+ return {"output": str(response.output)}
+ except Exception as e:
+ logger.error(lg.MCP_ASK_AGENT_ERROR.format(error=e))
+ return {"error": cs.MCP_ASK_AGENT_ERROR.format(error=e)}
+
async def query_code_graph(self, natural_language_query: str) -> QueryResultDict:
logger.info(lg.MCP_QUERY_CODE_GRAPH.format(query=natural_language_query))
try:
@@ -374,7 +646,14 @@ async def read_file(
logger.info(lg.MCP_READ_FILE.format(path=file_path, offset=offset, limit=limit))
try:
if offset is not None or limit is not None:
- full_path = Path(self.project_root) / file_path
+ project_root = Path(self.project_root).resolve()
+ try:
+ full_path = (project_root / file_path).resolve()
+ full_path.relative_to(project_root)
+ except (ValueError, RuntimeError):
+ return te.ERROR_WRAPPER.format(
+ message=lg.FILE_OUTSIDE_ROOT.format(action="access")
+ )
start = offset if offset is not None else 0
with open(full_path, encoding=cs.ENCODING_UTF8) as f:
diff --git a/codebase_rag/models.py b/codebase_rag/models.py
index e189dbde0..0e06d15f4 100644
--- a/codebase_rag/models.py
+++ b/codebase_rag/models.py
@@ -1,11 +1,12 @@
from collections.abc import Callable
from dataclasses import dataclass, field
+from decimal import Decimal
from pathlib import Path
from typing import TYPE_CHECKING, NamedTuple
from rich.console import Console
-from .constants import SupportedLanguage
+from .constants import PermissionMode, SupportedLanguage
from .types_defs import MCPHandlerType, MCPInputSchema, PropertyValue
if TYPE_CHECKING:
@@ -15,12 +16,33 @@
@dataclass
class SessionState:
confirm_edits: bool = True
+ load_cgr_instructions: bool = True
log_file: Path | None = None
cancelled: bool = False
+ permission_mode: PermissionMode = PermissionMode.NORMAL
+ context_tokens: int = 0
+ target_repo: Path | None = None
+ # Cumulative token consumption and USD cost across the session (issue #80).
+ total_input_tokens: int = 0
+ total_output_tokens: int = 0
+ total_cost_usd: Decimal = field(default_factory=lambda: Decimal(0))
+ # Set once any turn cannot be priced, so the session total is a floor.
+ cost_incomplete: bool = False
def reset_cancelled(self) -> None:
self.cancelled = False
+ def is_yolo(self) -> bool:
+ return self.permission_mode == PermissionMode.YOLO
+
+ def cycle_permission_mode(self) -> PermissionMode:
+ self.permission_mode = (
+ PermissionMode.YOLO
+ if self.permission_mode == PermissionMode.NORMAL
+ else PermissionMode.NORMAL
+ )
+ return self.permission_mode
+
def _default_console() -> Console:
return Console(width=None, force_terminal=True)
diff --git a/codebase_rag/parser_fingerprint.py b/codebase_rag/parser_fingerprint.py
new file mode 100644
index 000000000..0845357e0
--- /dev/null
+++ b/codebase_rag/parser_fingerprint.py
@@ -0,0 +1,70 @@
+# A graph is a function of (source files, parser code, parser config). The
+# incremental hash cache keys only the source files, so a parser or config
+# change with unchanged sources leaves stale old-parser edges. This
+# fingerprint keys the other inputs: parse-relevant source files, the pinned
+# grammar wheel versions, and the frontend settings, so a sync can detect the
+# graph was built by a different parser or frontend config.
+import hashlib
+from importlib import metadata
+from pathlib import Path
+
+from . import constants as cs
+from .config import settings
+
+
+def compute_parser_fingerprint(package_root: Path | None = None) -> str:
+ root = package_root if package_root is not None else Path(__file__).resolve().parent
+ hasher = hashlib.md5(usedforsecurity=False)
+ for source in _fingerprint_sources(root):
+ hasher.update(source.relative_to(root).as_posix().encode())
+ hasher.update(source.read_bytes())
+ for entry in _grammar_versions():
+ hasher.update(entry.encode())
+ # The active frontend selection changes which edges are produced for
+ # unchanged sources (e.g. the C# Roslyn hybrid rewrites
+ # INHERITS/IMPLEMENTS), so it is part of the parser identity and must
+ # trip the staleness warning.
+ for entry in _frontend_settings():
+ hasher.update(entry.encode())
+ return hasher.hexdigest()
+
+
+def _frontend_settings() -> list[str]:
+ # The C# entry records the RESOLVED mode, not the setting: under AUTO a
+ # graph built with dotnet present carries hybrid edges and one without
+ # does not, so the two must not share a fingerprint. Imported lazily to
+ # keep this module free of the parsers package at import time.
+ from .parsers.csharp_frontend import resolve_csharp_frontend
+
+ return [
+ f"CPP_FRONTEND={settings.CPP_FRONTEND.value}",
+ f"CSHARP_FRONTEND={resolve_csharp_frontend().value}",
+ ]
+
+
+def _fingerprint_sources(root: Path) -> list[Path]:
+ sources: list[Path] = []
+ for dirname in cs.PARSER_FINGERPRINT_SOURCE_DIRS:
+ sources.extend(
+ path for path in (root / dirname).rglob(cs.PY_SOURCE_GLOB) if path.is_file()
+ )
+ sources.extend(
+ path
+ for name in cs.PARSER_FINGERPRINT_SOURCE_FILES
+ if (path := root / name).is_file()
+ )
+ # The bundled Roslyn frontend tool (.cs/.csproj) is parser code though not
+ # Python; an edit changes the semantic edges produced, so a tool change
+ # must trip the staleness warning.
+ tool_dir = root / cs.PARSER_FINGERPRINT_TOOL_DIR
+ for pattern in cs.PARSER_FINGERPRINT_TOOL_GLOBS:
+ sources.extend(path for path in tool_dir.glob(pattern) if path.is_file())
+ return sorted(sources)
+
+
+def _grammar_versions() -> list[str]:
+ return sorted(
+ cs.GRAMMAR_VERSION_FMT.format(name=dist.name.lower(), version=dist.version)
+ for dist in metadata.distributions()
+ if dist.name and dist.name.lower().startswith(cs.GRAMMAR_DIST_PREFIX)
+ )
diff --git a/codebase_rag/parser_loader.py b/codebase_rag/parser_loader.py
index 69ddabda3..2c5a78b32 100644
--- a/codebase_rag/parser_loader.py
+++ b/codebase_rag/parser_loader.py
@@ -1,6 +1,8 @@
import importlib
import subprocess
import sys
+import threading
+from collections.abc import Callable, Iterator, Mapping
from copy import deepcopy
from pathlib import Path
@@ -33,7 +35,7 @@ def _try_load_from_submodule(lang_name: cs.SupportedLanguage) -> LanguageLoader:
setup_py_path = submodule_path / cs.SETUP_PY
if setup_py_path.exists():
- logger.debug(ls.BUILDING_BINDINGS.format(lang=lang_name))
+ logger.debug(ls.BUILDING_BINDINGS, lang=lang_name)
result = subprocess.run(
[sys.executable, cs.SETUP_PY, cs.BUILD_EXT_CMD, cs.INPLACE_FLAG],
check=False,
@@ -44,14 +46,15 @@ def _try_load_from_submodule(lang_name: cs.SupportedLanguage) -> LanguageLoader:
if result.returncode != 0:
logger.debug(
- ls.BUILD_FAILED.format(
- lang=lang_name, stdout=result.stdout, stderr=result.stderr
- )
+ ls.BUILD_FAILED,
+ lang=lang_name,
+ stdout=result.stdout,
+ stderr=result.stderr,
)
return None
- logger.debug(ls.BUILD_SUCCESS.format(lang=lang_name))
+ logger.debug(ls.BUILD_SUCCESS, lang=lang_name)
- logger.debug(ls.IMPORTING_MODULE.format(module=module_name))
+ logger.debug(ls.IMPORTING_MODULE, module=module_name)
module = importlib.import_module(module_name)
language_attrs: list[str] = [
@@ -63,21 +66,19 @@ def _try_load_from_submodule(lang_name: cs.SupportedLanguage) -> LanguageLoader:
for attr_name in language_attrs:
if hasattr(module, attr_name):
logger.debug(
- ls.LOADED_FROM_SUBMODULE.format(lang=lang_name, attr=attr_name)
+ ls.LOADED_FROM_SUBMODULE, lang=lang_name, attr=attr_name
)
loader: LanguageLoader = getattr(module, attr_name)
return loader
- logger.debug(
- ls.NO_LANG_ATTR.format(module=module_name, available=dir(module))
- )
+ logger.debug(ls.NO_LANG_ATTR, module=module_name, available=dir(module))
finally:
if python_bindings_str in sys.path:
sys.path.remove(python_bindings_str)
except Exception as e:
- logger.debug(ls.SUBMODULE_LOAD_FAILED.format(lang=lang_name, error=e))
+ logger.debug(ls.SUBMODULE_LOAD_FAILED, lang=lang_name, error=e)
return None
@@ -85,15 +86,18 @@ def _try_load_from_submodule(lang_name: cs.SupportedLanguage) -> LanguageLoader:
def _try_import_language(
module_path: str, attr_name: str, lang_name: cs.SupportedLanguage
) -> LanguageLoader:
+ # AttributeError covers a pip package too old to export the requested
+ # grammar variant (tree_sitter_typescript without language_tsx); fall
+ # back rather than crash parser init.
try:
module = importlib.import_module(module_path)
loader: LanguageLoader = getattr(module, attr_name)
return loader
- except ImportError:
+ except (ImportError, AttributeError):
return _try_load_from_submodule(lang_name)
-def _import_language_loaders() -> dict[cs.SupportedLanguage, LanguageLoader]:
+def _language_imports() -> list[LanguageImport]:
language_imports: list[LanguageImport] = [
LanguageImport(
cs.SupportedLanguage.PYTHON,
@@ -113,6 +117,16 @@ def _import_language_loaders() -> dict[cs.SupportedLanguage, LanguageLoader]:
cs.LANG_ATTR_TYPESCRIPT,
cs.SupportedLanguage.TS,
),
+ # Same pip package ships both grammar variants; .tsx needs the tsx
+ # one or JSX parses as an ERROR forest. The submodule name is TSX on
+ # purpose: no grammars/tree-sitter-tsx exists, so a too-old pip
+ # package leaves TSX unavailable, not bound to the wrong grammar.
+ LanguageImport(
+ cs.SupportedLanguage.TSX,
+ cs.TreeSitterModule.TS,
+ cs.LANG_ATTR_TSX,
+ cs.SupportedLanguage.TSX,
+ ),
LanguageImport(
cs.SupportedLanguage.RUST,
cs.TreeSitterModule.RUST,
@@ -137,6 +151,12 @@ def _import_language_loaders() -> dict[cs.SupportedLanguage, LanguageLoader]:
cs.QUERY_LANGUAGE,
cs.SupportedLanguage.JAVA,
),
+ LanguageImport(
+ cs.SupportedLanguage.C,
+ cs.TreeSitterModule.C,
+ cs.QUERY_LANGUAGE,
+ cs.SupportedLanguage.C,
+ ),
LanguageImport(
cs.SupportedLanguage.CPP,
cs.TreeSitterModule.CPP,
@@ -149,27 +169,53 @@ def _import_language_loaders() -> dict[cs.SupportedLanguage, LanguageLoader]:
cs.QUERY_LANGUAGE,
cs.SupportedLanguage.LUA,
),
+ LanguageImport(
+ cs.SupportedLanguage.PHP,
+ cs.TreeSitterModule.PHP,
+ cs.LANG_ATTR_PHP,
+ cs.SupportedLanguage.PHP,
+ ),
+ LanguageImport(
+ cs.SupportedLanguage.CSHARP,
+ cs.TreeSitterModule.CSHARP,
+ cs.QUERY_LANGUAGE,
+ cs.SupportedLanguage.CSHARP,
+ ),
+ LanguageImport(
+ cs.SupportedLanguage.DART,
+ cs.TreeSitterModule.DART,
+ cs.QUERY_LANGUAGE,
+ cs.SupportedLanguage.DART,
+ ),
]
- loaders: dict[cs.SupportedLanguage, LanguageLoader] = {
- lang_import.lang_key: _try_import_language(
- lang_import.module_path,
- lang_import.attr_name,
- lang_import.submodule_name,
- )
- for lang_import in language_imports
- }
- for lang_key in LANGUAGE_SPECS:
- lang_name = cs.SupportedLanguage(lang_key)
- if lang_name not in loaders or loaders[lang_name] is None:
- loaders[lang_name] = _try_load_from_submodule(lang_name)
+ return language_imports
+
- return loaders
+_IMPORT_SPECS: dict[cs.SupportedLanguage, LanguageImport] = {
+ lang_import.lang_key: lang_import for lang_import in _language_imports()
+}
+_loader_cache: dict[cs.SupportedLanguage, LanguageLoader] = {}
-_language_loaders = _import_language_loaders()
-LANGUAGE_LIBRARIES: dict[cs.SupportedLanguage, LanguageLoader] = _language_loaders
+def _get_language_library(lang_name: cs.SupportedLanguage) -> LanguageLoader:
+ # One grammar module import per language, on first use, cached for the
+ # process (issue #68: importing all 14 grammars up front made every
+ # startup pay for languages the repo does not contain).
+ if lang_name in _loader_cache:
+ return _loader_cache[lang_name]
+ lang_import = _IMPORT_SPECS.get(lang_name)
+ if lang_import is not None:
+ loader = _try_import_language(
+ lang_import.module_path,
+ lang_import.attr_name,
+ lang_import.submodule_name,
+ )
+ else:
+ loader = _try_load_from_submodule(lang_name)
+ _loader_cache[lang_name] = loader
+ return loader
def _build_query_pattern(node_types: tuple[str, ...], capture_name: str) -> str:
@@ -180,7 +226,7 @@ def _get_locals_pattern(lang_name: cs.SupportedLanguage) -> str | None:
match lang_name:
case cs.SupportedLanguage.JS:
return cs.JS_LOCALS_PATTERN
- case cs.SupportedLanguage.TS:
+ case cs.SupportedLanguage.TS | cs.SupportedLanguage.TSX:
return cs.TS_LOCALS_PATTERN
case _:
return None
@@ -215,10 +261,56 @@ def _create_locals_query(
try:
return Query(language, locals_pattern)
except Exception as e:
- logger.debug(ls.LOCALS_QUERY_FAILED.format(lang=lang_name, error=e))
+ logger.debug(ls.LOCALS_QUERY_FAILED, lang=lang_name, error=e)
return None
+def _create_highlights_query(
+ language: Language, lang_name: cs.SupportedLanguage
+) -> Query | None:
+ query_str = ""
+
+ # TSX shares the TypeScript grammar for highlights
+ query_lang_name = (
+ cs.SupportedLanguage.TS if lang_name == cs.SupportedLanguage.TSX else lang_name
+ )
+
+ try:
+ module_name = (
+ f"{cs.TREE_SITTER_MODULE_PREFIX}{query_lang_name.replace('-', '_')}"
+ )
+ module = importlib.import_module(module_name)
+ if hasattr(module, "HIGHLIGHTS_QUERY"):
+ query_str = module.HIGHLIGHTS_QUERY
+ except Exception as e:
+ logger.debug(
+ f"Failed to load standard highlights query for {query_lang_name}: {e}"
+ )
+
+ try:
+ fallback_path = (
+ Path(__file__).parent / "queries" / "highlights" / f"{query_lang_name}.scm"
+ )
+ if fallback_path.exists():
+ custom_queries = fallback_path.read_text(encoding="utf-8")
+ query_str = (
+ query_str + "\n" + custom_queries if query_str else custom_queries
+ )
+
+ if query_str:
+ return Query(language, query_str)
+ except Exception as e:
+ logger.debug(
+ f"Failed to load fallback highlights query for {query_lang_name}: {e}"
+ )
+
+ return None
+
+
+COMBINED_FUNC_CLASS_QUERIES: dict[cs.SupportedLanguage, Query | None] = {}
+COMBINED_FUNC_CLASS_IMPORT_QUERIES: dict[cs.SupportedLanguage, Query | None] = {}
+
+
def _create_language_queries(
language: Language,
parser: Parser,
@@ -236,12 +328,29 @@ def _create_language_queries(
)
combined_import_patterns = _build_combined_import_pattern(lang_config)
+ combined_fc_pattern = f"{function_patterns} {class_patterns}".strip()
+ try:
+ COMBINED_FUNC_CLASS_QUERIES[lang_name] = (
+ Query(language, combined_fc_pattern) if combined_fc_pattern else None
+ )
+ except Exception:
+ COMBINED_FUNC_CLASS_QUERIES[lang_name] = None
+
+ combined_fci_pattern = f"{function_patterns} {class_patterns} {combined_import_patterns} {call_patterns}".strip()
+ try:
+ COMBINED_FUNC_CLASS_IMPORT_QUERIES[lang_name] = (
+ Query(language, combined_fci_pattern) if combined_fci_pattern else None
+ )
+ except Exception:
+ COMBINED_FUNC_CLASS_IMPORT_QUERIES[lang_name] = None
+
return LanguageQueries(
functions=_create_optional_query(language, function_patterns),
classes=_create_optional_query(language, class_patterns),
calls=_create_optional_query(language, call_patterns),
imports=_create_optional_query(language, combined_import_patterns),
locals=_create_locals_query(language, lang_name),
+ highlights=_create_highlights_query(language, lang_name),
config=lang_config,
language=language,
parser=parser,
@@ -254,9 +363,9 @@ def _process_language(
parsers: dict[cs.SupportedLanguage, Parser],
queries: dict[cs.SupportedLanguage, LanguageQueries],
) -> bool:
- lang_lib = LANGUAGE_LIBRARIES.get(lang_name)
+ lang_lib = _get_language_library(lang_name)
if not lang_lib:
- logger.debug(ls.LIB_NOT_AVAILABLE.format(lang=lang_name))
+ logger.debug(ls.LIB_NOT_AVAILABLE, lang=lang_name)
return False
try:
@@ -269,24 +378,117 @@ def _process_language(
logger.success(ls.GRAMMAR_LOADED.format(lang=lang_name))
return True
except Exception as e:
+ # query compilation can fail AFTER the parser insert; drop the orphan
+ # so the store never exposes a parser without its queries
+ parsers.pop(lang_name, None)
logger.warning(ls.GRAMMAR_LOAD_FAILED.format(lang=lang_name, error=e))
return False
-def load_parsers() -> tuple[
- dict[cs.SupportedLanguage, Parser], dict[cs.SupportedLanguage, LanguageQueries]
-]:
- parsers: dict[cs.SupportedLanguage, Parser] = {}
- queries: dict[cs.SupportedLanguage, LanguageQueries] = {}
- available_languages: list[cs.SupportedLanguage] = []
+class _LazyLanguageView[V](Mapping[cs.SupportedLanguage, V]):
+ # A read-only Mapping over the store's per-language dict; a missing key
+ # triggers a one-time grammar load (issue #68). Mapping derives get,
+ # keys, values, items, and __contains__ from the three methods below,
+ # so laziness needs no dict-override tricks: membership and get load on
+ # demand through __getitem__, and full-view operations (iteration, len,
+ # and everything derived from them) probe every spec first so no
+ # consumer ever sees a partial availability picture.
+ __slots__ = ("_data", "_ensure", "_probe_all")
+
+ def __init__(
+ self,
+ data: dict[cs.SupportedLanguage, V],
+ ensure: Callable[[object], bool],
+ probe_all: Callable[[], None],
+ ) -> None:
+ self._data = data
+ self._ensure = ensure
+ self._probe_all = probe_all
+
+ def __getitem__(self, lang_name: cs.SupportedLanguage) -> V:
+ if lang_name not in self._data:
+ self._ensure(lang_name)
+ return self._data[lang_name]
+
+ def __iter__(self) -> Iterator[cs.SupportedLanguage]:
+ self._probe_all()
+ return iter(self._data)
+
+ def __len__(self) -> int:
+ self._probe_all()
+ return len(self._data)
+
+
+class _LazyGrammarStore:
+ # Process-wide cache: grammars load once per interpreter, not once per
+ # load_parsers() call (which previously recompiled EVERY language's
+ # query set on every call). The lock serializes loads: without it a
+ # thread probing a language MID-LOAD would see it in _attempted but not
+ # yet in the dict and wrongly report it unavailable (PR #802 review).
+ def __init__(self) -> None:
+ self._parser_data: dict[cs.SupportedLanguage, Parser] = {}
+ self._query_data: dict[cs.SupportedLanguage, LanguageQueries] = {}
+ self._attempted: set[object] = set()
+ self._lock = threading.Lock()
+ self.parsers: Mapping[cs.SupportedLanguage, Parser] = _LazyLanguageView(
+ self._parser_data, self._ensure, self._probe_all
+ )
+ self.queries: Mapping[cs.SupportedLanguage, LanguageQueries] = (
+ _LazyLanguageView(self._query_data, self._ensure, self._probe_all)
+ )
- for lang_key, lang_config in deepcopy(LANGUAGE_SPECS).items():
- lang_name = cs.SupportedLanguage(lang_key)
- if _process_language(lang_name, lang_config, parsers, queries):
- available_languages.append(lang_name)
+ def _ensure(self, lang_name: object) -> bool:
+ # the lock-free fast path must see BOTH twin entries: the loader
+ # writes the parser before the queries, so a lone parser entry means
+ # mid-load or failed-load, not a completed language
+ if lang_name in self._parser_data and lang_name in self._query_data:
+ return True
+ with self._lock:
+ if lang_name in self._attempted:
+ return lang_name in self._parser_data and lang_name in self._query_data
+ self._attempted.add(lang_name)
+ if not isinstance(lang_name, str):
+ return False
+ try:
+ lang = cs.SupportedLanguage(lang_name)
+ except ValueError:
+ return False
+ spec = LANGUAGE_SPECS.get(lang)
+ if spec is None:
+ return False
+ return _process_language(
+ lang, deepcopy(spec), self._parser_data, self._query_data
+ )
+
+ def _probe_all(self) -> None:
+ for lang_key in LANGUAGE_SPECS:
+ self._ensure(lang_key)
+ if not self._parser_data:
+ raise RuntimeError(ex.NO_LANGUAGES)
+
+
+_store: _LazyGrammarStore | None = None
+_store_lock = threading.Lock()
- if not available_languages:
- raise RuntimeError(ex.NO_LANGUAGES)
- logger.info(ls.INITIALIZED_PARSERS.format(languages=", ".join(available_languages)))
- return parsers, queries
+def _reset_parser_cache() -> None:
+ # Test hook: discard every cached grammar so laziness can be observed
+ # from a clean slate.
+ global _store
+ with _store_lock:
+ _store = None
+
+
+def load_parsers() -> tuple[
+ Mapping[cs.SupportedLanguage, Parser],
+ Mapping[cs.SupportedLanguage, LanguageQueries],
+]:
+ global _store
+ store = _store
+ if store is None:
+ with _store_lock:
+ if _store is None:
+ _store = _LazyGrammarStore()
+ logger.info(ls.PARSERS_LAZY_READY)
+ store = _store
+ return store.parsers, store.queries
diff --git a/codebase_rag/parsers/ast_grep_patterns/README.md b/codebase_rag/parsers/ast_grep_patterns/README.md
new file mode 100644
index 000000000..4ac88c096
--- /dev/null
+++ b/codebase_rag/parsers/ast_grep_patterns/README.md
@@ -0,0 +1,59 @@
+# ast-grep language patterns
+
+Basic structural support for a language that has **no tree-sitter `LanguageSpec`**
+in cgr can be added with a single YAML file here, instead of a hand-written
+tree-sitter traversal. The `AstGrepTier` (`../ast_grep_tier.py`) loads every
+`*.yaml` in this directory and, for files whose extension matches, emits
+`Module`, `Function`, and `Class` nodes plus `DEFINES` and `IMPORTS`
+relationships, using [ast-grep](https://ast-grep.github.io/) patterns.
+
+This is a **basic** tier: names are flat (no nested-namespace qualification) and
+there is no call-graph (`CALLS`) resolution. Languages that need that get a full
+tree-sitter `LanguageSpec`. The tier is active only when the `ast-grep` extra is
+installed (`pip install code-graph-rag[ast-grep]`); otherwise it is a no-op.
+
+## Config format
+
+```yaml
+language: ruby # human-readable name (documentation only)
+ast_grep_id: ruby # ast-grep language id (see AST_GREP_LANGUAGES)
+extensions: # file extensions routed to this config
+ - ".rb"
+functions: # patterns whose match becomes a Function node
+ - "def self.$NAME"
+ - "def $NAME"
+classes: # patterns whose match becomes a Class node
+ - "class $NAME"
+ - "module $NAME"
+imports: # patterns whose match becomes an IMPORTS edge
+ - "require $PATH"
+ - "require_relative $PATH"
+```
+
+`extensions` and `ast_grep_id` are required; the pattern lists are optional.
+
+## Metavariable conventions
+
+- Definition patterns (`functions`, `classes`) must capture the name as **`$NAME`**.
+- Import patterns must capture the imported path as **`$PATH`** (surrounding
+ quotes are stripped automatically).
+
+## Ordering
+
+Patterns are tried in order and **the first pattern to match a source line
+claims it**. Put specific patterns before general ones so, for example,
+`def self.$NAME` (captures `build`) wins over `def $NAME` (would capture `self`)
+for `def self.build`.
+
+## Testing a new language
+
+Write patterns against a snippet first:
+
+```python
+from ast_grep_py import SgRoot
+root = SgRoot(source, "ruby").root()
+for node in root.find_all(pattern="def $NAME"):
+ print(node.get_match("NAME").text(), node.range().start.line + 1)
+```
+
+Then add an end-to-end test mirroring `tests/test_ast_grep_tier.py`.
diff --git a/codebase_rag/parsers/ast_grep_patterns/ruby.yaml b/codebase_rag/parsers/ast_grep_patterns/ruby.yaml
new file mode 100644
index 000000000..35605ac86
--- /dev/null
+++ b/codebase_rag/parsers/ast_grep_patterns/ruby.yaml
@@ -0,0 +1,21 @@
+# ast-grep pattern config for Ruby (issue #414).
+# Ruby has no tree-sitter LanguageSpec in cgr, so the ast-grep tier extracts
+# structural nodes from these patterns. See README.md for the format.
+#
+# Convention: definition patterns capture the name as $NAME; import patterns
+# capture the imported path as $PATH. Patterns are tried in order; the first
+# pattern to match a given source line claims it (so put specific patterns,
+# like `def self.$NAME`, before general ones, like `def $NAME`).
+language: ruby
+ast_grep_id: ruby
+extensions:
+ - ".rb"
+functions:
+ - "def self.$NAME"
+ - "def $NAME"
+classes:
+ - "class $NAME"
+ - "module $NAME"
+imports:
+ - "require $PATH"
+ - "require_relative $PATH"
diff --git a/codebase_rag/parsers/ast_grep_tier.py b/codebase_rag/parsers/ast_grep_tier.py
new file mode 100644
index 000000000..2b8dd49e7
--- /dev/null
+++ b/codebase_rag/parsers/ast_grep_tier.py
@@ -0,0 +1,278 @@
+# ast-grep pattern-driven language tier (issue #414). For languages with no
+# tree-sitter LanguageSpec, this extracts Module/Function/Class nodes and
+# DEFINES/IMPORTS edges from per-language YAML pattern configs, so adding a
+# new language is a config file rather than a hand-written tree-sitter
+# traversal. It is a BASIC structural tier: names are flat (no nested
+# namespace qualification) and there is no call-graph resolution.
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from .. import constants as cs
+from ..utils.path_utils import cached_relative_path, cached_resolve_posix
+
+if TYPE_CHECKING:
+ from ast_grep_py import SgNode
+
+ from ..services import IngestorProtocol
+
+logger = logging.getLogger(__name__)
+
+_PATTERNS_DIR = Path(__file__).parent / "ast_grep_patterns"
+# Metavar conventions contributors must follow in the YAML patterns.
+_NAME_METAVAR = "NAME"
+_PATH_METAVAR = "PATH"
+
+
+@dataclass(frozen=True)
+class _LangConfig:
+ ast_grep_id: str
+ functions: tuple[str, ...]
+ classes: tuple[str, ...]
+ imports: tuple[str, ...]
+
+
+def load_pattern_configs() -> dict[str, _LangConfig]:
+ """Load every ast_grep_patterns/*.yaml, keyed by file extension."""
+ import yaml
+
+ configs: dict[str, _LangConfig] = {}
+ for path in sorted(_PATTERNS_DIR.glob("*.yaml")):
+ data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
+ extensions = data.get("extensions")
+ ast_grep_id = data.get("ast_grep_id")
+ if not extensions or not ast_grep_id:
+ raise ValueError(
+ f"{path.name}: 'extensions' and 'ast_grep_id' are required"
+ )
+ if isinstance(extensions, str):
+ extensions = [ext.strip() for ext in extensions.split(",") if ext.strip()]
+ config = _LangConfig(
+ ast_grep_id=str(ast_grep_id),
+ functions=tuple(data.get("functions") or ()),
+ classes=tuple(data.get("classes") or ()),
+ imports=tuple(data.get("imports") or ()),
+ )
+ for extension in extensions:
+ configs[extension] = config
+ return configs
+
+
+def _strip_quotes(text: str) -> str:
+ if len(text) >= 2 and text[0] in "\"'" and text[-1] == text[0]:
+ return text[1:-1]
+ return text
+
+
+class AstGrepTier:
+ """Structural extractor for languages without a tree-sitter LanguageSpec."""
+
+ __slots__ = ("_ingestor", "_repo_path", "_project_name", "_configs")
+
+ def __init__(
+ self, ingestor: IngestorProtocol, repo_path: Path, project_name: str
+ ) -> None:
+ self._ingestor = ingestor
+ self._repo_path = repo_path
+ self._project_name = project_name
+ try:
+ import ast_grep_py # noqa: F401
+
+ self._configs = load_pattern_configs()
+ except ImportError:
+ # ast-grep/pyyaml are the [ast-grep] extra; no-op if absent.
+ logger.warning("ast-grep-py unavailable; ast-grep language tier disabled")
+ self._configs = {}
+ except Exception as exc: # noqa: BLE001
+ # a malformed shipped config must not crash GraphUpdater
+ # construction; disable the tier and surface the reason.
+ logger.warning("ast-grep language tier disabled: %s", exc)
+ self._configs = {}
+
+ def handles(self, suffix: str) -> bool:
+ return suffix in self._configs
+
+ def process_file(
+ self, file_path: Path, structural_elements: dict[Path, str | None]
+ ) -> None:
+ config = self._configs.get(file_path.suffix)
+ if config is None:
+ return
+ try:
+ source = file_path.read_text(encoding="utf-8")
+ except (OSError, UnicodeDecodeError):
+ return
+ from ast_grep_py import SgRoot
+
+ try:
+ root = SgRoot(source, config.ast_grep_id).root()
+ except (RuntimeError, ValueError) as exc:
+ logger.warning("ast-grep failed to parse %s: %s", file_path, exc)
+ return
+
+ module_qn = self._emit_module(file_path, structural_elements)
+ relative_path = cached_relative_path(file_path, self._repo_path).as_posix()
+ absolute_path = cached_resolve_posix(file_path)
+
+ # Functions then classes; dedupe by start line PER label so a specific
+ # pattern (def self.$NAME) wins over a general one (def $NAME) on the
+ # same line, while a class and function sharing a line both still land.
+ for label, patterns in (
+ (cs.NodeLabel.FUNCTION, config.functions),
+ (cs.NodeLabel.CLASS, config.classes),
+ ):
+ self._extract_definitions(
+ root,
+ label,
+ patterns,
+ file_path,
+ module_qn,
+ relative_path,
+ absolute_path,
+ )
+ self._extract_imports(root, config.imports, file_path, module_qn)
+
+ def _extract_definitions(
+ self,
+ root: SgNode,
+ label: cs.NodeLabel,
+ patterns: tuple[str, ...],
+ file_path: Path,
+ module_qn: str,
+ relative_path: str,
+ absolute_path: str,
+ ) -> None:
+ claimed: set[int] = set()
+ for pattern in patterns:
+ for node in self._find_all(root, pattern, file_path):
+ name_node = node.get_match(_NAME_METAVAR)
+ if name_node is None:
+ continue
+ line = node.range().start.line
+ if line in claimed:
+ continue
+ claimed.add(line)
+ self._emit_definition(
+ label,
+ name_node.text(),
+ node,
+ module_qn,
+ relative_path,
+ absolute_path,
+ )
+
+ def _extract_imports(
+ self,
+ root: SgNode,
+ patterns: tuple[str, ...],
+ file_path: Path,
+ module_qn: str,
+ ) -> None:
+ for pattern in patterns:
+ for node in self._find_all(root, pattern, file_path):
+ target_node = node.get_match(_PATH_METAVAR)
+ if target_node is not None:
+ self._emit_import(_strip_quotes(target_node.text()), module_qn)
+
+ def _find_all(self, root: SgNode, pattern: str, file_path: Path) -> list[SgNode]:
+ try:
+ return root.find_all(pattern=pattern)
+ except RuntimeError as exc:
+ logger.warning(
+ "bad ast-grep pattern %r for %s: %s", pattern, file_path, exc
+ )
+ return []
+
+ def _emit_module(
+ self, file_path: Path, structural_elements: dict[Path, str | None]
+ ) -> str:
+ relative_path = cached_relative_path(file_path, self._repo_path)
+ # flat module qn, no init/mod special-case or stem disambiguation;
+ # add if a config language collides with another file's stem in the
+ # same directory.
+ module_qn = cs.SEPARATOR_DOT.join(
+ [self._project_name, *relative_path.with_suffix("").parts]
+ )
+ self._ingestor.ensure_node_batch(
+ cs.NodeLabel.MODULE,
+ {
+ cs.KEY_QUALIFIED_NAME: module_qn,
+ cs.KEY_NAME: file_path.name,
+ cs.KEY_PATH: relative_path.as_posix(),
+ cs.KEY_ABSOLUTE_PATH: cached_resolve_posix(file_path),
+ },
+ )
+ parent_rel_path = relative_path.parent
+ parent_container_qn = structural_elements.get(parent_rel_path)
+ if parent_container_qn:
+ parent = (cs.NodeLabel.PACKAGE, cs.KEY_QUALIFIED_NAME, parent_container_qn)
+ elif parent_rel_path != Path("."):
+ parent = (
+ cs.NodeLabel.FOLDER,
+ cs.KEY_ABSOLUTE_PATH,
+ cached_resolve_posix(self._repo_path / parent_rel_path),
+ )
+ else:
+ parent = (cs.NodeLabel.PROJECT, cs.KEY_NAME, self._project_name)
+ self._ingestor.ensure_relationship_batch(
+ parent,
+ cs.RelationshipType.CONTAINS_MODULE,
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ )
+ return module_qn
+
+ def _emit_definition(
+ self,
+ label: cs.NodeLabel,
+ name: str,
+ node: SgNode,
+ module_qn: str,
+ relative_path: str,
+ absolute_path: str,
+ ) -> None:
+ qualified_name = f"{module_qn}{cs.SEPARATOR_DOT}{name}"
+ node_range = node.range()
+ self._ingestor.ensure_node_batch(
+ label,
+ {
+ cs.KEY_QUALIFIED_NAME: qualified_name,
+ cs.KEY_NAME: name,
+ cs.KEY_MODIFIERS: [],
+ cs.KEY_DECORATORS: [],
+ cs.KEY_START_LINE: node_range.start.line + 1,
+ cs.KEY_END_LINE: node_range.end.line + 1,
+ cs.KEY_DOCSTRING: None,
+ # no visibility analysis for these languages; mark exported
+ # so dead-code does not false-flag everything.
+ cs.KEY_IS_EXPORTED: True,
+ cs.KEY_PATH: relative_path,
+ cs.KEY_ABSOLUTE_PATH: absolute_path,
+ },
+ )
+ self._ingestor.ensure_relationship_batch(
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ cs.RelationshipType.DEFINES,
+ (label, cs.KEY_QUALIFIED_NAME, qualified_name),
+ )
+
+ def _emit_import(self, target: str, module_qn: str) -> None:
+ if not target:
+ return
+ # every require target is treated as an external module; local
+ # require_relative resolution needs path handling this tier skips.
+ self._ingestor.ensure_node_batch(
+ cs.NodeLabel.EXTERNAL_MODULE,
+ {
+ cs.KEY_NAME: target,
+ cs.KEY_QUALIFIED_NAME: target,
+ cs.KEY_PATH: target,
+ },
+ )
+ self._ingestor.ensure_relationship_batch(
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ cs.RelationshipType.IMPORTS,
+ (cs.NodeLabel.EXTERNAL_MODULE, cs.KEY_QUALIFIED_NAME, target),
+ )
diff --git a/codebase_rag/parsers/call_processor.py b/codebase_rag/parsers/call_processor.py
index 0e53cbe73..09a377c3f 100644
--- a/codebase_rag/parsers/call_processor.py
+++ b/codebase_rag/parsers/call_processor.py
@@ -1,23 +1,539 @@
from __future__ import annotations
+from bisect import bisect_left, bisect_right
+from collections import defaultdict
+from collections.abc import Callable, Mapping
from pathlib import Path
+from typing import NamedTuple
from loguru import logger
from tree_sitter import Node, QueryCursor
from .. import constants as cs
from .. import logs as ls
+from ..capture import ALL_ENABLED, CaptureSelection
from ..language_spec import LanguageSpec
+from ..parser_loader import COMBINED_FUNC_CLASS_QUERIES
from ..services import IngestorProtocol
-from ..types_defs import FunctionRegistryTrieProtocol, LanguageQueries
+from ..types_defs import (
+ FunctionLocation,
+ FunctionRegistryTrieProtocol,
+ FunctionSpanKey,
+ LanguageQueries,
+ NodeType,
+)
+from ..utils.path_utils import cached_relative_path
from .call_resolver import CallResolver
+from .class_ingest.identity import build_nested_qualified_name_for_class
from .cpp import utils as cpp_utils
+from .cpp.type_inference import CppTypeInferenceEngine
+from .csharp import type_inference as csharp_ti
+from .dart import utils as dart_utils
+from .flow_access import FlowProcessor
+from .go import utils as go_utils
from .import_processor import ImportProcessor
+from .io_access import IOAccessProcessor
+from .java import utils as java_utils
+from .lua import utils as lua_utils
+from .rs import utils as rs_utils
from .type_inference import TypeInferenceEngine
-from .utils import get_function_captures, is_method_node
+from .utils import (
+ cpp_parameter_names,
+ function_span_key,
+ get_function_captures,
+ go_parameter_names,
+ is_method_node,
+ js_ts_parameter_names,
+ python_parameter_names,
+ safe_decode_text,
+ sorted_captures,
+)
+
+
+class _CallableFlowArg(NamedTuple):
+ # One call-site argument that may carry a callable: bound to a concrete
+ # function (source_concrete) or to a caller parameter (source_caller +
+ # source_param), keyed to the callee parameter by position or keyword.
+ callee_qn: str
+ position: int
+ keyword: str
+ source_concrete: str
+ source_caller: str
+ source_param: str
+
+
+class _FactoryCall(NamedTuple):
+ # A call `x(args)` where x was bound by `x = factory(...)`. Each returned
+ # closure of factory receives args, so a callback argument flows into that
+ # closure's callable parameter. Resolved in finalize once every function's
+ # returned callables are known.
+ scope_qn: str
+ factory_qn: str
+ positional: tuple[str, ...]
+ keyword: tuple[tuple[str, str], ...]
+
+
+_TYPED_LANGUAGES = frozenset(
+ {
+ cs.SupportedLanguage.PYTHON,
+ cs.SupportedLanguage.JS,
+ cs.SupportedLanguage.TS,
+ cs.SupportedLanguage.TSX,
+ cs.SupportedLanguage.JAVA,
+ cs.SupportedLanguage.CSHARP,
+ cs.SupportedLanguage.LUA,
+ cs.SupportedLanguage.GO,
+ cs.SupportedLanguage.CPP,
+ cs.SupportedLanguage.RUST,
+ cs.SupportedLanguage.DART,
+ }
+)
+
+# C and C++ share the function_definition/declarator shape, so the callee
+# name lives in a nested declarator (no `name` field), needing the libclang
+# declarator-aware extractor rather than a plain child_by_field_name("name").
+_C_FAMILY_LANGUAGES = frozenset({cs.SupportedLanguage.C, cs.SupportedLanguage.CPP})
+_JS_TS_LANGUAGES = cs.JS_TS_LANGUAGES
+# Languages with argument-REFERENCE edges but no interprocedural
+# callable-param flow: passing a function keeps it reachable (REFERENCES),
+# while invocation edges stay with the flow languages.
+_ARG_REF_ONLY_LANGUAGES = frozenset(
+ {cs.SupportedLanguage.CSHARP, cs.SupportedLanguage.DART}
+)
+_DART_VALUE_WRAPPER_TYPES = frozenset(
+ {cs.TS_DART_LIST_LITERAL, cs.TS_DART_SET_OR_MAP_LITERAL, cs.TS_DART_ARGUMENT}
+)
+# Scopes that bound a Dart local/parameter declaration for getter-read
+# shadowing; a declaration hides a same-name getter only inside them. A loop
+# variable scopes its for_statement, a catch parameter its try_statement.
+_DART_SHADOW_SCOPE_TYPES = frozenset(
+ {
+ cs.TS_DART_BLOCK,
+ cs.TS_DART_FUNCTION_EXPRESSION,
+ cs.TS_DART_LOCAL_FUNCTION_DECLARATION,
+ cs.TS_DART_FOR_STATEMENT,
+ cs.TS_DART_TRY_STATEMENT,
+ }
+)
+_DART_LOCAL_DECLARATION_TYPES = frozenset(
+ {
+ cs.TS_DART_INITIALIZED_VARIABLE_DEFINITION,
+ cs.TS_DART_INITIALIZED_IDENTIFIER,
+ }
+)
+# Statement scopes whose binder is live only AFTER its declaration: the
+# for-in iterable and the try body precede theirs and read the getter. A
+# block-scoped local needs no such split: Dart scopes it to the whole block
+# and rejects reads before the declaration at compile time.
+_DART_STATEMENT_SCOPE_TYPES = frozenset(
+ {cs.TS_DART_FOR_STATEMENT, cs.TS_DART_TRY_STATEMENT}
+)
+# Parents whose direct identifier is never a value read: member/cascade
+# selectors carry the chain passes' members, a label names a parameter,
+# catch parameters only BIND names, and a signature's identifier is the
+# DECLARED name (the module pass walks class bodies for field initializers,
+# so signatures are in view there).
+_DART_NON_READ_PARENT_TYPES = (
+ frozenset(
+ {
+ cs.TS_DART_LABEL,
+ cs.TS_DART_UNCONDITIONAL_ASSIGNABLE_SELECTOR,
+ cs.TS_DART_CONDITIONAL_ASSIGNABLE_SELECTOR,
+ cs.TS_DART_CASCADE_SELECTOR,
+ cs.TS_DART_CATCH_PARAMETERS,
+ }
+ )
+ | cs.DART_SIGNATURE_TYPES
+)
+
+
+def _dart_scope_span(decl: Node, walk_root: Node) -> tuple[int, int]:
+ # The byte span a declaration shadows: its nearest enclosing scope, with
+ # statement scopes (for/try) starting at the declaration END because the
+ # for-in iterable and the try body precede their binder. No scope
+ # ancestor (a signature parameter) falls through to the whole body.
+ anc = decl.parent
+ while anc is not None and anc is not walk_root:
+ if anc.type in _DART_SHADOW_SCOPE_TYPES:
+ if anc.type in _DART_STATEMENT_SCOPE_TYPES:
+ return (decl.end_byte, anc.end_byte)
+ return (anc.start_byte, anc.end_byte)
+ anc = anc.parent
+ return (walk_root.start_byte, walk_root.end_byte)
+
+
+def _dart_is_owned_function_body(node: Node) -> bool:
+ # A function_body belonging to a top-level function or class member has
+ # its OWN caller pass; a closure's or local function's body does not (a
+ # Dart lambda's reads attribute to the enclosing scope), so only bodies
+ # NOT under a nested-scope node are skipped by the module walks.
+ return (
+ node.type == cs.TS_DART_FUNCTION_BODY
+ and (parent := node.parent) is not None
+ and parent.type not in cs.DART_NESTED_SCOPE_NODE_TYPES
+ )
+
+
+def _dart_declared_names(node: Node) -> list[str]:
+ # The name(s) a parameter, local declaration, loop variable, catch
+ # parameter, or pattern binding binds; anything else declares nothing.
+ node_type = node.type
+ if node_type in (cs.TS_DART_FORMAL_PARAMETER, cs.TS_DART_CATCH_PARAMETERS):
+ return [
+ name
+ for child in node.named_children
+ if child.type == cs.TS_DART_IDENTIFIER and (name := safe_decode_text(child))
+ ]
+ if node_type in _DART_LOCAL_DECLARATION_TYPES:
+ declared = next(
+ (
+ child
+ for child in node.named_children
+ if child.type == cs.TS_DART_IDENTIFIER
+ ),
+ None,
+ )
+ if declared is not None and (name := safe_decode_text(declared)):
+ return [name]
+ if node_type == cs.TS_DART_FOR_LOOP_PARTS:
+ # `for (final total in xs)`: the FIRST identifier is the loop
+ # variable, the second the iterable expression.
+ first = next(
+ (
+ child
+ for child in node.named_children
+ if child.type == cs.TS_DART_IDENTIFIER
+ ),
+ None,
+ )
+ if first is not None and (name := safe_decode_text(first)):
+ return [name]
+ if node_type == cs.TS_DART_PATTERN_VARIABLE_DECLARATION:
+ return _dart_pattern_bound_names(node)
+ return []
+
+
+def _dart_pattern_bound_names(node: Node) -> list[str]:
+ # `var (alpha, beta) = rhs` binds every identifier inside the *_pattern
+ # subtree; the RHS expression binds nothing.
+ names: list[str] = []
+ stack = [
+ child
+ for child in node.named_children
+ if child.type.endswith(cs.DART_PATTERN_NODE_SUFFIX)
+ ]
+ while stack:
+ current = stack.pop()
+ if current.type == cs.TS_DART_IDENTIFIER and (
+ name := safe_decode_text(current)
+ ):
+ names.append(name)
+ stack.extend(current.named_children)
+ return names
+
+
+def _dart_is_bare_read(node: Node) -> bool:
+ # A bare-identifier getter read, INCLUDING a receiver-position chain head:
+ # `_wonders.length` reads the `_wonders` getter even though the member
+ # pass only emits for the final member (issue #873). Only a head invoked
+ # directly (`f(x)` = identifier + selector(argument_part)) belongs to the
+ # call pass; a label's identifier names a parameter, and a selector's
+ # own identifier is the member the chain pass already handled.
+ following = node.next_named_sibling
+ if following is not None and (
+ following.type == cs.TS_DART_ARGUMENT_PART
+ or (
+ following.type == cs.TS_DART_SELECTOR
+ and any(
+ child.type == cs.TS_DART_ARGUMENT_PART
+ for child in following.named_children
+ )
+ )
+ ):
+ return False
+ parent = node.parent
+ if parent is None or parent.type in _DART_NON_READ_PARENT_TYPES:
+ return False
+ if parent.type == cs.TS_DART_FOR_LOOP_PARTS:
+ # The FIRST identifier of a for-in is the loop BINDER, not a read;
+ # the iterable position after it reads normally.
+ first = next(
+ (
+ child
+ for child in parent.named_children
+ if child.type == cs.TS_DART_IDENTIFIER
+ ),
+ None,
+ )
+ if first is not None and first.start_byte == node.start_byte:
+ return False
+ return True
+
+
+# Python nested-scope boundaries and sequence-literal node types used when
+# scanning a scope for dispatch tables of function references.
+_PY_SCOPE_BOUNDARY_TYPES = frozenset(
+ {
+ cs.TS_PY_FUNCTION_DEFINITION,
+ cs.TS_PY_CLASS_DEFINITION,
+ cs.TS_PY_DECORATED_DEFINITION,
+ }
+)
+_PY_SEQUENCE_LITERAL_TYPES = frozenset({cs.TS_PY_LIST, cs.TS_PY_SET, cs.TS_PY_TUPLE})
+# Dispatch-table literals whose values may name handler functions: Python dict
+# and JS/TS object (key/value pairs), Python list/set/tuple and JS/TS array
+# (positional elements). All use the `pair`/named-child shapes handled below.
+_DICT_LIKE_COLLECTION_TYPES = frozenset({cs.TS_PY_DICTIONARY, cs.TS_OBJECT})
+_SEQUENCE_LIKE_COLLECTION_TYPES = _PY_SEQUENCE_LITERAL_TYPES | frozenset({cs.TS_ARRAY})
+# Python nodes that transparently wrap first-class values one level down:
+# sequence literals, a bare multi-value return (expression_list), and
+# parentheses. Dict pairs and ternaries need field-aware handling, matched
+# separately in _expand_py_first_class_values.
+_PY_VALUE_WRAPPER_TYPES = _PY_SEQUENCE_LITERAL_TYPES | frozenset(
+ {cs.TS_PY_EXPRESSION_LIST, cs.TS_PARENTHESIZED_EXPRESSION}
+)
+_CALLABLE_NODE_LABELS = (
+ cs.NodeLabel.FUNCTION,
+ cs.NodeLabel.METHOD,
+ cs.NodeLabel.CLASS,
+)
+# Node types of a call argument that may name a callable: a bare identifier
+# (Python/Go/JS/TS), a Python attribute (self.method), a Go selector (x.Method),
+# or a JS/TS member expression (obj.method).
+_FLOW_ARG_REF_TYPES = frozenset(
+ {
+ cs.TS_PY_IDENTIFIER,
+ cs.TS_PY_ATTRIBUTE,
+ cs.TS_SELECTOR_EXPRESSION,
+ cs.TS_MEMBER_EXPRESSION,
+ }
+)
+# Qualified-name prefix marking a resolved callee as a builtin rather than a
+# first-party function whose body the call chain can be followed into.
+_BUILTIN_QN_PREFIX = f"{cs.BUILTIN_PREFIX}{cs.SEPARATOR_DOT}"
+# C/C++ expression nodes whose call name is a synthesized operator_*, the
+# ones whose operand type may direct or suppress the binding.
+_CPP_OPERATOR_EXPRESSION_TYPES = frozenset(
+ {
+ cs.TS_CPP_BINARY_EXPRESSION,
+ cs.TS_CPP_UNARY_EXPRESSION,
+ cs.TS_CPP_UPDATE_EXPRESSION,
+ }
+)
+# Transparent wrappers a bound arrow may sit behind in its declarator:
+# parens and TS casts (`const f = ((x) => ...) as T`). Climbed when
+# recovering the arrow's binding name so it is not treated as anonymous.
+_TS_BINDING_WRAPPER_TYPES = cs.TS_CAST_WRAPPER_TYPES | {cs.TS_PARENTHESIZED_EXPRESSION}
+# Assignment node type -> RHS field, per language family: Python `assignment`
+# and JS/TS `assignment_expression` (client.post = fn) carry the RHS in
+# `right`; a JS/TS `variable_declarator` (const cb = handler) carries it in
+# `value`. Go binds a func value the same way; its RHS sits behind an
+# expression_list, unwrapped in the walker.
+_ASSIGNMENT_RHS_FIELDS = {
+ cs.TS_PY_ASSIGNMENT: cs.TS_FIELD_RIGHT,
+ cs.TS_ASSIGNMENT_EXPRESSION: cs.TS_FIELD_RIGHT,
+ cs.TS_VARIABLE_DECLARATOR: cs.FIELD_VALUE,
+ cs.TS_GO_VAR_SPEC: cs.FIELD_VALUE,
+ cs.TS_GO_SHORT_VAR_DECLARATION: cs.TS_FIELD_RIGHT,
+ cs.TS_GO_ASSIGNMENT_STATEMENT: cs.TS_FIELD_RIGHT,
+}
+# RHS node types that name a callable value: a bare identifier, a Python
+# attribute (mod.fn), a JS/TS member expression (handlers.run), or a Go
+# selector (pkg.Fn).
+_ASSIGNMENT_RHS_REF_TYPES = frozenset(
+ {
+ cs.TS_PY_IDENTIFIER,
+ cs.TS_PY_ATTRIBUTE,
+ cs.TS_MEMBER_EXPRESSION,
+ cs.TS_GO_SELECTOR_EXPRESSION,
+ }
+)
+# JSX nodes that carry a component name (self-closing and opening; a
+# closing element repeats the name, so a paired element emits once).
+_JSX_NAMED_ELEMENT_TYPES = frozenset(
+ {cs.TS_JSX_SELF_CLOSING_ELEMENT, cs.TS_JSX_OPENING_ELEMENT}
+)
+# Inline function values in an object literal (`{ onSuccess: () => {} }`): the
+# JS/TS definition pass registers these as their own nodes named by the key
+# (scope.onSuccess), so a passed object of callbacks must reference each or
+# every TanStack-style callback reports as dead.
+_INLINE_FUNC_VALUE_TYPES = frozenset({cs.TS_ARROW_FUNCTION, cs.TS_FUNCTION_EXPRESSION})
+
+
+def _scope_qn_candidates(scope_qn: str) -> list[str]:
+ # The scope itself plus its duplicate-variant-stripped form (`useStore@27`
+ # -> `useStore`): the def pass registers nested/anon members under the
+ # NATURAL qn while the caller may carry the variant suffix. Registry-guarded,
+ # so a scope without a twin adds nothing.
+ last = scope_qn.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ if cs.DUP_QN_MARKER not in last:
+ return [scope_qn]
+ natural = scope_qn[: len(scope_qn) - len(last)] + last.split(cs.DUP_QN_MARKER, 1)[0]
+ return [scope_qn, natural]
+
+
+def _first_class_value_children(node: Node, is_dart: bool) -> list[Node] | None:
+ # The wrapped value nodes a container/branch node hands onward, or None
+ # for a leaf that IS the first-class value.
+ if node.type in _PY_VALUE_WRAPPER_TYPES:
+ return list(node.named_children)
+ if is_dart and node.type in _DART_VALUE_WRAPPER_TYPES:
+ return _dart_collection_values(node)
+ if node.type == cs.TS_PY_DICTIONARY:
+ return _py_dict_values(node)
+ if node.type == cs.TS_PY_CONDITIONAL_EXPRESSION:
+ return _conditional_result_operands(node, is_dart)
+ if (
+ is_dart
+ and node.type.endswith(cs.DART_EXPRESSION_NODE_SUFFIX)
+ # A scope-opening node (function_expression) IS the first-class
+ # value; only flat operator nodes hand a swallowed ternary onward.
+ and node.type not in cs.DART_NESTED_SCOPE_NODE_TYPES
+ and (kids := node.named_children)
+ and kids[-1].type == cs.TS_PY_CONDITIONAL_EXPRESSION
+ ):
+ # tree-sitter-dart parses low-precedence operators OVER the ternary
+ # (`a + b > 0 ? f : null` -> additive_expression(a, +,
+ # conditional_expression(...))), swallowing the conditional as the
+ # LAST child of the operator node; its result operands are still the
+ # handed-over values (issue #873).
+ return [kids[-1]]
+ if node.type == cs.TS_PY_BOOLEAN_OPERATOR:
+ return [
+ operand
+ for operand in (
+ node.child_by_field_name(cs.TS_FIELD_LEFT),
+ node.child_by_field_name(cs.TS_FIELD_RIGHT),
+ )
+ if operand is not None
+ ]
+ return None
+
+
+def _dart_collection_values(node: Node) -> list[Node]:
+ # A Dart MAP stores each value inside a `pair` node's value field
+ # (`{"tap": onTap}`); a set exposes its elements directly; a typed
+ # literal's `type_arguments` child carries types, never values.
+ children: list[Node] = []
+ for child in node.named_children:
+ if child.type == cs.TS_DART_TYPE_ARGUMENTS:
+ continue
+ if child.type == cs.TS_DART_PAIR:
+ if (pair_value := child.child_by_field_name(cs.FIELD_VALUE)) is not None:
+ children.append(pair_value)
+ continue
+ children.append(child)
+ return children
+
+
+def _py_dict_values(node: Node) -> list[Node]:
+ return [
+ pair_value
+ for pair in node.named_children
+ if pair.type == cs.TS_PY_PAIR
+ and (pair_value := pair.child_by_field_name(cs.FIELD_VALUE)) is not None
+ ]
+
+
+def _conditional_result_operands(node: Node, is_dart: bool) -> list[Node]:
+ # tree-sitter-python exposes NO field names on conditional_expression
+ # (child_by_field_name returns None for every operand), so the result
+ # operands are positional: [body, condition, alternative]. A shape that
+ # is not exactly three named operands falls back to all of them,
+ # over-referencing rather than dropping a branch.
+ operands = list(node.named_children)
+ if len(operands) != 3:
+ return operands
+ # Dart orders [condition, consequence, alternative]; Python orders
+ # [body, condition, alternative]. Pick the two result operands, never
+ # the truthiness-tested one.
+ return [operands[1], operands[2]] if is_dart else [operands[0], operands[2]]
+
+
+def _find_call_arguments_node(call_node: Node) -> Node | None:
+ args_node = call_node.child_by_field_name(cs.FIELD_ARGUMENTS)
+ if args_node is not None:
+ return args_node
+ # C# target-typed `new(...)` exposes NO fields at all; its argument_list
+ # is an unfielded named child (Serilog hands its CreateLogger local
+ # functions to `return new(...)`, which the fielded lookup silently
+ # dropped).
+ args_node = next(
+ (
+ child
+ for child in call_node.named_children
+ if child.type == cs.TS_CSHARP_ARGUMENT_LIST
+ ),
+ None,
+ )
+ if args_node is not None:
+ return args_node
+ # Dart has no call-expression node: a selector/cascade_section wraps its
+ # arguments in an argument_part holding the real `arguments` node one
+ # level down.
+ for part in call_node.named_children:
+ if part.type == cs.TS_DART_ARGUMENT_PART:
+ return next(
+ (
+ grand
+ for grand in part.named_children
+ if grand.type == cs.TS_DART_ARGUMENTS
+ ),
+ None,
+ )
+ return None
+
+
+def _add_dart_named_argument(child: Node, keyword: dict[str, Node]) -> None:
+ # A named value is a `named_argument` whose label child names it and
+ # whose last named child is the expression.
+ label = next(
+ (grand for grand in child.named_children if grand.type == cs.TS_DART_LABEL),
+ None,
+ )
+ value_node = child.named_children[-1] if child.named_children else None
+ name_node = (
+ label.named_children[0] if label is not None and label.named_children else None
+ )
+ if (
+ name_node is not None
+ and value_node is not None
+ and value_node is not label
+ and (name := safe_decode_text(name_node)) is not None
+ ):
+ keyword[name] = value_node
+
+
+def _add_py_keyword_argument(child: Node, keyword: dict[str, Node]) -> None:
+ name_node = child.child_by_field_name(cs.FIELD_NAME)
+ value_node = child.child_by_field_name(cs.FIELD_VALUE)
+ if (
+ name_node is not None
+ and value_node is not None
+ and (name := safe_decode_text(name_node)) is not None
+ ):
+ keyword[name] = value_node
class CallProcessor:
+ __slots__ = (
+ "ingestor",
+ "repo_path",
+ "project_name",
+ "module_qn_to_file_path",
+ "_path_to_module_qn",
+ "cpp_out_of_class_methods",
+ "function_locations",
+ "macro_qns",
+ "_resolver",
+ "_flow_param_names",
+ "_flow_args",
+ "_returned_callables",
+ "_factory_calls",
+ "_io_processor",
+ "_flow_processor",
+ )
+
def __init__(
self,
ingestor: IngestorProtocol,
@@ -27,16 +543,52 @@ def __init__(
import_processor: ImportProcessor,
type_inference: TypeInferenceEngine,
class_inheritance: dict[str, list[str]],
+ type_aliases: dict[str, str] | None = None,
+ interface_implementers: dict[str, set[str]] | None = None,
+ capture: CaptureSelection | None = None,
+ module_qn_to_file_path: dict[str, Path] | None = None,
+ cpp_out_of_class_methods: dict[tuple[str, int], tuple[str, str]] | None = None,
+ function_locations: dict[FunctionSpanKey, FunctionLocation] | None = None,
+ macro_qns: set[str] | None = None,
) -> None:
self.ingestor = ingestor
self.repo_path = repo_path
self.project_name = project_name
+ self.module_qn_to_file_path = module_qn_to_file_path or {}
+ self._path_to_module_qn: dict[Path, str] | None = None
+ self.cpp_out_of_class_methods = cpp_out_of_class_methods or {}
+ self.function_locations = function_locations or {}
+ self.macro_qns = macro_qns if macro_qns is not None else set()
self._resolver = CallResolver(
function_registry=function_registry,
import_processor=import_processor,
type_inference=type_inference,
class_inheritance=class_inheritance,
+ type_aliases=type_aliases,
+ interface_implementers=interface_implementers,
+ )
+ # Inter-procedural callable-parameter flow: ordered params per function and
+ # the per-call-site argument bindings, resolved to a fixpoint in finalize.
+ self._flow_param_names: dict[str, list[str]] = {}
+ self._flow_args: list[_CallableFlowArg] = []
+ # Return-value / factory tracing: functions each function may return
+ # (nested closures), and call sites `x = factory(...); x(cb)` where cb
+ # flows into the returned closure's callable parameter. Resolved to a
+ # fixpoint in finalize, so factory and call site may be in any order.
+ self._returned_callables: dict[str, set[str]] = {}
+ self._factory_calls: list[_FactoryCall] = []
+ selection = capture if capture is not None else ALL_ENABLED
+ self._io_processor = IOAccessProcessor(
+ ingestor,
+ import_processor,
+ selection=selection,
+ )
+ self._flow_processor = FlowProcessor(
+ ingestor,
+ import_processor,
+ self._resolver,
+ selection=selection,
)
def _get_node_name(self, node: Node, field: str = cs.FIELD_NAME) -> str | None:
@@ -46,60 +598,784 @@ def _get_node_name(self, node: Node, field: str = cs.FIELD_NAME) -> str | None:
text = name_node.text
return None if text is None else text.decode(cs.ENCODING_UTF8)
+ def _collect_all_call_nodes(
+ self,
+ root_node: Node,
+ language: cs.SupportedLanguage,
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
+ ) -> tuple[list[Node], list[int]]:
+ calls_query = queries[language].get(cs.QUERY_CALLS)
+ if not calls_query:
+ return [], []
+ cursor = QueryCursor(calls_query)
+ captures = sorted_captures(cursor, root_node)
+ call_nodes = captures.get(cs.CAPTURE_CALL, [])
+ call_starts = [n.start_byte for n in call_nodes]
+ return call_nodes, call_starts
+
+ def _filtered_calls_for(
+ self,
+ func_node: Node,
+ language: cs.SupportedLanguage,
+ all_call_nodes: list[Node] | None,
+ call_starts: list[int] | None,
+ owned_func_nodes: list[Node],
+ ) -> list[Node] | None:
+ # The caller's own calls, sliced from the module-wide call list. A Rust
+ # named nested fn owns its body's calls, so its slice must exclude them
+ # rather than take the whole byte range; every other language filters by
+ # the plain byte span.
+ if all_call_nodes is None or call_starts is None:
+ return None
+ if language == cs.SupportedLanguage.RUST:
+ return self._calls_owned_by(
+ func_node, owned_func_nodes, all_call_nodes, call_starts
+ )
+ return self._filter_calls_in_node(all_call_nodes, call_starts, func_node)
+
+ def _filter_calls_in_node(
+ self,
+ all_call_nodes: list[Node],
+ call_starts: list[int],
+ container: Node,
+ ) -> list[Node]:
+ start = container.start_byte
+ end = container.end_byte
+ # A Dart definition is a signature node whose body is a SIBLING;
+ # widen to the body's end or every body call escapes its caller.
+ if container.type in cs.DART_SIGNATURE_TYPES:
+ end = dart_utils.dart_definition_end_byte(container)
+ lo = bisect_left(call_starts, start)
+ hi = bisect_right(call_starts, end)
+ return [n for n in all_call_nodes[lo:hi] if n.end_byte <= end]
+
+ def _filter_top_level_calls(
+ self,
+ all_call_nodes: list[Node],
+ call_starts: list[int],
+ func_nodes: list[Node],
+ ) -> list[Node]:
+ # Calls inside a function's BODY belong to that function, not the
+ # module; only genuine top-level calls are module-attributed. The body
+ # (not the whole node) is the boundary so def-time calls in the
+ # signature (default args like `def f(x=make_default())` and
+ # decorators) run at module load and stay module-attributed. A node
+ # with no body is not a real function scope (e.g. a file-scope
+ # declaration `int x = top();` captured as a function); its calls run
+ # at load time, so it excludes nothing.
+ nested_starts: set[int] = set()
+ for func_node in func_nodes:
+ body = func_node.child_by_field_name(cs.FIELD_BODY)
+ if body is None:
+ # a Dart body is a SIBLING of its signature, not a field
+ body = dart_utils.dart_body_node(func_node)
+ if body is None:
+ continue
+ for call in self._filter_calls_in_node(all_call_nodes, call_starts, body):
+ nested_starts.add(call.start_byte)
+ return [c for c in all_call_nodes if c.start_byte not in nested_starts]
+
+ def _bare_decorator_name(self, decorator_node: Node) -> str | None:
+ # A bare decorator `@task` / `@pkg.deco` (no call parens) is not a
+ # `call` node, so the normal call pass misses it even though applying
+ # it runs `task(func)` at module load. A call decorator `@deco(...)`
+ # IS already captured, so skip it here.
+ named = decorator_node.named_children
+ if not named:
+ return None
+ expr = named[0]
+ if expr.type in (cs.TS_IDENTIFIER, cs.TS_ATTRIBUTE) and expr.text is not None:
+ return expr.text.decode(cs.ENCODING_UTF8)
+ return None
+
+ def _runs_at_module_load(self, node: Node) -> bool:
+ # A definition runs at module load only at module or class-body scope;
+ # nested inside a function body it runs at that function's call time,
+ # so its decorator is not a module-load call.
+ ancestor = node.parent
+ while ancestor is not None:
+ if ancestor.type == cs.TS_PY_FUNCTION_DEFINITION:
+ return False
+ ancestor = ancestor.parent
+ return True
+
+ def _ingest_decorator_calls(
+ self,
+ nodes: list[Node],
+ module_qn: str,
+ root_node: Node,
+ lang_config: LanguageSpec,
+ ) -> None:
+ # Emit `(Module)->decorator` CALLS for bare decorators on functions,
+ # methods, AND classes: the decoration executes at module-load time,
+ # so the module is the caller. Only first-party callables get an edge.
+ resolver = self._resolver
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ qn_key = cs.KEY_QUALIFIED_NAME
+ module_spec = (cs.NodeLabel.MODULE, qn_key, module_qn)
+ callable_labels = (cs.NodeLabel.FUNCTION, cs.NodeLabel.METHOD)
+ alias_map: dict[str, str] | None = None
+ for node in nodes:
+ parent = node.parent
+ if parent is None or parent.type != cs.TS_PY_DECORATED_DEFINITION:
+ continue
+ if not self._runs_at_module_load(parent):
+ continue
+ for child in parent.children:
+ if child.type != cs.TS_PY_DECORATOR:
+ continue
+ name = self._bare_decorator_name(child)
+ if not name:
+ continue
+ callee = resolver.resolve_function_call(name, module_qn)
+ if not callee and cs.SEPARATOR_DOT not in name:
+ # `@alias` where `alias = task` still calls task at load;
+ # reuse the local-alias fallback the call pass uses.
+ if alias_map is None:
+ alias_map = self._build_local_alias_map(
+ root_node, lang_config, module_qn
+ )
+ if (rhs := alias_map.get(name)) is not None:
+ callee = resolver.resolve_function_call(rhs, module_qn)
+ if callee and callee[0] in callable_labels:
+ ensure_rel(
+ module_spec,
+ cs.RelationshipType.CALLS,
+ (callee[0], qn_key, callee[1]),
+ )
+
+ def _module_qn(self, file_path: Path, relative_path: Path) -> str:
+ # The definition pass is the single source of truth for module qns:
+ # same-stem siblings (Aggregator.cpp / Aggregator.h) get a
+ # collision-disambiguated qn there, and recomputing from the path
+ # here attributed every caller inside such a header to a module qn
+ # with no node, silently dropping its CALLS (issue #652).
+ if self._path_to_module_qn is None:
+ self._path_to_module_qn = {
+ path: qn for qn, path in self.module_qn_to_file_path.items()
+ }
+ if registered := self._path_to_module_qn.get(file_path):
+ return registered
+ file_name = file_path.name
+ if file_name in (cs.INIT_PY, cs.MOD_RS):
+ return cs.SEPARATOR_DOT.join(
+ [self.project_name] + list(relative_path.parent.parts)
+ )
+ return cs.SEPARATOR_DOT.join(
+ [self.project_name] + list(relative_path.with_suffix("").parts)
+ )
+
+ def collect_callable_field_bindings(
+ self,
+ file_path: Path,
+ root_node: Node,
+ language: cs.SupportedLanguage,
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
+ func_class_captures_cache: dict[Path, dict] | None = None,
+ ) -> None:
+ # Pre-pass: record which functions are bound to a class's callable
+ # fields (FQNSpec(get_name=_python_get_name, ...)). Runs before call
+ # resolution so a field invocation resolves regardless of which file
+ # the construction site lives in. Bindings are recorded PENDING
+ # (keyword name or positional index) and resolved by
+ # finalize_field_bindings after every file's ctor metadata (param
+ # order + param->attribute renames) is collected.
+ if language != cs.SupportedLanguage.PYTHON:
+ return
+ try:
+ module_qn = self._module_qn(
+ file_path, cached_relative_path(file_path, self.repo_path)
+ )
+ resolver = self._resolver
+ self._collect_ctor_field_metadata(root_node, module_qn)
+ if (
+ func_class_captures_cache is not None
+ and file_path in func_class_captures_cache
+ ):
+ call_nodes = func_class_captures_cache[file_path].get(cs.CAPTURE_CALL)
+ else:
+ call_nodes = None
+ if call_nodes is None:
+ call_nodes, _ = self._collect_all_call_nodes(
+ root_node, language, queries
+ )
+ registry = resolver.function_registry
+ callable_labels = (cs.NodeLabel.FUNCTION, cs.NodeLabel.METHOD)
+ for call_node in call_nodes:
+ positional, keyword = self._parse_call_arguments(call_node)
+ if not positional and not keyword:
+ continue
+ name = self._get_call_target_name(call_node)
+ if not name:
+ continue
+ callee = resolver.resolve_function_call(name, module_qn)
+ if not callee or callee[0] != cs.NodeLabel.CLASS:
+ continue
+ arg_entries: list[tuple[int | str, Node]] = list(enumerate(positional))
+ arg_entries.extend(keyword.items())
+ for key, value_node in arg_entries:
+ if not (value_text := safe_decode_text(value_node)):
+ continue
+ bound = resolver.resolve_function_call(value_text, module_qn)
+ if bound and bound[0] in callable_labels and bound[1] in registry:
+ resolver.record_pending_field_binding(callee[1], key, bound[1])
+ except Exception as e:
+ logger.error(ls.CALL_PROCESSING_FAILED, path=file_path, error=e)
+
+ def finalize_callable_field_bindings(self) -> None:
+ self._resolver.finalize_field_bindings()
+
+ def _collect_ctor_field_metadata(self, root_node: Node, module_qn: str) -> None:
+ # For every class in this file, record the ordered ctor param names
+ # (__init__ params, or annotated class-body fields for
+ # NamedTuple/dataclass classes without __init__) and the
+ # param -> attribute renames from `self.attr = param` statements.
+ # The stack carries each node's ENCLOSING class qn so a nested class
+ # (Inner inside Outer) resolves to module.Outer.Inner: a bare
+ # resolve_function_call("Inner", module_qn) would miss it. A class
+ # inside a FUNCTION is skipped (qn is caller-scoped), so
+ # nested-in-method classes never reach here.
+ resolver = self._resolver
+ registry = resolver.function_registry
+ stack: list[tuple[Node, str | None]] = [
+ (child, None) for child in root_node.children
+ ]
+ while stack:
+ node, enclosing_qn = stack.pop()
+ if node.type == cs.TS_PY_DECORATED_DEFINITION:
+ stack.extend((c, enclosing_qn) for c in node.children)
+ continue
+ if node.type == cs.TS_PY_FUNCTION_DEFINITION:
+ continue
+ if node.type != cs.TS_PY_CLASS_DEFINITION:
+ stack.extend((c, enclosing_qn) for c in node.children)
+ continue
+ name_node = node.child_by_field_name(cs.FIELD_NAME)
+ class_name = safe_decode_text(name_node) if name_node else None
+ body = node.child_by_field_name(cs.FIELD_BODY)
+ if not class_name or body is None:
+ continue
+ if enclosing_qn is not None:
+ candidate_qn = f"{enclosing_qn}{cs.SEPARATOR_DOT}{class_name}"
+ class_qn = (
+ candidate_qn
+ if registry.get(candidate_qn) == cs.NodeLabel.CLASS
+ else None
+ )
+ else:
+ resolved = resolver.resolve_function_call(class_name, module_qn)
+ class_qn = (
+ resolved[1]
+ if resolved and resolved[0] == cs.NodeLabel.CLASS
+ else None
+ )
+ # Descend into the body with THIS class as the enclosing scope so
+ # its nested classes resolve; skip metadata when unresolved.
+ stack.extend((c, class_qn) for c in body.children)
+ if class_qn is None:
+ continue
+ if init_node := self._find_init_method(body):
+ params = self._ordered_param_names(init_node)
+ resolver.record_ctor_params(class_qn, params)
+ self._record_param_attr_renames(init_node, class_qn, set(params))
+ else:
+ resolver.record_ctor_params(class_qn, self._annotated_field_names(body))
+
+ @staticmethod
+ def _find_init_method(class_body: Node) -> Node | None:
+ for child in class_body.children:
+ candidate = child
+ if candidate.type == cs.TS_PY_DECORATED_DEFINITION:
+ candidate = (
+ candidate.child_by_field_name(cs.FIELD_DEFINITION) or candidate
+ )
+ if candidate.type != cs.TS_PY_FUNCTION_DEFINITION:
+ continue
+ name_node = candidate.child_by_field_name(cs.FIELD_NAME)
+ if name_node is not None and safe_decode_text(name_node) == (
+ cs.PY_METHOD_INIT
+ ):
+ return candidate
+ return None
+
+ @staticmethod
+ def _ordered_param_names(init_node: Node) -> tuple[str, ...]:
+ params_node = init_node.child_by_field_name(cs.FIELD_PARAMETERS)
+ if params_node is None:
+ return ()
+ names: list[str] = []
+ for child in params_node.named_children:
+ match child.type:
+ case cs.TS_PY_IDENTIFIER:
+ name = safe_decode_text(child)
+ case cs.TS_PY_DEFAULT_PARAMETER | cs.TS_PY_TYPED_DEFAULT_PARAMETER:
+ name_node = child.child_by_field_name(cs.FIELD_NAME)
+ name = safe_decode_text(name_node) if name_node else None
+ case cs.TS_PY_TYPED_PARAMETER:
+ inner = child.named_children[0] if child.named_children else None
+ name = (
+ safe_decode_text(inner)
+ if inner is not None and inner.type == cs.TS_PY_IDENTIFIER
+ else None
+ )
+ case _:
+ # *args/**kwargs/keyword_separator never bind fields.
+ name = None
+ if name and name != cs.PY_KEYWORD_SELF:
+ names.append(name)
+ return tuple(names)
+
+ def _record_param_attr_renames(
+ self, init_node: Node, class_qn: str, params: set[str]
+ ) -> None:
+ # `self.ctx_factory = create_context` stores the param under a
+ # DIFFERENT name; the field invocation goes through the attribute.
+ body = init_node.child_by_field_name(cs.FIELD_BODY)
+ if body is None:
+ return
+ self_prefix = f"{cs.PY_KEYWORD_SELF}{cs.SEPARATOR_DOT}"
+ stack: list[Node] = list(body.children)
+ while stack:
+ node = stack.pop()
+ # A `self.x = param` inside a nested helper (def later():
+ # self.cb = handler) is that helper's store, not a constructor
+ # rename; descending would let it override the real __init__
+ # store, so stop at nested scopes.
+ if node.type in _PY_SCOPE_BOUNDARY_TYPES:
+ continue
+ if node.type == cs.TS_PY_ASSIGNMENT:
+ left = node.child_by_field_name(cs.TS_FIELD_LEFT)
+ right = node.child_by_field_name(cs.TS_FIELD_RIGHT)
+ if (
+ left is not None
+ and right is not None
+ and left.type == cs.TS_PY_ATTRIBUTE
+ and right.type == cs.TS_PY_IDENTIFIER
+ and (left_text := safe_decode_text(left))
+ and left_text.startswith(self_prefix)
+ and (param := safe_decode_text(right)) in params
+ ):
+ attr = left_text[len(self_prefix) :]
+ if attr != param:
+ self._resolver.record_ctor_param_attr(class_qn, param, attr)
+ stack.extend(node.children)
+
+ @staticmethod
+ def _annotated_field_names(class_body: Node) -> tuple[str, ...]:
+ # NamedTuple/dataclass field order = annotated class-body assignments
+ # (`fetch_name: Callable`, `other: int = 3`) in declaration order.
+ names: list[str] = []
+ for child in class_body.children:
+ if child.type != cs.TS_PY_EXPRESSION_STATEMENT or not child.named_children:
+ continue
+ stmt = child.named_children[0]
+ if stmt.type != cs.TS_PY_ASSIGNMENT:
+ continue
+ left = stmt.child_by_field_name(cs.TS_FIELD_LEFT)
+ has_type = stmt.child_by_field_name(cs.FIELD_TYPE) is not None
+ if (
+ left is not None
+ and left.type == cs.TS_PY_IDENTIFIER
+ and has_type
+ and (name := safe_decode_text(left))
+ ):
+ names.append(name)
+ return tuple(names)
+
def process_calls_in_file(
self,
file_path: Path,
root_node: Node,
language: cs.SupportedLanguage,
- queries: dict[cs.SupportedLanguage, LanguageQueries],
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
+ func_class_captures_cache: dict[Path, dict] | None = None,
) -> None:
- relative_path = file_path.relative_to(self.repo_path)
- logger.debug(ls.CALL_PROCESSING_FILE.format(path=relative_path))
+ relative_path = cached_relative_path(file_path, self.repo_path)
+ logger.debug(ls.CALL_PROCESSING_FILE, path=relative_path)
try:
- module_qn = cs.SEPARATOR_DOT.join(
- [self.project_name] + list(relative_path.with_suffix("").parts)
- )
- if file_path.name in (cs.INIT_PY, cs.MOD_RS):
- module_qn = cs.SEPARATOR_DOT.join(
- [self.project_name] + list(relative_path.parent.parts)
+ module_qn = self._module_qn(file_path, relative_path)
+
+ call_name_cache: dict[int, str | None] = {}
+
+ if (
+ func_class_captures_cache is not None
+ and file_path in func_class_captures_cache
+ ):
+ combined_captures = func_class_captures_cache[file_path]
+ else:
+ combined_query = COMBINED_FUNC_CLASS_QUERIES.get(language)
+ if combined_query:
+ cursor = QueryCursor(combined_query)
+ combined_captures = sorted_captures(cursor, root_node)
+ else:
+ combined_captures = {}
+
+ cached_calls = combined_captures.get(cs.CAPTURE_CALL)
+ if cached_calls is not None:
+ all_call_nodes = cached_calls
+ call_starts: list[int] | None = None
+ else:
+ all_call_nodes, call_starts = self._collect_all_call_nodes(
+ root_node, language, queries
+ )
+
+ sorted_func_nodes = combined_captures.get(cs.CAPTURE_FUNCTION)
+ if sorted_func_nodes or combined_captures.get(cs.CAPTURE_CLASS):
+ if cached_calls is not None:
+ call_starts = [n.start_byte for n in all_call_nodes]
+ func_node_starts = (
+ [n.start_byte for n in sorted_func_nodes]
+ if sorted_func_nodes
+ else None
)
+ else:
+ call_starts = None
+ func_node_starts = None
- self._process_calls_in_functions(root_node, module_qn, language, queries)
- self._process_calls_in_classes(root_node, module_qn, language, queries)
- self._process_module_level_calls(root_node, module_qn, language, queries)
+ self._process_calls_in_functions(
+ root_node,
+ module_qn,
+ language,
+ queries,
+ all_call_nodes,
+ call_starts,
+ call_name_cache=call_name_cache,
+ combined_captures=combined_captures or None,
+ )
+ # Bare decorators (`@task`) are not call nodes; emit their
+ # module-load CALLS before the empty-`all_call_nodes` early return,
+ # since a file may have decorators but no other calls. Classes can
+ # be decorated too, so include captured class nodes.
+ # A dispatch table (HANDLERS = {"k": fn}) at module scope keeps its
+ # entries reachable; scan before the no-calls early return so a file
+ # that only defines functions and a table is still covered. Runs for
+ # every flow language with an object/array/dict literal form.
+ if language == cs.SupportedLanguage.DART and (
+ dart_prop_names := self._resolver.function_registry.property_names()
+ ):
+ # Field initializers execute at construction time even in a
+ # file holding nothing but classes (no module caller pass
+ # runs there), so each class subtree is scanned at FILE
+ # level, before the no-calls early return.
+ dart_config = queries[language][cs.QUERY_CONFIG]
+ module_spec = (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn)
+ for child in root_node.named_children:
+ if child.type in dart_config.class_node_types:
+ self._ingest_dart_class_initializer_reads(
+ child,
+ module_spec,
+ module_qn,
+ module_qn,
+ None,
+ dart_config,
+ dart_prop_names,
+ )
+ if language == cs.SupportedLanguage.PYTHON or language in _JS_TS_LANGUAGES:
+ collection_boundaries = self._flow_scope_boundaries(
+ queries[language][cs.QUERY_CONFIG]
+ )
+ if language == cs.SupportedLanguage.PYTHON:
+ # A Python class body executes at import time, so a
+ # dispatch table stored as a class ATTRIBUTE (django's
+ # backend `data_types = {"CharField": _get_varchar_...}`)
+ # is module-load wiring like a module-level table; the scan
+ # descends through classes but still stops at function
+ # scopes. ponytail: decorated classes stay boundaries
+ # (decorated_definition also wraps functions).
+ collection_boundaries = collection_boundaries - frozenset(
+ queries[language][cs.QUERY_CONFIG].class_node_types
+ )
+ self._ingest_collection_function_references(
+ root_node,
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ module_qn,
+ None,
+ None,
+ collection_boundaries,
+ )
+ # A module-scope first-class assignment (registry_handler =
+ # handle_event, module.exports.run = run) references its target
+ # even in a file with no calls, so scan before the no-calls
+ # early return.
+ self._ingest_assignment_function_references(
+ root_node,
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ module_qn,
+ None,
+ None,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ )
+ if language == cs.SupportedLanguage.GO:
+ # A module-scope Go func map/slice (var funcMap = map[...]{...})
+ # keeps its function entries reachable; scan before the no-calls
+ # early return so a file defining only funcs and a table counts.
+ self._ingest_go_composite_function_references(
+ root_node,
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ module_qn,
+ None,
+ None,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ )
+ # A module-scope Go var bound to a bare function value
+ # (var preExecHookFn = preExecHook) references it even in a file
+ # with no calls, so scan before the no-calls early return.
+ self._ingest_assignment_function_references(
+ root_node,
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ module_qn,
+ None,
+ None,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ )
+ if language in _JS_TS_LANGUAGES:
+ # A module-scope JSX element (export default ) can sit
+ # in a file with no call expressions; scan before the early
+ # return like the assignment pass.
+ self._ingest_jsx_component_references(
+ root_node,
+ (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+ module_qn,
+ None,
+ None,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ )
+ if language == cs.SupportedLanguage.PYTHON:
+ decorator_targets = list(sorted_func_nodes or [])
+ if combined_captures and (
+ class_nodes := combined_captures.get(cs.CAPTURE_CLASS)
+ ):
+ decorator_targets.extend(class_nodes)
+ if decorator_targets:
+ self._ingest_decorator_calls(
+ decorator_targets,
+ module_qn,
+ root_node,
+ queries[language][cs.QUERY_CONFIG],
+ )
+ if not all_call_nodes and language not in (
+ cs.SupportedLanguage.CSHARP,
+ cs.SupportedLanguage.CPP,
+ cs.SupportedLanguage.DART,
+ ):
+ # A file with no call expressions has nothing further to
+ # process, except in C#, where a class can still READ
+ # properties (`return Size;`), C++, where a ctor's
+ # member initializer list (`: buffer(g, 0)`) runs base
+ # ctors without any call_expression node, and Dart, where
+ # a getter body can read another getter (`=> _wonders.length`,
+ # issue #873); these passes run per caller inside class
+ # processing, so they proceed.
+ return
+ self._process_calls_in_classes(
+ root_node,
+ module_qn,
+ language,
+ queries,
+ all_call_nodes,
+ call_starts,
+ call_name_cache=call_name_cache,
+ combined_captures=combined_captures,
+ sorted_func_nodes=sorted_func_nodes,
+ func_node_starts=func_node_starts,
+ )
+ if sorted_func_nodes and call_starts is not None:
+ # JS/TS: exclude only calls owned by ATTRIBUTABLE functions, so a
+ # call nested purely in anonymous scopes (`create((set) => ({
+ # inc: () => set((state) => ...) }))`, zustand's store shape) is
+ # processed at module scope instead of by nobody: an anon arrow
+ # gets no caller pass, and a call inside a NAMED function is
+ # still excluded because that function's flat filter owns it.
+ exclusion_nodes = (
+ self._attributable_func_nodes(sorted_func_nodes, language)
+ if language in _JS_TS_LANGUAGES
+ else sorted_func_nodes
+ )
+ module_calls = self._filter_top_level_calls(
+ all_call_nodes, call_starts, exclusion_nodes
+ )
+ else:
+ module_calls = all_call_nodes
+ self._ingest_function_calls(
+ root_node,
+ module_qn,
+ cs.NodeLabel.MODULE,
+ module_qn,
+ language,
+ queries,
+ call_nodes=module_calls,
+ call_name_cache=call_name_cache,
+ )
except Exception as e:
- logger.error(ls.CALL_PROCESSING_FAILED.format(path=file_path, error=e))
+ logger.error(ls.CALL_PROCESSING_FAILED, path=file_path, error=e)
def _process_calls_in_functions(
self,
root_node: Node,
module_qn: str,
language: cs.SupportedLanguage,
- queries: dict[cs.SupportedLanguage, LanguageQueries],
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
+ all_call_nodes: list[Node] | None = None,
+ call_starts: list[int] | None = None,
+ call_name_cache: dict[int, str | None] | None = None,
+ combined_captures: dict[str, list[Node]] | None = None,
) -> None:
- result = get_function_captures(root_node, language, queries)
- if not result:
- return
-
- lang_config, captures = result
- func_nodes = captures.get(cs.CAPTURE_FUNCTION, [])
+ if combined_captures is not None:
+ lang_config = queries[language][cs.QUERY_CONFIG]
+ func_nodes = combined_captures.get(cs.CAPTURE_FUNCTION, [])
+ has_classes = bool(combined_captures.get(cs.CAPTURE_CLASS))
+ else:
+ result = get_function_captures(root_node, language, queries)
+ if not result:
+ return
+ lang_config, captures = result
+ func_nodes = captures.get(cs.CAPTURE_FUNCTION, [])
+ has_classes = bool(captures.get(cs.CAPTURE_CLASS))
+ # Rust only: calls inside a NAMED nested `fn` (which gets its own caller
+ # node) must be owned by that nested fn, not double-counted onto the
+ # enclosing free function. Anonymous closures (not attributable) stay
+ # excluded so their calls still bubble up. Other languages keep the flat
+ # _filter_calls_in_node behavior their flow-tracing relies on.
+ owned_func_nodes = self._attributable_func_nodes(func_nodes, language)
for func_node in func_nodes:
- if not isinstance(func_node, Node):
- continue
- if self._is_method(func_node, lang_config):
+ if has_classes and self._is_method(func_node, lang_config):
continue
- if language == cs.SupportedLanguage.CPP:
+ if language in _C_FAMILY_LANGUAGES:
+ # A macro-invocation artifact the ingest pass declined to
+ # register (no class bears its name) must not become a call
+ # target either; mirror ingest's decision via the recorded
+ # locations so the two passes never diverge.
+ if (
+ language == cs.SupportedLanguage.CPP
+ and cpp_utils.is_macro_invocation_artifact(func_node)
+ and self._recorded_caller(func_node, module_qn) is None
+ ):
+ continue
func_name = cpp_utils.extract_function_name(func_node)
else:
func_name = self._get_node_name(func_node)
+ if not func_name and language in _JS_TS_LANGUAGES:
+ func_name = self._js_ts_arrow_binding_name(func_node)
+ if (
+ not func_name
+ and language == cs.SupportedLanguage.LUA
+ and func_node.type == cs.TS_LUA_FUNCTION_DEFINITION
+ ):
+ # A function expression bound to a variable or table field
+ # (`local f = function()`, `M.f = function()`) has no name field;
+ # the definition pass names it after its assignment target, so
+ # recover the same name here or the whole body is skipped.
+ func_name = lua_utils.extract_assigned_name(
+ func_node,
+ accepted_var_types=(cs.TS_DOT_INDEX_EXPRESSION, cs.TS_IDENTIFIER),
+ )
if not func_name:
+ # A nameless JS/TS function expression that a NAMED pass
+ # registered (`exports.f = function`, `x: function`) has a
+ # real node; its body's calls belong to that node, not the
+ # module, so adopt the record's simple name and fall through
+ # (the recorded-caller branch below reuses the registered qn).
+ # A GENERATED record (anonymous callback, IIFE) keeps the
+ # historical bubble-to-module attribution.
+ if (
+ language not in _JS_TS_LANGUAGES
+ or (recorded := self._recorded_caller(func_node, module_qn)) is None
+ or not recorded.is_named
+ ):
+ continue
+ func_name = recorded.qualified_name.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ # The definition pass records where every function/method node
+ # landed; reuse that qn/label instead of re-deriving them from
+ # the AST -- the walks diverge on preprocessor-distorted C++
+ # class bodies, TS declaration merging, and duplicate-suffixed
+ # qns, and every divergence is a phantom caller (issue #652).
+ if loc := self._recorded_caller(func_node, module_qn):
+ filtered = self._filtered_calls_for(
+ func_node, language, all_call_nodes, call_starts, owned_func_nodes
+ )
+ self._ingest_function_calls(
+ func_node,
+ loc.qualified_name,
+ loc.label,
+ module_qn,
+ language,
+ queries,
+ loc.container_qn,
+ call_nodes=filtered,
+ call_name_cache=call_name_cache,
+ )
+ continue
+ # An out-of-line C++ method definition (`Ret Class::method() {...}`
+ # at namespace/file scope) is bound by the definition pass to its
+ # class node (qn `class_qn.method`). Attribute its body's calls to
+ # that method node, not a phantom module-rooted qn, so the CALLS
+ # edges join to a real node.
+ if language == cs.SupportedLanguage.CPP and (
+ bound := self._cpp_out_of_class_method_caller(
+ func_node, func_name, module_qn
+ )
+ ):
+ caller_qn, class_qn = bound
+ filtered = self._filtered_calls_for(
+ func_node, language, all_call_nodes, call_starts, owned_func_nodes
+ )
+ self._ingest_function_calls(
+ func_node,
+ caller_qn,
+ cs.NodeLabel.METHOD,
+ module_qn,
+ language,
+ queries,
+ class_qn,
+ call_nodes=filtered,
+ call_name_cache=call_name_cache,
+ )
continue
- if func_qn := self._build_nested_qualified_name(
- func_node, module_qn, func_name, lang_config
+ # A Go receiver method (`func (t T) m()`) is declared at file scope
+ # but the definition pass binds it to its receiver type's node
+ # (qn `module.T.m`). Attribute its body's calls to that method node,
+ # not the receiver-dropping `module.m`, so the CALLS edges join to
+ # a real node.
+ if language == cs.SupportedLanguage.GO and (
+ bound := self._go_receiver_method_caller(
+ func_node, func_name, module_qn
+ )
):
+ caller_qn, container_qn = bound
+ filtered = self._filtered_calls_for(
+ func_node, language, all_call_nodes, call_starts, owned_func_nodes
+ )
+ self._ingest_function_calls(
+ func_node,
+ caller_qn,
+ cs.NodeLabel.METHOD,
+ module_qn,
+ language,
+ queries,
+ container_qn,
+ call_nodes=filtered,
+ call_name_cache=call_name_cache,
+ )
+ continue
+ # A C++ free function inside a namespace is bound by the definition
+ # pass via build_qualified_name (qn `module.ns.fn`); _build_nested...
+ # ignores namespace_definition ancestors and would drop the namespace
+ # (`module.fn`), dangling the CALLS source. Use the same builder so
+ # the qns agree.
+ func_qn = (
+ cpp_utils.build_qualified_name(func_node, module_qn, func_name)
+ if language == cs.SupportedLanguage.CPP
+ else self._build_nested_qualified_name(
+ func_node, module_qn, func_name, lang_config
+ )
+ )
+ if func_qn:
+ filtered = self._filtered_calls_for(
+ func_node, language, all_call_nodes, call_starts, owned_func_nodes
+ )
self._ingest_function_calls(
func_node,
func_qn,
@@ -107,20 +1383,83 @@ def _process_calls_in_functions(
module_qn,
language,
queries,
+ call_nodes=filtered,
+ call_name_cache=call_name_cache,
)
- def _get_rust_impl_class_name(self, class_node: Node) -> str | None:
- class_name = self._get_node_name(class_node, cs.FIELD_TYPE)
- if class_name:
- return class_name
- return next(
- (
- child.text.decode(cs.ENCODING_UTF8)
- for child in class_node.children
- if child.type == cs.TS_TYPE_IDENTIFIER and child.is_named and child.text
- ),
- None,
+ def _go_receiver_method_caller(
+ self, func_node: Node, method_name: str, module_qn: str
+ ) -> tuple[str, str] | None:
+ # Resolve a Go receiver method to its (method_qn, container_qn),
+ # mirroring the definition pass's receiver-type binding. The receiver
+ # type resolves to its node qn (same or sibling file in the package),
+ # and the registry check ensures the method node exists before
+ # overriding the default attribution.
+ if not go_utils.is_receiver_method(func_node):
+ return None
+ receiver_type = go_utils.extract_receiver_type_name(func_node)
+ if not receiver_type:
+ return None
+ container_qn = self._resolver._resolve_class_name(receiver_type, module_qn) or (
+ f"{module_qn}{cs.SEPARATOR_DOT}{receiver_type}"
+ )
+ caller_qn = f"{container_qn}{cs.SEPARATOR_DOT}{method_name}"
+ if caller_qn in self._resolver.function_registry:
+ return caller_qn, container_qn
+ return None
+
+ def _recorded_caller(
+ self, func_node: Node, module_qn: str
+ ) -> FunctionLocation | None:
+ # The registry membership check guards incremental runs, where an
+ # unchanged file's locations were not re-recorded this run.
+ loc = self.function_locations.get(function_span_key(module_qn, func_node))
+ if loc is None or loc.qualified_name not in self._resolver.function_registry:
+ return None
+ return loc
+
+ def _cpp_out_of_class_method_caller(
+ self, func_node: Node, method_name: str, module_qn: str
+ ) -> tuple[str, str] | None:
+ # Resolve an out-of-line C++ method definition to its (method_qn,
+ # class_qn), mirroring the definition pass's class binding. The leaf
+ # class name resolves the class across files (header-declared classes);
+ # `endswith(normalized)` guards against a leaf collision binding to the
+ # wrong class, and the registry check ensures the method node exists
+ # before overriding the default attribution.
+ if not cpp_utils.is_out_of_class_method_definition(func_node):
+ return None
+ # The definition pass already bound this exact definition (keyed by
+ # module + start line); reuse its decision so the caller qn matches
+ # the registered Method node by construction.
+ recorded = self.cpp_out_of_class_methods.get(
+ (module_qn, func_node.start_point[0] + 1)
)
+ if recorded is not None:
+ method_qn, class_qn = recorded
+ if method_qn in self._resolver.function_registry:
+ return method_qn, class_qn
+ class_name = cpp_utils.extract_class_name_from_out_of_class_method(func_node)
+ if not class_name:
+ return None
+ normalized = class_name.replace(cs.SEPARATOR_DOUBLE_COLON, cs.SEPARATOR_DOT)
+ leaf = normalized.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ class_qn = self._resolver._resolve_class_name(leaf, module_qn)
+ if not class_qn or not class_qn.endswith(normalized):
+ return None
+ caller_qn = f"{class_qn}{cs.SEPARATOR_DOT}{method_name}"
+ if caller_qn in self._resolver.function_registry:
+ return caller_qn, class_qn
+ return None
+
+ def _get_rust_impl_class_name(self, class_node: Node) -> str | None:
+ # Use the same bare-type extraction as the definition pass
+ # (rs_utils.extract_impl_target), which strips generic arguments
+ # (`Chars<'a>` -> `Chars`). _get_node_name returns the full generic
+ # text, so a call inside a generic impl block was attributed to a caller
+ # qn bearing the generics (crate.lib.Chars<'a>.go) matching no
+ # registered node, silently dropping the CALLS edge.
+ return rs_utils.extract_impl_target(class_node)
def _get_class_name_for_node(
self, class_node: Node, language: cs.SupportedLanguage
@@ -135,69 +1474,315 @@ def _process_methods_in_class(
class_qn: str,
module_qn: str,
language: cs.SupportedLanguage,
- queries: dict[cs.SupportedLanguage, LanguageQueries],
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
+ all_call_nodes: list[Node] | None = None,
+ call_starts: list[int] | None = None,
+ call_name_cache: dict[int, str | None] | None = None,
+ sorted_func_nodes: list[Node] | None = None,
+ func_node_starts: list[int] | None = None,
) -> None:
- method_query = queries[language][cs.QUERY_FUNCTIONS]
- if not method_query:
- return
- method_cursor = QueryCursor(method_query)
- method_captures = method_cursor.captures(body_node)
- method_nodes = method_captures.get(cs.CAPTURE_FUNCTION, [])
+ if sorted_func_nodes is not None and func_node_starts is not None:
+ body_start = body_node.start_byte
+ body_end = body_node.end_byte
+ lo = bisect_left(func_node_starts, body_start)
+ hi = bisect_right(func_node_starts, body_end)
+ method_nodes = [
+ n for n in sorted_func_nodes[lo:hi] if n.end_byte <= body_end
+ ]
+ else:
+ method_query = queries[language][cs.QUERY_FUNCTIONS]
+ if not method_query:
+ return
+ method_cursor = QueryCursor(method_query)
+ method_captures = sorted_captures(method_cursor, body_node)
+ method_nodes = method_captures.get(cs.CAPTURE_FUNCTION, [])
+ lang_config = queries[language][cs.QUERY_CONFIG]
+ # Only functions that get their own caller node exclude their calls from
+ # the enclosing scope; anonymous arrows (skipped below) must not, so
+ # their calls bubble up instead of dropping.
+ owned_func_nodes = self._attributable_func_nodes(method_nodes, language)
for method_node in method_nodes:
- if not isinstance(method_node, Node):
+ # The body byte-range slice also captures functions of a NESTED
+ # class (Outer body contains Inner.run); those belong to the
+ # nested class and are processed when it is iterated, so skip any
+ # whose nearest enclosing class is not this one (else run also
+ # emits as the phantom Outer.run).
+ if not self._method_in_class_body(method_node, body_node, lang_config):
continue
- method_name = self._get_node_name(method_node)
+ if language in _C_FAMILY_LANGUAGES:
+ method_name = cpp_utils.extract_function_name(method_node)
+ else:
+ method_name = self._get_node_name(method_node)
+ if not method_name and language in _JS_TS_LANGUAGES:
+ method_name = self._js_ts_arrow_binding_name(method_node)
if not method_name:
continue
- method_qn = f"{class_qn}{cs.SEPARATOR_DOT}{method_name}"
+ # method_nodes includes functions nested inside methods. Build the
+ # qn through the enclosing-function chain (Class.method.nested, not
+ # the method-dropping Class.nested) and label a nested function
+ # FUNCTION, so the CALLS edge joins the real node.
+ class_context = class_qn
+ if loc := self._recorded_caller(method_node, module_qn):
+ # Reuse the definition pass's recorded qn/label; the class
+ # walk's structural derivation diverges from it on
+ # preprocessor-distorted C++ class bodies and on TS
+ # declaration merging (issue #652).
+ caller_qn, caller_label = loc.qualified_name, loc.label
+ class_context = loc.container_qn or class_qn
+ else:
+ caller_qn, caller_label = self._class_member_qn_and_label(
+ method_node, class_qn, method_name, lang_config, language
+ )
+ filtered = (
+ self._calls_owned_by(
+ method_node, owned_func_nodes, all_call_nodes, call_starts
+ )
+ if all_call_nodes is not None and call_starts is not None
+ else None
+ )
self._ingest_function_calls(
method_node,
- method_qn,
- cs.NodeLabel.METHOD,
+ caller_qn,
+ caller_label,
module_qn,
language,
queries,
- class_qn,
+ class_context,
+ call_nodes=filtered,
+ call_name_cache=call_name_cache,
+ )
+
+ def _class_member_qn_and_label(
+ self,
+ func_node: Node,
+ class_qn: str,
+ func_name: str,
+ lang_config: LanguageSpec,
+ language: str,
+ ) -> tuple[str, str]:
+ # Build a class-body function's qn through the chain of enclosing
+ # functions up to the class: a direct method is Class.method (METHOD);
+ # a function nested in a method is Class.method.nested (FUNCTION).
+ path_parts: list[str] = []
+ current = func_node.parent
+ while current and current.type not in lang_config.class_node_types:
+ if current.type in lang_config.function_node_types:
+ if (name_node := current.child_by_field_name(cs.FIELD_NAME)) and (
+ name_node.text is not None
+ ):
+ path_parts.append(name_node.text.decode(cs.ENCODING_UTF8))
+ current = current.parent
+ path_parts.reverse()
+ if path_parts:
+ joined = cs.SEPARATOR_DOT.join([*path_parts, func_name])
+ return f"{class_qn}{cs.SEPARATOR_DOT}{joined}", cs.NodeLabel.FUNCTION
+ member = self._java_method_member(func_node, func_name, language)
+ return f"{class_qn}{cs.SEPARATOR_DOT}{member}", cs.NodeLabel.METHOD
+
+ def _java_method_member(
+ self, func_node: Node, func_name: str, language: str
+ ) -> str:
+ # A Java Method node is registered with its parameter signature
+ # (definition pass: class_qn.name(params)), so the caller endpoint of a
+ # CALLS edge must carry the same signature to join that node. Mirrors
+ # class_ingest.mixin's method-qn build exactly.
+ if language != cs.SupportedLanguage.JAVA:
+ return func_name
+ info = java_utils.extract_method_info(func_node)
+ name = info.get(cs.KEY_NAME) or func_name
+ parameters = info.get(cs.KEY_PARAMETERS, [])
+ param_sig = f"({','.join(parameters)})" if parameters else cs.EMPTY_PARENS
+ return f"{name}{param_sig}"
+
+ @staticmethod
+ def _method_in_class_body(
+ method_node: Node, class_body: Node, lang_config: LanguageSpec
+ ) -> bool:
+ # True when method_node's nearest enclosing class is the one whose
+ # body is class_body: walk up, and the first class ancestor's body
+ # must be this body (compared by byte span). A method with no enclosing
+ # class (out-of-class C++ definition captured elsewhere) returns True
+ # so existing handling is unaffected.
+ current = method_node.parent
+ while current is not None:
+ if current.type in lang_config.class_node_types:
+ body = current.child_by_field_name(cs.FIELD_BODY)
+ return body is not None and (
+ body.start_byte == class_body.start_byte
+ and body.end_byte == class_body.end_byte
+ )
+ current = current.parent
+ return True
+
+ def _calls_owned_by(
+ self,
+ func_node: Node,
+ sibling_func_nodes: list[Node],
+ all_call_nodes: list[Node],
+ call_starts: list[int],
+ ) -> list[Node]:
+ # Calls inside func_node MINUS calls owned by functions nested within
+ # it, so a call in a nested function is attributed only to the nested
+ # function, never also to the enclosing one.
+ own = self._filter_calls_in_node(all_call_nodes, call_starts, func_node)
+ descendant_bodies = [
+ body
+ for n in sibling_func_nodes
+ if n is not func_node
+ and n.start_byte >= func_node.start_byte
+ and n.end_byte <= func_node.end_byte
+ and (body := n.child_by_field_name(cs.FIELD_BODY)) is not None
+ ]
+ if not descendant_bodies:
+ return own
+ return [
+ call
+ for call in own
+ if not any(
+ body.start_byte <= call.start_byte and call.end_byte <= body.end_byte
+ for body in descendant_bodies
)
+ ]
def _process_calls_in_classes(
self,
root_node: Node,
module_qn: str,
language: cs.SupportedLanguage,
- queries: dict[cs.SupportedLanguage, LanguageQueries],
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
+ all_call_nodes: list[Node] | None = None,
+ call_starts: list[int] | None = None,
+ call_name_cache: dict[int, str | None] | None = None,
+ combined_captures: dict[str, list] | None = None,
+ sorted_func_nodes: list[Node] | None = None,
+ func_node_starts: list[int] | None = None,
) -> None:
- query = queries[language][cs.QUERY_CLASSES]
- if not query:
- return
- cursor = QueryCursor(query)
- captures = cursor.captures(root_node)
- class_nodes = captures.get(cs.CAPTURE_CLASS, [])
+ if combined_captures is not None:
+ class_nodes = combined_captures.get(cs.CAPTURE_CLASS, [])
+ else:
+ query = queries[language][cs.QUERY_CLASSES]
+ if not query:
+ return
+ cursor = QueryCursor(query)
+ captures = sorted_captures(cursor, root_node)
+ class_nodes = captures.get(cs.CAPTURE_CLASS, [])
for class_node in class_nodes:
- if not isinstance(class_node, Node):
- continue
class_name = self._get_class_name_for_node(class_node, language)
if not class_name:
continue
- class_qn = f"{module_qn}{cs.SEPARATOR_DOT}{class_name}"
+ # A C++ class inside a namespace, or a NESTED class (Outer.Inner),
+ # is bound by the definition pass through its enclosing scope
+ # (qn `module.ns.Class` / `module.Outer.Inner`); the bare
+ # `module.class_name` join drops those ancestors, dangling every
+ # inline method's CALLS source off a phantom node. Use the SAME
+ # builders the definition pass uses so the qns agree.
+ if language == cs.SupportedLanguage.CPP:
+ class_qn = cpp_utils.build_qualified_name(
+ class_node, module_qn, class_name
+ )
+ else:
+ class_qn = (
+ build_nested_qualified_name_for_class(
+ class_node,
+ module_qn,
+ class_name,
+ queries[language][cs.QUERY_CONFIG],
+ )
+ or f"{module_qn}{cs.SEPARATOR_DOT}{class_name}"
+ )
if body_node := class_node.child_by_field_name(cs.FIELD_BODY):
self._process_methods_in_class(
- body_node, class_qn, module_qn, language, queries
+ body_node,
+ class_qn,
+ module_qn,
+ language,
+ queries,
+ all_call_nodes,
+ call_starts,
+ call_name_cache=call_name_cache,
+ sorted_func_nodes=sorted_func_nodes,
+ func_node_starts=func_node_starts,
)
- def _process_module_level_calls(
+ def _overlay_match_arm_binding(
self,
- root_node: Node,
- module_qn: str,
- language: cs.SupportedLanguage,
- queries: dict[cs.SupportedLanguage, LanguageQueries],
- ) -> None:
- self._ingest_function_calls(
- root_node, module_qn, cs.NodeLabel.MODULE, module_qn, language, queries
+ call_name: str,
+ call_node: Node,
+ local_var_types: dict[str, str] | None,
+ match_arm_bindings: list[tuple[int, int, str, str]],
+ ) -> dict[str, str] | None:
+ # If the call's receiver base is a match-arm binding whose arm range
+ # contains this call, overlay that arm's variant type (shadowing the flat
+ # map's last-arm value) so `cmd.apply()` dispatches to the arm's variant.
+ # The innermost (smallest) containing arm wins for nested matches.
+ if local_var_types is None or cs.SEPARATOR_DOT not in call_name:
+ return local_var_types
+ base = call_name.split(cs.SEPARATOR_DOT, 1)[0]
+ pos = call_node.start_byte
+ best: tuple[int, str] | None = None
+ for start, end, name, variant_type in match_arm_bindings:
+ if name == base and start <= pos < end:
+ span = end - start
+ if best is None or span < best[0]:
+ best = (span, variant_type)
+ if best is None or local_var_types.get(base) == best[1]:
+ return local_var_types
+ return {**local_var_types, base: best[1]}
+
+ def _cpp_operator_operand_name(self, call_node: Node) -> str | None:
+ # The receiver-analog operand of an operator expression: the LEFT
+ # side of a binary op, the sole argument of a unary/update op. Only
+ # a bare identifier is returned; anything more complex stays with
+ # the legacy paths.
+ field = (
+ cs.FIELD_LEFT
+ if call_node.type == cs.TS_CPP_BINARY_EXPRESSION
+ else cs.TS_FIELD_ARGUMENT
)
+ operand = call_node.child_by_field_name(field)
+ if operand is None or operand.type != cs.TS_IDENTIFIER:
+ return None
+ return safe_decode_text(operand)
+
+ def _macro_call_name(self, ident: Node) -> str | None:
+ # Reconstruct a `.method` chain from a macro token stream by walking
+ # the method identifier's preceding siblings over `("." )*`.
+ # `server . run` -> "server.run"; `self . shutdown . recv` ->
+ # "self.shutdown.recv". A method with no preceding `.` stays bare.
+ if not (method := ident.text.decode(cs.ENCODING_UTF8) if ident.text else None):
+ return None
+ parts = [method]
+ cur = ident.prev_sibling
+ while cur is not None and cur.type == cs.TS_RS_TOKEN_DOT:
+ if (recv := cur.prev_sibling) is None or (
+ recv.type not in cs.RS_MACRO_RECEIVER_TYPES
+ ):
+ break
+ if not recv.text:
+ break
+ parts.append(recv.text.decode(cs.ENCODING_UTF8))
+ cur = recv.prev_sibling
+ parts.reverse()
+ return cs.SEPARATOR_DOT.join(parts)
- def _get_call_target_name(self, call_node: Node) -> str | None:
+ def _get_call_target_name(
+ self, call_node: Node, language: cs.SupportedLanguage | None = None
+ ) -> str | None:
+ # A macro-internal call (Rust `name(args)` inside a token_tree) is
+ # captured as the bare identifier node; its text is the callee name. A
+ # macro tokenizes `server.run()` into loose tokens (`server . run ( )`),
+ # dropping the field_expression, so reconstruct any `.method`
+ # receiver chain from the preceding sibling tokens; else the bare method
+ # mis-resolves (`server.run()` in tokio::select! to the same-module free
+ # fn `run` instead of Listener.run).
+ if call_node.type == cs.TS_IDENTIFIER and call_node.text is not None:
+ return self._macro_call_name(call_node)
+ # A Dart call node is a selector/cascade_section holding the
+ # argument_part; the target name lives in the PRECEDING sibling
+ # chain, not inside the node.
+ if language == cs.SupportedLanguage.DART:
+ return dart_utils.dart_call_name(call_node)
if func_child := call_node.child_by_field_name(cs.TS_FIELD_FUNCTION):
match func_child.type:
case (
@@ -206,17 +1791,187 @@ def _get_call_target_name(self, call_node: Node) -> str | None:
| cs.TS_MEMBER_EXPRESSION
| cs.CppNodeType.QUALIFIED_IDENTIFIER
| cs.TS_SCOPED_IDENTIFIER
+ | cs.TS_SELECTOR_EXPRESSION
+ | cs.TS_PHP_NAME
):
if func_child.text is not None:
- return str(func_child.text.decode(cs.ENCODING_UTF8))
+ return func_child.text.decode(cs.ENCODING_UTF8)
+ case cs.TS_GENERIC_FUNCTION:
+ # turbofish: unwrap to the underlying callee identifier
+ inner = func_child.child_by_field_name(cs.TS_FIELD_FUNCTION)
+ if inner and inner.text:
+ return inner.text.decode(cs.ENCODING_UTF8)
+ case cs.TS_RS_FIELD_EXPRESSION if language == cs.SupportedLanguage.RUST:
+ # Rust member call `a.b.method()`: use the full dotted receiver
+ # chain as the call name so the resolver can map the receiver to
+ # its inferred type (`self.shutdown.is_shutdown`, `cmd.apply`). A
+ # chain containing a call (`x.f().g`) is left to the chained-call
+ # path; a paren-free chain ends at the bare-method trie fallback
+ # when the receiver type is unknown.
+ if (text := func_child.text) is not None:
+ return text.decode(cs.ENCODING_UTF8)
case cs.TS_CPP_FIELD_EXPRESSION:
field_node = func_child.child_by_field_name(cs.FIELD_FIELD)
if field_node and field_node.text:
- return str(field_node.text.decode(cs.ENCODING_UTF8))
+ method = field_node.text.decode(cs.ENCODING_UTF8)
+ # Prepend a simple-identifier receiver (`obj->m`/`obj.m`
+ # -> `obj.m`) so the resolver can map obj to its type and
+ # bind the correct class method; a `.`-joined two-part name
+ # falls back to the bare method-name trie when the receiver
+ # type is unknown. Complex receivers (chains, calls, `this`)
+ # keep the bare method name.
+ arg = func_child.child_by_field_name(cs.TS_FIELD_ARGUMENT)
+ if (
+ arg is not None
+ and arg.type == cs.TS_IDENTIFIER
+ and arg.text
+ ):
+ receiver = arg.text.decode(cs.ENCODING_UTF8)
+ return f"{receiver}{cs.SEPARATOR_DOT}{method}"
+ # A factory-call receiver (`parser(ia, cb).parse(...)`,
+ # nlohmann's basic_json::parse) is a call_expression on a
+ # bare identifier: emit the chain form so the resolver can
+ # type the factory's return and bind the method on it. A
+ # template/qualified callee (`Reader(...)`,
+ # `detail::Reader(...)`) is a CONSTRUCTOR TEMPORARY: the
+ # callee names the receiver's class directly. Deeper receiver
+ # chains keep the bare method-name trie fallback.
+ if (
+ arg is not None
+ and arg.type == cs.TS_CPP_CALL_EXPRESSION
+ and (
+ callee := arg.child_by_field_name(cs.TS_FIELD_FUNCTION)
+ )
+ is not None
+ and callee.type
+ in (
+ cs.TS_IDENTIFIER,
+ cs.TS_CPP_TEMPLATE_FUNCTION,
+ cs.TS_CPP_QUALIFIED_IDENTIFIER,
+ )
+ and arg.text
+ ):
+ receiver = arg.text.decode(cs.ENCODING_UTF8)
+ return f"{receiver}{cs.SEPARATOR_DOT}{method}"
+ return method
+ case cs.TS_CSHARP_GENERIC_NAME if (
+ language == cs.SupportedLanguage.CSHARP
+ ):
+ # Bare generic call `Handle(...)`: the callee
+ # name is the identifier without its type arguments
+ # (methods register generic-free). Leaving the `<...>` on
+ # yielded no call name, so the call site vanished from the
+ # graph (Polly's parameterless HandleInner overload
+ # delegating to its Func sibling).
+ if func_child.text is not None:
+ full = func_child.text.decode(cs.ENCODING_UTF8)
+ return full.split(cs.CHAR_ANGLE_OPEN, 1)[0]
+ case cs.TS_CSHARP_MEMBER_ACCESS_EXPRESSION if (
+ language == cs.SupportedLanguage.CSHARP
+ ):
+ # C# member call `recv.Method(...)`: emit the `recv.Method`
+ # chain so the resolver can type `recv` and bind the method on
+ # it; an unknown-type receiver falls back to the bare
+ # method-name trie. `name` is the method, `expression` the
+ # receiver.
+ name_node = func_child.child_by_field_name(cs.FIELD_NAME)
+ expr_node = func_child.child_by_field_name(
+ cs.TS_CSHARP_FIELD_EXPRESSION
+ )
+ if name_node and name_node.text:
+ method = name_node.text.decode(cs.ENCODING_UTF8)
+ # A generic member (`recv.Handle`) registers
+ # generic-free; strip the type arguments so the
+ # name-keyed fallbacks can match.
+ method = method.split(cs.CHAR_ANGLE_OPEN, 1)[0]
+ if expr_node and expr_node.text:
+ receiver = expr_node.text.decode(cs.ENCODING_UTF8)
+ return f"{receiver}{cs.SEPARATOR_DOT}{method}"
+ return method
+ case cs.TS_CSHARP_CONDITIONAL_ACCESS_EXPRESSION if (
+ language == cs.SupportedLanguage.CSHARP
+ ):
+ # C# conditional call `recv?.Method(...)`: the method name
+ # lives on the member_binding child. Emit the same
+ # `recv.Method` chain as the unconditional form so the
+ # resolver (or its exact Roslyn call fact) can bind it.
+ binding = next(
+ (
+ child
+ for child in func_child.children
+ if child.type == cs.TS_CSHARP_MEMBER_BINDING_EXPRESSION
+ ),
+ None,
+ )
+ name_node = (
+ binding.child_by_field_name(cs.FIELD_NAME)
+ if binding is not None
+ else None
+ )
+ if name_node and name_node.text:
+ method = name_node.text.decode(cs.ENCODING_UTF8)
+ receiver_node = (
+ func_child.named_children[0]
+ if (func_child.named_children)
+ else None
+ )
+ if (
+ receiver_node is not None
+ and receiver_node is not binding
+ and receiver_node.text
+ ):
+ receiver = receiver_node.text.decode(cs.ENCODING_UTF8)
+ return f"{receiver}{cs.SEPARATOR_DOT}{method}"
+ return method
case cs.TS_PARENTHESIZED_EXPRESSION:
return self._get_iife_target_name(func_child)
match call_node.type:
+ case cs.TS_NEW_EXPRESSION if language in _JS_TS_LANGUAGES:
+ # JS/TS `new Foo(...)` names the class via the `constructor` field
+ # (no `function` field). Returning the constructor name routes
+ # construction through the normal resolve loop: a first-party class
+ # gets INSTANTIATES (+ CALLS to its constructor), and an inline
+ # callback argument (`new CancelablePromise(cb)`) is referenced so
+ # it is not reported as dead.
+ ctor = call_node.child_by_field_name(cs.FIELD_CONSTRUCTOR)
+ if ctor is not None and ctor.text is not None:
+ return ctor.text.decode(cs.ENCODING_UTF8)
+ case cs.TS_OBJECT_CREATION_EXPRESSION if language in (
+ cs.SupportedLanguage.JAVA,
+ cs.SupportedLanguage.CSHARP,
+ ):
+ # Java/C# `new Foo(...)` names the class via the `type` field (no
+ # `function` field). Returning the base type name routes construction
+ # through the normal resolve loop: the class gets INSTANTIATES and its
+ # constructor(s) get CALLS. Strip generic args (`new ArrayList()`
+ # -> ArrayList); a scoped name (`Outer.Inner`) is left for the resolver.
+ type_node = call_node.child_by_field_name(cs.FIELD_TYPE)
+ if type_node is not None and type_node.text is not None:
+ return type_node.text.decode(cs.ENCODING_UTF8).split(
+ cs.CHAR_ANGLE_OPEN, 1
+ )[0]
+ case cs.TS_NEW_EXPRESSION if language == cs.SupportedLanguage.CPP:
+ # C++ `new Foo(...)` names the class via the `type` field (no
+ # `function` field), so it fell through nameless and heap
+ # construction emitted nothing (issue #896, the singleton
+ # `new WindowClassRegistrar()`). Returning the type name routes
+ # it through the normal resolve loop: the class gets
+ # INSTANTIATES and its constructor CALLS. The type reduces
+ # STRUCTURALLY to its dotted class path (`new Foo()` ->
+ # Foo, `new Outer::Inner()` -> Outer.Inner; a textual cut
+ # at `<` would drop the nested suffix); a primitive type
+ # reduces to nothing and stays silent.
+ type_node = call_node.child_by_field_name(cs.FIELD_TYPE)
+ if type_node is not None:
+ return cpp_utils.new_expression_class_path(type_node)
+ case cs.TS_CSHARP_IMPLICIT_OBJECT_CREATION_EXPRESSION if (
+ language == cs.SupportedLanguage.CSHARP
+ ):
+ # C# 9 target-typed `new()` has no `type` field; the
+ # constructed type is named by the enclosing declaration
+ # (issue #773).
+ return self._csharp_target_typed_new_name(call_node)
case (
cs.TS_CPP_BINARY_EXPRESSION
| cs.TS_CPP_UNARY_EXPRESSION
@@ -230,16 +1985,99 @@ def _get_call_target_name(self, call_node: Node) -> str | None:
object_node = call_node.child_by_field_name(cs.FIELD_OBJECT)
name_node = call_node.child_by_field_name(cs.FIELD_NAME)
if name_node and name_node.text:
- method_name = str(name_node.text.decode(cs.ENCODING_UTF8))
+ method_name = name_node.text.decode(cs.ENCODING_UTF8)
if not object_node or not object_node.text:
return method_name
- object_text = str(object_node.text.decode(cs.ENCODING_UTF8))
+ object_text = object_node.text.decode(cs.ENCODING_UTF8)
return f"{object_text}{cs.SEPARATOR_DOT}{method_name}"
+ # Scala infix operator call (`a ~> b`, `xs map f`): the callee is the
+ # `operator` field's method name. tree-sitter has no `function` field
+ # here, so it is unreachable above. Gated to Scala since the node type
+ # string is Scala-specific and the guard keeps other languages inert.
+ # Infix is unambiguously a method call; a bare `field_expression`
+ # (`obj.done` with no parens) is deliberately NOT named here because
+ # Scala's uniform access makes a nullary call and a `val` read
+ # syntactically identical, so resolving it by simple name would turn a
+ # same-named field read into a spurious CALLS edge.
+ case cs.TS_SCALA_INFIX_EXPRESSION if language == cs.SupportedLanguage.SCALA:
+ operator_node = call_node.child_by_field_name(cs.FIELD_OPERATOR)
+ if operator_node and operator_node.text:
+ return operator_node.text.decode(cs.ENCODING_UTF8)
+ # Rust `square!(3)`: the callee lives in the `macro` field (no
+ # `function`/`name` field), so the invocation was captured as a
+ # call but dropped nameless here; unresolvable even now that
+ # macro_rules! definitions register as Function nodes.
+ case cs.TS_RS_MACRO_INVOCATION if language == cs.SupportedLanguage.RUST:
+ macro_node = call_node.child_by_field_name(cs.FIELD_MACRO)
+ if macro_node is not None and macro_node.text is not None:
+ return macro_node.text.decode(cs.ENCODING_UTF8)
if name_node := call_node.child_by_field_name(cs.FIELD_NAME):
if name_node.text is not None:
- return str(name_node.text.decode(cs.ENCODING_UTF8))
+ return name_node.text.decode(cs.ENCODING_UTF8)
+
+ return None
+
+ def _csharp_target_typed_new_name(self, creation_node: Node) -> str | None:
+ # The target type of a bare `new()` is named by the enclosing
+ # declaration: a local/field `T x = new()` (initializer hangs directly
+ # off the variable_declarator), a property initializer
+ # `public T P { get; } = new()`, or a return position (`return new();`
+ # / `=> new()`) typed by the enclosing member. Any other position (an
+ # argument, an operand) needs overload resolution tree-sitter cannot
+ # do: bail rather than guess.
+ node = creation_node.parent
+ while node is not None and node.type in (
+ cs.TS_CSHARP_VARIABLE_DECLARATOR,
+ cs.TS_CSHARP_EQUALS_VALUE_CLAUSE,
+ cs.TS_PARENTHESIZED_EXPRESSION,
+ ):
+ node = node.parent
+ if node is None:
+ return None
+ if node.type in (
+ cs.TS_CSHARP_VARIABLE_DECLARATION,
+ cs.TS_CSHARP_PROPERTY_DECLARATION,
+ ):
+ type_node = node.child_by_field_name(cs.FIELD_TYPE)
+ elif node.type in (
+ cs.TS_RETURN_STATEMENT,
+ cs.TS_CSHARP_ARROW_EXPRESSION_CLAUSE,
+ ):
+ type_node = self._csharp_enclosing_return_type(node)
+ else:
+ return None
+ if (
+ type_node is None
+ or type_node.text is None
+ # `var x = new();` is ill-formed C# (no target type); if it
+ # appears anyway, "var" is not a class name.
+ or type_node.type == cs.TS_CSHARP_IMPLICIT_TYPE
+ ):
+ return None
+ return type_node.text.decode(cs.ENCODING_UTF8).split(cs.CHAR_ANGLE_OPEN, 1)[0]
+ def _csharp_enclosing_return_type(self, node: Node) -> Node | None:
+ # A return position is typed by the nearest enclosing callable:
+ # methods and local functions name it in `returns`, a property or
+ # indexer (or their accessor bodies) in `type`. A lambda/anonymous
+ # method carries no syntactic return type, so a `new()` returned from
+ # one is unresolvable.
+ ancestor = node.parent
+ while ancestor is not None:
+ if ancestor.type in (
+ cs.TS_CSHARP_METHOD_DECLARATION,
+ cs.TS_CSHARP_LOCAL_FUNCTION_STATEMENT,
+ ):
+ return ancestor.child_by_field_name(cs.TS_CSHARP_FIELD_RETURNS)
+ if ancestor.type in (
+ cs.TS_CSHARP_PROPERTY_DECLARATION,
+ cs.TS_CSHARP_INDEXER_DECLARATION,
+ ):
+ return ancestor.child_by_field_name(cs.FIELD_TYPE)
+ if ancestor.type in cs.TS_CSHARP_NESTED_SCOPE_TYPES:
+ return None
+ ancestor = ancestor.parent
return None
def _get_iife_target_name(self, parenthesized_expr: Node) -> str | None:
@@ -258,106 +2096,3432 @@ def _ingest_function_calls(
caller_type: str,
module_qn: str,
language: cs.SupportedLanguage,
- queries: dict[cs.SupportedLanguage, LanguageQueries],
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
class_context: str | None = None,
+ call_nodes: list[Node] | None = None,
+ call_name_cache: dict[int, str | None] | None = None,
) -> None:
- calls_query = queries[language].get(cs.QUERY_CALLS)
- if not calls_query:
- return
+ if language in _TYPED_LANGUAGES:
+ local_var_types = (
+ self._resolver.type_inference.build_local_variable_type_map(
+ caller_node, module_qn, language, class_context
+ )
+ )
+ else:
+ local_var_types = None
+
+ # Rust match arms often reuse one binding name (`cmd`) for different variant
+ # types; the flat map keeps only the last arm's type. These per-arm scoped
+ # bindings let each call inside an arm resolve against ITS arm's type.
+ match_arm_bindings: list[tuple[int, int, str, str]] = []
+ if language == cs.SupportedLanguage.RUST and local_var_types is not None:
+ match_arm_bindings = self._resolver.type_inference.rust_type_inference.collect_match_arm_bindings(
+ caller_node
+ )
+
+ caller_spec = (caller_type, cs.KEY_QUALIFIED_NAME, caller_qn)
- local_var_types = self._resolver.type_inference.build_local_variable_type_map(
- caller_node, module_qn, language
+ self._io_processor.process_io_for_caller(
+ caller_node, caller_spec, module_qn, language
+ )
+ self._flow_processor.process_flow_for_caller(
+ caller_node,
+ caller_spec,
+ caller_qn,
+ module_qn,
+ language,
+ class_context,
+ local_var_types,
)
- cursor = QueryCursor(calls_query)
- captures = cursor.captures(caller_node)
- call_nodes = captures.get(cs.CAPTURE_CALL, [])
+ caller_params: frozenset[str] = frozenset()
+ ordered_params: list[str] | None = None
+ if language == cs.SupportedLanguage.PYTHON:
+ ordered_params = python_parameter_names(caller_node)
+ elif language == cs.SupportedLanguage.GO:
+ ordered_params = go_parameter_names(caller_node)
+ elif language in _JS_TS_LANGUAGES:
+ ordered_params = js_ts_parameter_names(caller_node)
+ elif language == cs.SupportedLanguage.CPP:
+ ordered_params = cpp_parameter_names(caller_node)
+ if ordered_params is not None:
+ # Every flow-traced language records its callable params and the
+ # closures it returns, so a later `x = factory(); x(cb)` alias call can
+ # flow cb into the returned closure regardless of source language.
+ self._flow_param_names[caller_qn] = ordered_params
+ caller_params = frozenset(ordered_params)
+ self._collect_returned_callables(
+ caller_node,
+ caller_qn,
+ module_qn,
+ local_var_types,
+ class_context,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ )
+
+ # Runs independently of call_nodes: a getter access is an attribute, not
+ # a call, so callers that read a property but make no other call must
+ # still reach this pass before the early return below.
+ if language == cs.SupportedLanguage.PYTHON and (
+ prop_names := self._resolver.function_registry.property_names()
+ ):
+ self._ingest_property_accesses(
+ caller_node,
+ caller_spec,
+ caller_qn,
+ module_qn,
+ local_var_types,
+ class_context,
+ queries[language][cs.QUERY_CONFIG],
+ prop_names,
+ )
+
+ # Same need as the Python pass above, C# shape: a property getter
+ # access is a member_access_expression (usually in RECEIVER position),
+ # never an invocation, so callers that only READ a property emit no
+ # edge to it and dead-code flags it (Polly's Context.WrappedDictionary,
+ # ResiliencePipeline.Pipeline).
+ if language == cs.SupportedLanguage.CSHARP and (
+ csharp_prop_names := self._resolver.function_registry.property_names()
+ ):
+ self._ingest_csharp_property_reads(
+ caller_node,
+ caller_spec,
+ caller_qn,
+ module_qn,
+ local_var_types,
+ queries[language][cs.QUERY_CONFIG],
+ csharp_prop_names,
+ )
+
+ # Same need again, Dart shape (issue #869): a getter access is a bare
+ # identifier or a member selector, never an invocation, so callers
+ # that only READ a getter emit nothing and dead-code flags it
+ # (wonderous' _enableVideo/startYr family).
+ if language == cs.SupportedLanguage.DART and (
+ dart_prop_names := self._resolver.function_registry.property_names()
+ ):
+ self._ingest_dart_getter_reads(
+ caller_node,
+ caller_spec,
+ caller_qn,
+ module_qn,
+ local_var_types,
+ class_context,
+ queries[language][cs.QUERY_CONFIG],
+ dart_prop_names,
+ )
+
+ # Operator syntax (k in r, r[k], r[k]=v, len(r)) dispatches to dunder
+ # methods; emit those edges when the operand is a first-party type.
+ if language == cs.SupportedLanguage.PYTHON:
+ self._ingest_operator_dispatch_calls(
+ caller_node, caller_spec, module_qn, local_var_types
+ )
+ if (
+ language == cs.SupportedLanguage.PYTHON
+ or language in _JS_TS_LANGUAGES
+ or language == cs.SupportedLanguage.GO
+ ):
+ self._ingest_assignment_function_references(
+ caller_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ caller_qn,
+ )
+ if language in _JS_TS_LANGUAGES:
+ self._ingest_jsx_component_references(
+ caller_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ caller_qn,
+ )
+ # A DEFAULT PARAMETER value naming a function (`useStore(api,
+ # selector = identity as any)`, zustand) references it: the default
+ # is invoked through the parameter when the caller omits the
+ # argument, never by a visible call.
+ self._ingest_default_param_references(
+ caller_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ caller_qn,
+ )
+ # A function handed back bare (`return defaultUsageFunc`) is a first-class
+ # value invoked by whoever receives it, never by a visible call; Go leans
+ # on this for factories/getters (cobra's getUsageFunc), Python on returned
+ # bound methods (django GEOSCoordSeq `return self._get_point_2d`).
+ # Reference it so the returned function is reachable. These languages share
+ # the `return_statement` node type, and the emit path resolves bare names
+ # and self-attributes alike.
+ if language in _JS_TS_LANGUAGES or language in (
+ cs.SupportedLanguage.GO,
+ cs.SupportedLanguage.PYTHON,
+ ):
+ self._ingest_returned_function_references(
+ caller_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ caller_qn,
+ )
+ # Dispatch-table handler references, for every flow language. Module-scope
+ # literals are scanned explicitly in process_calls_in_file (before the
+ # no-calls early return), so only nested scopes here.
+ if (
+ language == cs.SupportedLanguage.PYTHON or language in _JS_TS_LANGUAGES
+ ) and caller_type != cs.NodeLabel.MODULE:
+ self._ingest_collection_function_references(
+ caller_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ )
+ if language == cs.SupportedLanguage.GO and caller_type != cs.NodeLabel.MODULE:
+ self._ingest_go_composite_function_references(
+ caller_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ self._flow_scope_boundaries(queries[language][cs.QUERY_CONFIG]),
+ )
+ if language == cs.SupportedLanguage.CPP:
+ self._ingest_cpp_braced_return_instantiations(
+ caller_node, caller_spec, caller_qn, module_qn
+ )
+ self._ingest_cpp_member_init_ctor_calls(caller_node, caller_spec, module_qn)
+ self._ingest_cpp_implicit_base_lifecycle_calls(
+ caller_node, caller_spec, caller_qn, module_qn
+ )
+ self._ingest_cpp_declaration_ctor_calls(caller_node, caller_spec, module_qn)
+
+ if call_nodes is None:
+ calls_query = queries[language].get(cs.QUERY_CALLS)
+ if not calls_query:
+ return
+ cursor = QueryCursor(calls_query)
+ captures = sorted_captures(cursor, caller_node)
+ call_nodes = captures.get(cs.CAPTURE_CALL, [])
+
+ if not call_nodes:
+ return
- logger.debug(
- ls.CALL_FOUND_NODES.format(
- count=len(call_nodes), language=language, caller=caller_qn
+ is_java = language == cs.SupportedLanguage.JAVA
+ is_csharp = language == cs.SupportedLanguage.CSHARP
+ is_js_ts = language in _JS_TS_LANGUAGES
+ is_cpp = language == cs.SupportedLanguage.CPP
+ # Template type-parameter names in scope at this caller (`template` -> {"SAX"}): a receiver typed to one has no concrete type here, so the
+ # dispatch fan-out treats it like an untyped receiver rather than an external.
+ cpp_template_params = (
+ self._resolver.type_inference.cpp_type_inference.collect_template_param_names(
+ caller_node
)
+ if is_cpp
+ else frozenset()
+ )
+ method_invocation_type = cs.TS_METHOD_INVOCATION
+ resolver = self._resolver
+ resolve_func = resolver.resolve_function_call
+ resolve_builtin = resolver.resolve_builtin_call if is_js_ts else None
+ resolve_cpp_op = resolver.resolve_cpp_operator_call if is_cpp else None
+ get_target = self._get_call_target_name
+ class_label = cs.NodeLabel.CLASS
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ calls_rel = cs.RelationshipType.CALLS
+ qn_key = cs.KEY_QUALIFIED_NAME
+ _id = id
+ is_python = language == cs.SupportedLanguage.PYTHON
+ # Languages with interprocedural callable-parameter flow enabled: a
+ # callback passed to a first-party function whose parameter is invoked
+ # (directly or in a nested closure) is traced to the concrete callback.
+ is_flow_lang = (
+ is_python
+ or language == cs.SupportedLanguage.GO
+ or language in _JS_TS_LANGUAGES
+ or is_cpp
)
+ # C# and Dart get the argument-REFERENCE half only (a method group or
+ # tear-off passed as an argument keeps its target reachable, Polly's
+ # EmptyHandler family, Flutter's `onPressed: _handleTap` callbacks)
+ # without the interprocedural callable-param flow, which is untuned
+ # for them.
+ is_arg_ref_lang = is_flow_lang or language in _ARG_REF_ONLY_LANGUAGES
+ # A method-group pass is not an invocation: C# records it as
+ # REFERENCES everywhere (CALLS here put 282 phantom edges into the
+ # Polly call graph, retrieval precision 1.0 -> 0.92); flow languages
+ # keep their historical CALLS form for external/builtin callees.
+ arg_ref_rel = (
+ cs.RelationshipType.CALLS
+ if is_flow_lang
+ else cs.RelationshipType.REFERENCES
+ )
+ alias_map: dict[str, str] | None = None
+ factory_aliases: dict[str, str] | None = None
+ cpp_local_aliases: dict[str, list[tuple[str, int, int]]] | None = None
for call_node in call_nodes:
- if not isinstance(call_node, Node):
+ node_id = _id(call_node)
+ if call_name_cache is not None and node_id in call_name_cache:
+ call_name = call_name_cache[node_id]
+ else:
+ call_name = get_target(call_node, language)
+ if call_name_cache is not None:
+ call_name_cache[node_id] = call_name
+ # An inline function ARGUMENT is handed to the callee regardless of
+ # whether the callee resolves: an external/param callee
+ # (`create((set) => ...)` passing `set((state) => ...)`, zustand) or a
+ # cast-wrapped one (`;(set as NamedSet)((state) => reducer(...))`)
+ # still consumes it. Reference each inline arg from this scope, BEFORE
+ # the no-name bail (a cast-wrapped callee yields no call name).
+ # Registry-guarded and idempotent with the callable-params path.
+ if is_js_ts and call_node.type == cs.TS_CALL_EXPRESSION:
+ self._ingest_inline_call_arg_references(
+ call_node, caller_spec, ensure_rel, caller_qn, module_qn
+ )
+ if not call_name:
+ # A callee that is itself a call (`wraps(view_func)(_view_wrapper)`)
+ # or otherwise yields no name still consumes its arguments through
+ # whatever callable it produced; reference first-party functions
+ # passed to it or dead-code flags every django-style view
+ # decorator wrapper. REFERENCES (not arg_ref_rel) is
+ # deliberate and predates the C# split: with no callee name
+ # there is no invocation to assert, for ANY language.
+ if is_arg_ref_lang:
+ self._ingest_argument_function_references(
+ call_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ language,
+ )
continue
- # (H) tree-sitter finds ALL call nodes including nested; no recursive processing needed
+ call_var_types = local_var_types
+ if match_arm_bindings:
+ call_var_types = self._overlay_match_arm_binding(
+ call_name, call_node, local_var_types, match_arm_bindings
+ )
- call_name = self._get_call_target_name(call_node)
- if not call_name:
- continue
+ if is_cpp:
+ # A C++ member call through a template-parameter receiver has no
+ # concrete type, so precise resolution fails and the external-receiver
+ # guard drops the edge, orphaning every structural interface
+ # implementer (json_sax_* visitors dispatched via `sax->start_object()`).
+ # Fan such a call out to the method on every class defining it. Runs
+ # before the primary resolution/continue below so it fires even when
+ # that edge is dropped; a concretely-typed receiver is skipped inside
+ # the resolver. sorted(): the target label is a hash-randomized
+ # StrEnum, so sort for deterministic output.
+ for target_type, target_qn in sorted(
+ resolver.cpp_dispatch_targets(
+ call_name, call_var_types, cpp_template_params
+ )
+ ):
+ for variant in resolver.function_registry.variants(target_qn):
+ ensure_rel(
+ caller_spec, calls_rel, (target_type, qn_key, variant)
+ )
+ cpp_operand_type_qn: str | None = None
if (
- language == cs.SupportedLanguage.JAVA
- and call_node.type == cs.TS_METHOD_INVOCATION
+ is_cpp
+ and call_node.type in _CPP_OPERATOR_EXPRESSION_TYPES
+ and call_name.startswith(cs.OPERATOR_PREFIX)
):
- callee_info = self._resolver.resolve_java_method_call(
- call_node, module_qn, local_var_types
+ cpp_operand_type_qn = resolver.cpp_operand_class_qn(
+ self._cpp_operator_operand_name(call_node),
+ call_var_types,
+ module_qn,
+ )
+ if cpp_operand_type_qn is not None:
+ # The operand's type is KNOWN: the operator binds to that
+ # type's own overload or, when it has none, to nothing at
+ # all (a builtin enum/int operation), never rebound by bare
+ # name to an unrelated class's overload set. Only an untyped
+ # operand falls through to the legacy paths below.
+ callee_info = resolver.cpp_operator_for_type(
+ call_name, cpp_operand_type_qn
)
+ if callee_info is None:
+ continue
+ elif is_java and call_node.type == method_invocation_type:
+ callee_info = resolver.resolve_java_method_call(
+ call_node, module_qn, local_var_types, caller_qn
+ )
+ elif is_csharp and call_node.type == cs.TS_CSHARP_INVOCATION_EXPRESSION:
+ callee_info = resolver.resolve_csharp_method_call(
+ call_node, module_qn, call_var_types, caller_qn
+ )
+ if (
+ callee_info is not None
+ and callee_info != csharp_ti.CSHARP_EXTERNAL_TARGET
+ and (fn_node := call_node.child_by_field_name(cs.TS_FIELD_FUNCTION))
+ is not None
+ and fn_node.type
+ in (cs.TS_CSHARP_IDENTIFIER, cs.TS_CSHARP_GENERIC_NAME)
+ ):
+ # A bare call resolved by ARITY may have same-arity
+ # siblings differing only in parameter types (Serilog's
+ # FormatExactNumericValue switch dispatch); keep the
+ # whole family reachable. NOT when a Roslyn fact pinned
+ # the site: that is the compiler's exact overload
+ # choice, and widening it would revive dead siblings.
+ engine = resolver.type_inference.csharp_type_inference
+ if not engine.semantic_fact_resolved(call_node, module_qn):
+ for sibling_qn in engine.csharp_same_arity_family(
+ callee_info[1]
+ ):
+ ensure_rel(
+ caller_spec,
+ calls_rel,
+ (cs.NodeLabel.METHOD, qn_key, sibling_qn),
+ )
+ if callee_info == csharp_ti.CSHARP_EXTERNAL_TARGET:
+ # Provably external (base.X() with an external base, a
+ # static call on an unregistered type, an object
+ # virtual on an untyped receiver): no edge, and no
+ # name-trie fallback that would fabricate one onto an
+ # unrelated same-name first-party member.
+ callee_info = None
+ elif callee_info is None:
+ # A C# member call whose receiver could not be typed (or a
+ # bare call) falls back to the generic simple-name resolver,
+ # which keeps Phase 1 intra-file resolution working.
+ callee_info = resolve_func(
+ call_name,
+ module_qn,
+ call_var_types,
+ class_context,
+ caller_qn,
+ language,
+ )
else:
- callee_info = self._resolver.resolve_function_call(
- call_name, module_qn, local_var_types, class_context
+ callee_info = resolve_func(
+ call_name,
+ module_qn,
+ call_var_types,
+ class_context,
+ caller_qn,
+ language,
+ )
+ if callee_info and language == cs.SupportedLanguage.RUST:
+ # Rust macros and functions live in SEPARATE namespaces:
+ # a macro invocation (write!) must not bind a same-named fn
+ # (std-prelude macro names collide with common fn names and
+ # the false edge revives dead code), and a fn call must not
+ # bind a same-named macro.
+ is_macro_target = callee_info[1] in self.macro_qns
+ if is_macro_target != (call_node.type == cs.TS_RS_MACRO_INVOCATION):
+ callee_info = None
+ if not callee_info and resolve_builtin is not None:
+ callee_info = resolve_builtin(call_name)
+ if not callee_info and resolve_cpp_op is not None:
+ callee_info = resolve_cpp_op(call_name, module_qn)
+ if not callee_info and language == cs.SupportedLanguage.CPP:
+ # `using appender = basic_appender; appender(out)`:
+ # the alias is no registered node, so the bare call resolves
+ # to nothing and the constructed class's ctor stays edge-free.
+ # The cross-file typedef/using map covers file/namespace-scope
+ # aliases; a BODY-local alias is what that collector skips, so
+ # it comes from the caller-scoped map (_resolve_class_name then
+ # follows any further alias chain). Binding the callee to the
+ # class drops into the class branch below, emitting INSTANTIATES
+ # + ctor CALLS like any construction. Gated on alias-map
+ # membership so genuinely unknown names keep their unresolved
+ # handling.
+ if cpp_local_aliases is None:
+ cpp_local_aliases = (
+ CppTypeInferenceEngine().collect_local_type_aliases(caller_node)
+ )
+ lookup_name = call_name
+ # Declaration-ordered AND lexically-scoped lookup: a call
+ # BEFORE the body-local alias's declaration or AFTER its
+ # enclosing block/lambda closes can never mean it; among
+ # windows that do contain the call, the latest declaration
+ # wins (C++ shadowing).
+ best_decl_end = -1
+ for underlying, decl_end, scope_end in cpp_local_aliases.get(
+ call_name, ()
+ ):
+ if (
+ decl_end <= call_node.start_byte < scope_end
+ and decl_end > best_decl_end
+ ):
+ lookup_name = underlying
+ best_decl_end = decl_end
+ if (
+ lookup_name != call_name or call_name in resolver.type_aliases
+ ) and (
+ aliased_qn := resolver._resolve_class_name(lookup_name, module_qn)
+ ):
+ callee_info = (cs.NodeLabel.CLASS, aliased_qn)
+ if not callee_info and cs.SEPARATOR_DOT not in call_name:
+ if is_python:
+ # A bare name that resolves to nothing may be a local alias of a
+ # callable (do = self._start; do()). Resolve the assignment's
+ # right-hand side and treat the alias call as a call to it.
+ if alias_map is None:
+ alias_map = self._build_local_alias_map(
+ caller_node, queries[language][cs.QUERY_CONFIG], module_qn
+ )
+ if (rhs := alias_map.get(call_name)) is not None:
+ callee_info = resolve_func(
+ rhs, module_qn, local_var_types, class_context, caller_qn
+ )
+ if callee_info is None and is_flow_lang:
+ # `x = factory(...); x(cb)`: x holds a closure returned by a
+ # first-party factory (e.g. a retry/cache decorator applied
+ # imperatively). Record the call so cb flows into that closure's
+ # callable parameter once factory returns are known (finalize).
+ if factory_aliases is None:
+ factory_aliases = self._build_factory_alias_map(
+ caller_node,
+ module_qn,
+ local_var_types,
+ class_context,
+ self._flow_scope_boundaries(
+ queries[language][cs.QUERY_CONFIG]
+ ),
+ caller_qn,
+ )
+ if (factory_qn := factory_aliases.get(call_name)) is not None:
+ self._record_factory_call(
+ call_node,
+ caller_qn,
+ factory_qn,
+ module_qn,
+ local_var_types,
+ class_context,
+ )
+
+ if not callee_info and is_python and cs.SEPARATOR_DOT in call_name:
+ # recv.field(...) where field is a callable struct field:
+ # resolve to the functions bound to it at construction sites.
+ self._ingest_callable_field_calls(
+ call_name, caller_spec, local_var_types, ensure_rel
)
- if callee_info:
- callee_type, callee_qn = callee_info
- elif builtin_info := self._resolver.resolve_builtin_call(call_name):
- callee_type, callee_qn = builtin_info
- elif operator_info := self._resolver.resolve_cpp_operator_call(
- call_name, module_qn
+
+ if is_python and call_name.rsplit(cs.SEPARATOR_DOT, 1)[-1] in (
+ cs.HIGHER_ORDER_BUILTINS
):
- callee_type, callee_qn = operator_info
- else:
+ # sorted(xs, key=f) and friends invoke f synchronously in this
+ # frame, so the trace attributes the call to the enclosing fn.
+ self._ingest_higher_order_builtin_calls(
+ call_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ )
+
+ if not callee_info:
+ if is_arg_ref_lang:
+ # The callee is not first-party (a framework/stdlib call such as
+ # grpclib Handler(self.__rpc_x), JS setTimeout(target), or a
+ # runtime dispatcher), so the call chain cannot be followed into
+ # it. A first-party function handed to it as an argument is still
+ # invoked, so record it as referenced from this scope to keep it
+ # reachable, across every flow-traced language.
+ self._ingest_argument_function_references(
+ call_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ arg_ref_rel,
+ language,
+ )
+ continue
+
+ callee_type, callee_qn = callee_info
+
+ if (
+ language == cs.SupportedLanguage.CPP
+ and call_node.type == cs.TS_NEW_EXPRESSION
+ and callee_type != class_label
+ ):
+ # Inside X's own methods the bare name `X` prefers the class
+ # MEMBER (the ctor), so `new X()` resolves past the class
+ # branch and INSTANTIATES is lost (issue #896, the singleton's
+ # `new WindowClassRegistrar()` in GetInstance). Construction
+ # always targets the CLASS: rebind so the class branch below
+ # emits INSTANTIATES plus the full ctor/dtor redirect.
+ leaf = call_name.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ class_qn = resolver._resolve_class_name(leaf, module_qn)
+ if class_qn is not None and (
+ class_qn == call_name
+ or class_qn.endswith(f"{cs.SEPARATOR_DOT}{call_name}")
+ ):
+ callee_type, callee_qn = class_label, class_qn
+
+ # A callee that resolved to a builtin (e.g. JS setTimeout(target))
+ # has no first-party body to follow into, so pass-through flow is
+ # pointless; but a first-party callback handed to it is still invoked,
+ # so record a reference edge from this scope to keep it reachable.
+ # The synthetic builtin.* qn never has a node, so emitting a CALLS
+ # edge to it would only mint a phantom the database drops (issue
+ # #652: 485 across the fixture suite) -- mirror the C++ builtin
+ # operator rule and emit no edge at all.
+ callee_is_builtin = callee_qn.startswith(_BUILTIN_QN_PREFIX)
+ if callee_is_builtin:
+ if is_arg_ref_lang:
+ self._ingest_argument_function_references(
+ call_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ arg_ref_rel,
+ language,
+ )
continue
- logger.debug(
- ls.CALL_FOUND.format(
- caller=caller_qn,
- call_name=call_name,
- callee_type=callee_type,
- callee_qn=callee_qn,
+
+ if is_flow_lang:
+ self._collect_callable_flow(
+ call_node,
+ callee_qn,
+ caller_qn,
+ caller_params,
+ module_qn,
+ local_var_types,
+ class_context,
+ )
+ if is_arg_ref_lang:
+ # Functions are first-class values: a first-party callee may STORE
+ # a passed callback for later dynamic dispatch (config objects,
+ # codecs, registries), which callable-param flow cannot trace, or
+ # the callee may be a same-named misbind of an external method.
+ # Record the pass itself as a REFERENCES edge from the passing
+ # scope so the callback is never reported dead.
+ self._ingest_argument_function_references(
+ call_node,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ language,
)
- )
- self.ingestor.ensure_relationship_batch(
- (caller_type, cs.KEY_QUALIFIED_NAME, caller_qn),
- cs.RelationshipType.CALLS,
- (callee_type, cs.KEY_QUALIFIED_NAME, callee_qn),
+ if is_python and (
+ dispatch_targets := resolver.protocol_dispatch_targets(callee_qn)
+ ):
+ # The call resolved to a Protocol stub; the stub never runs, so emit
+ # edges to the method on every conformer instead of the stub.
+ for conformer_type, conformer_qn in dispatch_targets:
+ for target_qn in resolver.function_registry.variants(conformer_qn):
+ ensure_rel(
+ caller_spec,
+ calls_rel,
+ (conformer_type, qn_key, target_qn),
+ )
+ continue
+
+ if (
+ is_python
+ and class_context
+ and (
+ call_name.startswith(cs.PY_SELF_PREFIX)
+ or call_name.startswith(cs.PY_CLS_PREFIX)
+ )
+ ):
+ # self.M()/cls.M() statically targets the enclosing class's own or
+ # inherited M and dynamically dispatches to every concrete subclass
+ # override, so emit an edge to each in ADDITION to the resolved edge
+ # below; otherwise a base (or override) reached only through the
+ # self-call looks dead. Anchor on the enclosing class, not the
+ # resolved callee: when M is abstract with several overrides the trie
+ # resolves the call to an arbitrary sibling, so anchoring there would
+ # miss the base and the others. The self/cls receiver excludes
+ # super().M() and Base.M() (not virtual dispatch). Skip self.attr.M()
+ # (a call on a member, not on self).
+ _, _, self_method = call_name.partition(cs.SEPARATOR_DOT)
+ if cs.SEPARATOR_DOT not in self_method:
+ for target_type, target_qn in resolver.self_dispatch_targets(
+ class_context, self_method
+ ):
+ for variant in resolver.function_registry.variants(target_qn):
+ ensure_rel(
+ caller_spec,
+ calls_rel,
+ (target_type, qn_key, variant),
+ )
+
+ if is_flow_lang:
+ # f(...) invoked through a parameter: the edge runs from the
+ # callee to whatever each call site binds to that parameter.
+ self._ingest_callable_param_calls(
+ call_node,
+ callee_type,
+ callee_qn,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ )
+
+ if (
+ language in (cs.SupportedLanguage.JAVA, cs.SupportedLanguage.CSHARP)
+ and call_node.type
+ in (
+ cs.TS_OBJECT_CREATION_EXPRESSION,
+ cs.TS_CSHARP_IMPLICIT_OBJECT_CREATION_EXPRESSION,
+ )
+ and callee_type != class_label
+ ):
+ # `new X(...)` where X resolves to a non-class: a Java interface
+ # or annotation implemented by an anonymous class (`new
+ # Comparator(){ ... }`), or a C# type the resolver could not bind
+ # to a first-party class. There is no first-party constructor to
+ # call, and a bare CALLS edge to a non-callable node (Interface) is
+ # not valid, so drop the edge rather than emit it.
+ continue
+
+ if callee_type == class_label:
+ # Record construction as INSTANTIATES -> the class node (keeps
+ # CALLS function/method-only). When the class defines __init__,
+ # ALSO redirect a CALLS edge to it (the constructor runs); when
+ # it does not (dataclass/NamedTuple/pydantic), INSTANTIATES is
+ # the only edge.
+ for class_variant in resolver.function_registry.variants(callee_qn):
+ # A duplicate-suffixed variant may be a DIFFERENT kind
+ # of node (a merged TS namespace registers as a class,
+ # a colliding function as a Function); only class-typed
+ # variants are instantiable, and a mismatched label is
+ # a phantom the database drops (issue #652).
+ variant_type = resolver.function_registry.get(class_variant)
+ if variant_type is not None and variant_type != NodeType.CLASS:
+ continue
+ ensure_rel(
+ caller_spec,
+ cs.RelationshipType.INSTANTIATES,
+ (class_label, qn_key, class_variant),
+ )
+ if language in (
+ cs.SupportedLanguage.JAVA,
+ cs.SupportedLanguage.CSHARP,
+ cs.SupportedLanguage.CPP,
+ cs.SupportedLanguage.DART,
+ ):
+ # A Java/C#/C++/Dart constructor is a method named like its
+ # class (`Foo.Foo`), not `__init__`; `new Foo(...)` / `Foo(...)`
+ # runs one, so redirect a CALLS edge to every declared
+ # constructor (overload selection unneeded for reachability).
+ # C#, C++, and Dart default constructors use the same
+ # class-simple-name convention, so java_constructor_targets
+ # selects them too (a Dart NAMED constructor is invoked by its
+ # own name and resolves as an ordinary method). C++ additionally
+ # redirects to the destructor: the object's `~X` runs at end of
+ # lifetime with no call node of its own. sorted(): the target
+ # label is a hash-randomized StrEnum, so sort for determinism.
+ if language == cs.SupportedLanguage.CPP:
+ self._emit_cpp_ctor_calls(caller_spec, callee_qn)
+ continue
+ for ctor_type, ctor_qn in sorted(
+ resolver.java_constructor_targets(callee_qn)
+ ):
+ for variant in resolver.function_registry.variants(ctor_qn):
+ ensure_rel(
+ caller_spec, calls_rel, (ctor_type, qn_key, variant)
+ )
+ continue
+ init_qn = f"{callee_qn}{cs.SEPARATOR_DOT}{cs.PY_METHOD_INIT}"
+ if init_qn not in resolver.function_registry:
+ continue
+ callee_type = cs.NodeLabel.METHOD
+ callee_qn = init_qn
+
+ for target_qn in resolver.function_registry.variants(callee_qn):
+ # A duplicate-suffixed variant may be a DIFFERENT kind of
+ # node (a TS namespace merged onto a function registers as
+ # a class); only callable variants take a CALLS edge, and
+ # emitting the primary's label onto a differently-typed
+ # node is a phantom the database drops (issue #652). An
+ # unregistered target keeps its edge (resolver-derived
+ # callees like an unwrapped fn.call base may not register).
+ target_type = resolver.function_registry.get(target_qn)
+ if target_type is not None and target_type not in (
+ NodeType.FUNCTION,
+ NodeType.METHOD,
+ ):
+ continue
+ ensure_rel(
+ caller_spec,
+ calls_rel,
+ (callee_type, qn_key, target_qn),
+ )
+
+ if (
+ language == cs.SupportedLanguage.GO
+ and callee_type == cs.NodeLabel.FUNCTION
+ ):
+ # A bare Go call resolves to one file's copy of a package-level
+ # function; same-package same-name siblings are mutually-exclusive
+ # build-tag variants (gin's `validate`), so emit an edge to each so
+ # no build variant is reported dead. sorted(): the target label is a
+ # hash-randomized StrEnum, so sort for deterministic output.
+ for sibling_type, sibling_qn in sorted(
+ resolver.go_package_sibling_targets(callee_qn)
+ ):
+ for variant in resolver.function_registry.variants(sibling_qn):
+ ensure_rel(
+ caller_spec, calls_rel, (sibling_type, qn_key, variant)
+ )
+
+ if callee_type == cs.NodeLabel.METHOD:
+ # A call bound to an interface/trait method (the static callee;
+ # removing its declaration breaks the call) with exactly ONE
+ # implementer also runs the concrete method, so edge both. The
+ # old REPLACING redirect orphaned the interface stub (gson's
+ # FieldNamingStrategy.translateName reported dead); sorted():
+ # the target label is a hash-randomized StrEnum.
+ for impl_type, impl_qn in sorted(
+ resolver.interface_sole_impl_targets(callee_qn)
+ ):
+ for variant in resolver.function_registry.variants(impl_qn):
+ ensure_rel(caller_spec, calls_rel, (impl_type, qn_key, variant))
+
+ if (
+ is_js_ts
+ and cs.SEPARATOR_DOT in call_name
+ and callee_type in (cs.NodeLabel.FUNCTION, cs.NodeLabel.METHOD)
+ ):
+ # A JS member call that bound one twin of a double-registered
+ # prototype method (`this.lookup()` -> module-flat `view.lookup`,
+ # leaving `View.lookup` dead) edges the same-module same-name
+ # member twin too (duplicate-QN keep-both design; revive-only).
+ for twin_type, twin_qn in sorted(
+ resolver.js_member_twin_targets(callee_qn)
+ ):
+ for variant in resolver.function_registry.variants(twin_qn):
+ ensure_rel(caller_spec, calls_rel, (twin_type, qn_key, variant))
+
+ if (
+ is_python
+ and callee_type == cs.NodeLabel.FUNCTION
+ and cs.SEPARATOR_DOT not in call_name
+ ):
+ # A platform-conditional import with a local fallback def (click's
+ # `if WIN: from ._winconsole import X ... else: def X(...)`) is
+ # statically undecidable; the call resolves to the import, so the
+ # mutually-exclusive local def looks dead. When the name was bound
+ # by a CONDITIONAL import and the CURRENT module also defines it,
+ # fan the call out to the local twin too, mirroring the Go
+ # build-variant fan-out. An UNCONDITIONAL import shadowing a local
+ # def is plain shadowing: the local stays dead, so no edge.
+ local_qn = f"{module_qn}{cs.SEPARATOR_DOT}{call_name}"
+ if (
+ local_qn != callee_qn
+ and call_name
+ in resolver.import_processor.conditional_imports.get(module_qn, ())
+ and resolver.function_registry.get(local_qn) == NodeType.FUNCTION
+ ):
+ for variant in resolver.function_registry.variants(local_qn):
+ ensure_rel(
+ caller_spec,
+ calls_rel,
+ (cs.NodeLabel.FUNCTION, qn_key, variant),
+ )
+
+ def _ingest_operator_dispatch_calls(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ ) -> None:
+ boundary = (cs.TS_PY_FUNCTION_DEFINITION, cs.TS_PY_CLASS_DEFINITION)
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in boundary:
+ continue
+ match node.type:
+ case cs.TS_PY_SUBSCRIPT:
+ parent = node.parent
+ left = (
+ parent.child_by_field_name(cs.TS_FIELD_LEFT)
+ if parent is not None and parent.type == cs.TS_PY_ASSIGNMENT
+ else None
+ )
+ is_write = left is not None and left.id == node.id
+ self._emit_operator_dunder(
+ node.child_by_field_name(cs.FIELD_VALUE),
+ cs.PY_DUNDER_SETITEM if is_write else cs.PY_DUNDER_GETITEM,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ )
+ case cs.TS_PY_COMPARISON_OPERATOR:
+ operators = node.child_by_field_name(cs.TS_FIELD_OPERATORS)
+ if (
+ operators is not None
+ and (op_text := safe_decode_text(operators))
+ and cs.PY_OP_IN in op_text.split()
+ and node.named_children
+ ):
+ self._emit_operator_dunder(
+ node.named_children[-1],
+ cs.PY_DUNDER_CONTAINS,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ )
+ case cs.TS_PY_CALL:
+ func = node.child_by_field_name(cs.TS_FIELD_FUNCTION)
+ args = node.child_by_field_name(cs.FIELD_ARGUMENTS)
+ if (
+ func is not None
+ and safe_decode_text(func) == cs.PY_BUILTIN_LEN
+ and args is not None
+ and len(args.named_children) == 1
+ ):
+ self._emit_operator_dunder(
+ args.named_children[0],
+ cs.PY_DUNDER_LEN,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ )
+ case cs.TS_PY_BOOLEAN_OPERATOR:
+ self._emit_truthiness(
+ node.child_by_field_name(cs.TS_FIELD_LEFT),
+ caller_spec,
+ module_qn,
+ local_var_types,
+ )
+ self._emit_truthiness(
+ node.child_by_field_name(cs.TS_FIELD_RIGHT),
+ caller_spec,
+ module_qn,
+ local_var_types,
+ )
+ case cs.TS_PY_NOT_OPERATOR:
+ self._emit_truthiness(
+ node.child_by_field_name(cs.TS_FIELD_ARGUMENT),
+ caller_spec,
+ module_qn,
+ local_var_types,
+ )
+ case (
+ cs.TS_PY_IF_STATEMENT
+ | cs.TS_PY_WHILE_STATEMENT
+ | cs.TS_PY_ELIF_CLAUSE
+ | cs.TS_PY_CONDITIONAL_EXPRESSION
+ ):
+ # A bare object as a condition is tested for truthiness; nested
+ # boolean/not operators are handled when the walk reaches them.
+ self._emit_truthiness(
+ node.child_by_field_name(cs.TS_FIELD_CONDITION),
+ caller_spec,
+ module_qn,
+ local_var_types,
+ )
+ stack.extend(node.children)
+
+ def _emit_truthiness(
+ self,
+ operand: Node | None,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ ) -> None:
+ # Truthiness of an object calls __bool__ if defined, else __len__. Only a
+ # bare name/attribute operand names an object (a comparison/call is already
+ # a bool and is handled elsewhere); try __bool__ first, then __len__.
+ if operand is None or operand.type not in (
+ cs.TS_PY_IDENTIFIER,
+ cs.TS_PY_ATTRIBUTE,
+ ):
+ return
+ for dunder in (cs.PY_DUNDER_BOOL, cs.PY_DUNDER_LEN):
+ if self._emit_operator_dunder(
+ operand, dunder, caller_spec, module_qn, local_var_types
+ ):
+ return
+
+ def _emit_operator_dunder(
+ self,
+ operand: Node | None,
+ dunder: str,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ ) -> bool:
+ # Resolve the implied .__dunder__ call; resolution only succeeds
+ # for a first-party class that defines the dunder, so builtin containers
+ # (dict/list) yield no edge. Restrict to simple attribute/name operands.
+ # Returns whether an edge was emitted.
+ if operand is None or not (operand_text := safe_decode_text(operand)):
+ return False
+ if any(ch in operand_text for ch in cs.PY_OPERAND_REJECT_CHARS):
+ return False
+ targets = self._resolver.operator_dunder_targets(
+ operand_text, dunder, module_qn, local_var_types
+ )
+ if not targets:
+ return False
+ for callee_type, callee_qn in targets:
+ for target_qn in self._resolver.function_registry.variants(callee_qn):
+ self.ingestor.ensure_relationship_batch(
+ caller_spec,
+ cs.RelationshipType.CALLS,
+ (callee_type, cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+ return True
+
+ def _ingest_assignment_function_references(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ boundary_types: frozenset[str],
+ caller_qn: str | None = None,
+ ) -> None:
+ # `x = some_function` binds a first-class function value to a name; the
+ # alias is then stored, passed onward, or returned for dynamic dispatch
+ # (http_callback = llm_http_task_closure_with_context), so the assignment
+ # itself references the function and must keep it reachable. Only a plain
+ # name/attribute RHS counts (calls resolve as calls); the walk stops at
+ # nested scope boundaries, which own their own pass.
+ resolve_func = self._resolver.resolve_function_call
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ # Continue THROUGH an unowned anonymous arrow (zustand's curried
+ # middleware body `(config) => (set, get, api) => { api.setState =
+ # ... }`): it gets no caller pass, so its assignments would else be
+ # scanned by nobody and the stored functions report dead. Named
+ # nested scopes still own their own pass.
+ if node.type in boundary_types and not self._is_unowned_js_scope(node):
+ continue
+ if rhs_field := _ASSIGNMENT_RHS_FIELDS.get(node.type):
+ right = node.child_by_field_name(rhs_field)
+ # Go wraps every assignment RHS in an expression_list
+ # (`var f = fn`, `a, b = g, h`); scan each value so the bare
+ # func name(s) underneath are referenced. Other languages
+ # carry a lone RHS node.
+ values = (
+ list(right.named_children)
+ if right is not None and right.type == cs.TS_GO_EXPRESSION_LIST
+ else [right]
+ )
+ for value in values:
+ if value is None:
+ continue
+ # `export const persist = persistImpl as unknown as Persist`
+ # wraps the aliased impl in TS casts, and devtools' shape
+ # interleaves parens (`api.setState = ((s, r) => {...}) as
+ # SetState`); peel both so the bare name/arrow underneath is
+ # what we reference.
+ # `const bound = handler.bind(null)` stores the bound
+ # handler; .bind/.call/.apply are transparent for
+ # reference resolution like a cast.
+ value = self._peel_bound_callable(value, peel_parens=True)
+ # A bare-name RHS names a callable; an inline arrow/function-expr
+ # RHS (`OpenAPI.TOKEN = async () => {}`) stores an anonymous
+ # function on the target for later invocation, and
+ # _emit_callback_edge references it by position. A named
+ # arrow-const RHS is registered by its name, so the by-position
+ # lookup finds nothing.
+ # A LOGICAL DEFAULT RHS (`done = done || function (err,
+ # str) {...}`, express's render) hides the stored function
+ # one level down; scan the binary operands too.
+ if value.type == cs.TS_BINARY_EXPRESSION:
+ for operand in value.named_children:
+ operand = self._unwrap_ts_value(operand)
+ if operand.type in _INLINE_FUNC_VALUE_TYPES:
+ self._emit_callback_edge(
+ caller_spec,
+ operand,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ )
+ continue
+ # A Python TERNARY / BOOLEAN-DEFAULT RHS (`get_response =
+ # self._async if is_async else self._sync`, django's
+ # BaseHandler; `f = handler or fallback`) binds one of its
+ # RESULT operands; each result operand naming a callable is a
+ # possible referent. A ternary's condition is only
+ # truthiness-tested, never bound, so it is excluded; both
+ # boolean operands are possible results.
+ if value.type in (
+ cs.TS_PY_CONDITIONAL_EXPRESSION,
+ cs.TS_PY_BOOLEAN_OPERATOR,
+ ):
+ operands = list(value.named_children)
+ if (
+ value.type == cs.TS_PY_CONDITIONAL_EXPRESSION
+ and len(operands) == 3
+ ):
+ operands = [operands[0], operands[2]]
+ for operand in operands:
+ operand = self._unwrap_ts_value(operand)
+ if (
+ operand.type in _ASSIGNMENT_RHS_REF_TYPES
+ or operand.type in _INLINE_FUNC_VALUE_TYPES
+ ):
+ self._emit_callback_edge(
+ caller_spec,
+ operand,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ )
+ continue
+ if (
+ value.type in _ASSIGNMENT_RHS_REF_TYPES
+ or value.type in _INLINE_FUNC_VALUE_TYPES
+ ):
+ self._emit_callback_edge(
+ caller_spec,
+ value,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ )
+ # A member-assigned inline function (`api.setState =
+ # (s, r) => {...}`) is ALSO registered by the def pass
+ # under the PROPERTY name (scope.setState), which the
+ # by-position anonymous candidate above never matches;
+ # reference that registration too or it reports dead.
+ if value.type in _INLINE_FUNC_VALUE_TYPES:
+ self._emit_assigned_name_ref(
+ node, caller_spec, ensure_rel, caller_qn
+ )
+ stack.extend(node.children)
+
+ def _emit_assigned_name_ref(
+ self,
+ assign_node: Node,
+ caller_spec: tuple[str, str, str],
+ ensure_rel,
+ caller_qn: str | None,
+ ) -> None:
+ # Resolve the assignment target's simple name (`api.setState` ->
+ # setState, `listener` -> listener) to a def-pass registration in the
+ # enclosing scope. Registry-guarded: emits only when such a node exists,
+ # so a plain data assignment adds nothing.
+ if assign_node.type != cs.TS_ASSIGNMENT_EXPRESSION:
+ return
+ left = assign_node.child_by_field_name(cs.FIELD_LEFT)
+ if left is None:
+ return
+ name_node = (
+ left.child_by_field_name(cs.FIELD_PROPERTY)
+ if left.type == cs.TS_MEMBER_EXPRESSION
+ else left
+ )
+ if name_node is None or name_node.type not in (
+ cs.TS_IDENTIFIER,
+ cs.TS_PROPERTY_IDENTIFIER,
+ ):
+ return
+ if not (name := safe_decode_text(name_node)):
+ return
+ registry = self._resolver.function_registry
+ scope_qn = caller_qn or caller_spec[2]
+ candidate = f"{scope_qn}{cs.SEPARATOR_DOT}{name}"
+ for target_qn in registry.variants(candidate):
+ if registry.get(target_qn) is None:
+ continue
+ ensure_rel(
+ caller_spec,
+ cs.RelationshipType.REFERENCES,
+ (cs.NodeLabel.FUNCTION, cs.KEY_QUALIFIED_NAME, target_qn),
)
- def _build_nested_qualified_name(
+ def _ingest_jsx_component_references(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ boundary_types: frozenset[str],
+ caller_qn: str | None = None,
+ ) -> None:
+ # `` renders the Card component: the framework invokes it
+ # through the element, never by a call the graph can see, so the JSX
+ # usage references the component. Uppercase names only; lowercase
+ # tags are HTML elements and must not misbind to same-named locals.
+ # The walk stops at nested scope boundaries (each nested function's
+ # own pass covers its JSX) but continues THROUGH jsx elements so
+ # nested markup is covered by the scope that renders it.
+ # Resolve and emit directly rather than via _emit_callback_edge: a
+ # class component resolves to a CLASS node whose reference must point
+ # at the class itself, but that helper redirects CLASS -> __init__
+ # and drops the edge when __init__ is absent, as it always is for a
+ # JS/TS class.
+ resolve_func = self._resolver.resolve_function_call
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ registry = self._resolver.function_registry
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ # Stop at a nested scope that gets its OWN caller pass (a named
+ # function/arrow, a class), but continue THROUGH an anonymous arrow
+ # (a `.map()`/`cell`/forwardRef callback): those are skipped as
+ # callers, so their JSX, rendered on behalf of this scope, would
+ # otherwise be scanned by nobody and report as dead.
+ if node.type in boundary_types and not self._is_unowned_js_scope(node):
+ continue
+ if node.type in _JSX_NAMED_ELEMENT_TYPES:
+ name_node = node.child_by_field_name(cs.FIELD_NAME)
+ name_text = safe_decode_text(name_node) if name_node else None
+ if name_text and name_text[0].isupper():
+ resolved = resolve_func(
+ name_text, module_qn, local_var_types, class_context, caller_qn
+ )
+ if resolved:
+ res_type, res_qn = resolved
+ for target_qn in registry.variants(res_qn):
+ ensure_rel(
+ caller_spec,
+ cs.RelationshipType.REFERENCES,
+ (res_type, cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+ elif node.type == cs.TS_JSX_EXPRESSION:
+ # A `{...}` attribute value hands its inner expression to the
+ # element as a prop. A bare identifier (onClick={handleLogout})
+ # or inline arrow (onClick={() => x()}) is a function the
+ # framework invokes on the event, so reference it; other
+ # expressions resolve to nothing and are skipped by the helper.
+ for value in node.named_children:
+ self._emit_callback_edge(
+ caller_spec,
+ value,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn=caller_qn,
+ rel_type=cs.RelationshipType.REFERENCES,
+ )
+ stack.extend(node.children)
+
+ def _ingest_returned_function_references(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ boundary_types: frozenset[str],
+ caller_qn: str | None = None,
+ ) -> None:
+ # A function handed back via `return` (a useEffect cleanup
+ # `return () => unsubscribe()`, a factory `return handler`) is invoked by
+ # whoever receives it, never by a visible call. Reference it from the
+ # returning scope. Walk continues through anonymous arrows (the effect
+ # callback is anonymous, so its `return` bubbles here) but stops at named
+ # nested functions, which own their returns.
+ resolve_func = self._resolver.resolve_function_call
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ # An expression-bodied arrow (`const persistImpl = (config) =>
+ # (set, get, api) => {...}`, zustand's curried middleware shape) has NO
+ # return_statement; its body IS the implicit return. Reference the inner
+ # function directly, both on the caller itself and on any unowned anon
+ # arrow the walk continues through (deeper currying bubbles here too).
+ self._emit_expression_body_return(
+ caller_node, caller_spec, ensure_rel, caller_qn
+ )
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in boundary_types:
+ if not self._is_unowned_js_scope(node):
+ continue
+ self._emit_expression_body_return(
+ node, caller_spec, ensure_rel, caller_qn
+ )
+ if node.type == cs.TS_RETURN_STATEMENT:
+ for value in node.named_children:
+ # A returned TUPLE hides its function elements one level
+ # down (`return _load_field, (...)` in django Field's
+ # __reduce__: pickle later calls the first element);
+ # expand containers so each function handed back inside
+ # one is referenced like a bare return.
+ for expanded in self._expand_py_first_class_values(value):
+ self._emit_callback_edge(
+ caller_spec,
+ expanded,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn=caller_qn,
+ rel_type=cs.RelationshipType.REFERENCES,
+ )
+ stack.extend(node.children)
+
+ def _expand_py_first_class_values(
+ self, value: Node, language: cs.SupportedLanguage | None = None
+ ) -> list[Node]:
+ # Peel Python container literals and result-position conditional
+ # operands so a function stored in a tuple/list/set, a dict VALUE,
+ # a bare return-tuple (expression_list), or a ternary/boolean-default
+ # branch is treated like a bare first-class value; nesting expands
+ # recursively. A ternary's condition is only truthiness-tested, never
+ # bound, so it stays excluded. Any other node comes back unchanged, so
+ # non-Python shapes are unaffected.
+ is_dart = language == cs.SupportedLanguage.DART
+ out: list[Node] = []
+ stack = [value]
+ while stack:
+ node = stack.pop()
+ children = _first_class_value_children(node, is_dart)
+ if children is None:
+ out.append(node)
+ else:
+ stack.extend(reversed(children))
+ return out
+
+ def _emit_expression_body_return(
self,
func_node: Node,
+ caller_spec: tuple[str, str, str],
+ ensure_rel,
+ caller_qn: str | None,
+ ) -> None:
+ body = func_node.child_by_field_name(cs.FIELD_BODY)
+ if body is not None and body.type in _INLINE_FUNC_VALUE_TYPES:
+ self._emit_inline_arg_function_ref(
+ body,
+ caller_spec,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ )
+
+ def _ingest_collection_function_references(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
module_qn: str,
- func_name: str,
- lang_config: LanguageSpec,
- ) -> str | None:
- path_parts: list[str] = []
- current = func_node.parent
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ boundary_types: frozenset[str],
+ ) -> None:
+ # A function/method placed as a value in a dict/object or list/array literal
+ # is a dispatch table wired to be invoked later (handlers[key](...)),
+ # commonly dispatched by a dynamic string key or in another module where the
+ # call site is not statically resolvable. Treat each such reference as a call
+ # from the enclosing scope so the handler is reachable. The walk stops at
+ # nested function/class boundaries, so a table built inside a nested scope
+ # is attributed to that scope's own pass, EXCEPT an unowned JS/TS arrow (a
+ # Promise executor, a `.forEach` callback), which gets no caller pass; its
+ # calls bubble to this scope, so its object tables (a `defineProperty`
+ # getter descriptor) must be scanned here too or the callbacks inside are
+ # orphaned and report as dead.
+ resolve_func = self._resolver.resolve_function_call
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in boundary_types and not self._is_unowned_js_scope(node):
+ continue
+ if node.type in _DICT_LIKE_COLLECTION_TYPES:
+ for pair in node.named_children:
+ # An object-literal SHORTHAND METHOD (`return { then(x)
+ # {...}, catch(x) {...} }`, persist's thenable) is a stored
+ # callable like a pair value, but it is a method_definition
+ # node, not a pair; reference it by name so it is not dead
+ # unless the repo never calls it AND never hands the object
+ # out (it cannot know the consumer).
+ if pair.type == cs.TS_METHOD_DEFINITION:
+ self._emit_shorthand_method_ref(
+ pair, caller_spec, module_qn, ensure_rel
+ )
+ continue
+ if (
+ pair.type == cs.TS_PY_PAIR
+ and (value := pair.child_by_field_name(cs.FIELD_VALUE))
+ is not None
+ ):
+ if value.type in _INLINE_FUNC_VALUE_TYPES:
+ self._emit_inline_value_function_ref(
+ pair, value, caller_spec, module_qn, ensure_rel
+ )
+ continue
+ # A table VALUE wrapped in parens or a ternary
+ # (django SQLCompiler's `"local_setter": (partial(...)
+ # if ... else local_setter_noop)`) hides the handler
+ # candidates one level down; expand before emitting.
+ for expanded in self._expand_py_first_class_values(value):
+ self._emit_value_function_ref(
+ expanded,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ )
+ elif node.type in _SEQUENCE_LIKE_COLLECTION_TYPES:
+ for element in node.named_children:
+ for expanded in self._expand_py_first_class_values(element):
+ self._emit_value_function_ref(
+ expanded,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ )
+ stack.extend(node.children)
- if not isinstance(current, Node):
- logger.warning(
- ls.CALL_UNEXPECTED_PARENT.format(
- node=func_node, parent_type=type(current)
+ def _ingest_cpp_braced_return_instantiations(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ caller_qn: str,
+ module_qn: str,
+ ) -> None:
+ # `return {args};` (nlohmann's exception factories) constructs the
+ # caller's DECLARED return type through a bare initializer_list; no
+ # call node exists, so the constructed class's ctor gets no edge and
+ # reports dead even when its only factory is alive. Emit INSTANTIATES
+ # to the class and CALLS to its ctors, like an explicit construction.
+ # A lambda body is skipped: its returns are not the caller's.
+ # Revive-only: nothing is emitted unless the return type resolves to a
+ # registered first-party class.
+ has_braced_return = False
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type == cs.TS_CPP_LAMBDA_EXPRESSION:
+ continue
+ if node.type == cs.TS_RETURN_STATEMENT and any(
+ child.type == cs.TS_CPP_INITIALIZER_LIST
+ for child in node.named_children
+ ):
+ has_braced_return = True
+ break
+ stack.extend(node.children)
+ if not has_braced_return:
+ return
+ class_qn = self._resolver.cpp_braced_return_class(caller_qn, module_qn)
+ if class_qn is None:
+ return
+ registry = self._resolver.function_registry
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ for class_variant in registry.variants(class_qn):
+ variant_type = registry.get(class_variant)
+ if variant_type is not None and variant_type != NodeType.CLASS:
+ continue
+ ensure_rel(
+ caller_spec,
+ cs.RelationshipType.INSTANTIATES,
+ (cs.NodeLabel.CLASS, cs.KEY_QUALIFIED_NAME, class_variant),
+ )
+ self._emit_cpp_ctor_calls(caller_spec, class_qn)
+
+ def _ingest_cpp_member_init_ctor_calls(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ ) -> None:
+ # A ctor's member initializer list runs base-class ctors
+ # (`: buffer(g, 0)`) and delegated ctors (`: widget(0)`) with no
+ # call_expression node, so a base ctor only ever reached through
+ # derived member-init had zero incoming CALLS and reported dead
+ # (fmt buffer.buffer). Each initializer whose head name resolves to a
+ # registered class emits CALLS to that class's ctors; a member FIELD
+ # initializer resolves to no class and emits nothing (a field named
+ # exactly like a registered class still emits, the common
+ # field-shadows-its-own-type case where the ctor does run). The list
+ # is a DIRECT child of function_definition, so a nested lambda's or
+ # local class's initializers never leak in.
+ for init_list in caller_node.children:
+ if init_list.type != cs.CppNodeType.FIELD_INITIALIZER_LIST:
+ continue
+ for initializer in init_list.named_children:
+ if initializer.type != cs.CppNodeType.FIELD_INITIALIZER:
+ continue
+ name = self._cpp_member_init_head_name(initializer)
+ class_qn = (
+ self._resolver._resolve_type_to_class_qn(name, module_qn)
+ if name
+ else None
+ )
+ if class_qn is not None:
+ self._emit_cpp_ctor_calls(caller_spec, class_qn)
+
+ def _ingest_cpp_implicit_base_lifecycle_calls(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ caller_qn: str,
+ module_qn: str,
+ ) -> None:
+ # A ctor whose member initializer list does not name a base still
+ # runs that base's default ctor, and a dtor runs base dtors after
+ # its own body; neither has an AST node (issue #892), so base-chain
+ # lifecycles reached only implicitly reported dead (wonderous
+ # Win32Window). The caller's identity comes from its qn (leaf ==
+ # class simple name for a ctor, `~name` for a dtor, parent a
+ # registered class), which covers out-of-class definitions whose
+ # per-caller pass runs at module level. Registry guarded: only
+ # bases resolved to registered classes emit, and a delegating ctor
+ # (`: Derived(0)`) emits nothing because the delegated-to ctor owns
+ # the base call.
+ # Only a DEFINITION knows its member initializer list; the in-class
+ # prototype registers under the same qn and also runs a per-caller
+ # pass, and emitting from it would double every edge (and invent
+ # base calls a bodied definition's initializer list excludes). A
+ # `= default` member parses as function_definition, so the guard
+ # keeps it: its implicit base call is guaranteed by the standard.
+ if caller_node.type not in (
+ cs.CppNodeType.FUNCTION_DEFINITION,
+ cs.CppNodeType.INLINE_METHOD_DEFINITION,
+ ):
+ return
+ registry = self._resolver.function_registry
+ class_qn, sep, leaf = caller_qn.rpartition(cs.SEPARATOR_DOT)
+ if not sep or registry.get(class_qn) != NodeType.CLASS:
+ return
+ simple = class_qn.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ is_ctor = leaf == simple
+ is_dtor = leaf == f"{cs.CPP_DESTRUCTOR_PREFIX}{simple}"
+ if not is_ctor and not is_dtor:
+ return
+ # The full INHERITS closure, not just direct bases: construction and
+ # destruction run EVERY ancestor's ctor/dtor unconditionally, and an
+ # intermediate whose definition lives outside the parsed source has
+ # no bodied caller pass to carry the chain onward (PR #894 review).
+ ancestors = self._cpp_registered_ancestors(class_qn)
+ if not ancestors:
+ return
+ if is_dtor:
+ for base_qn in ancestors:
+ self._emit_cpp_lifecycle_targets(
+ caller_spec, self._resolver.cpp_destructor_targets(base_qn)
)
+ return
+ named = self._cpp_member_init_class_qns(caller_node, module_qn)
+ if class_qn in named:
+ return
+ for base_qn in ancestors:
+ if base_qn in named:
+ # The member-init pass owns this base's edge; its own deeper
+ # ancestors stay in the walk via the closure.
+ continue
+ self._emit_cpp_lifecycle_targets(
+ caller_spec, self._resolver.java_constructor_targets(base_qn)
)
- return None
- while current and current.type not in lang_config.module_node_types:
- if current.type in lang_config.function_node_types:
- if name_node := current.child_by_field_name(cs.FIELD_NAME):
- text = name_node.text
- if text is not None:
- path_parts.append(text.decode(cs.ENCODING_UTF8))
- elif current.type in lang_config.class_node_types:
- return None
+ def _cpp_registered_ancestors(self, class_qn: str) -> list[str]:
+ # Registered classes in the INHERITS closure, nearest first.
+ ancestors: list[str] = []
+ seen = {class_qn}
+ queue = list(self._resolver.class_inheritance.get(class_qn, []))
+ while queue:
+ current = queue.pop(0)
+ if current in seen:
+ continue
+ seen.add(current)
+ if self._resolver.function_registry.get(current) == NodeType.CLASS:
+ ancestors.append(current)
+ queue.extend(self._resolver.class_inheritance.get(current, []))
+ return ancestors
- current = current.parent
+ def _emit_cpp_lifecycle_targets(
+ self,
+ caller_spec: tuple[str, str, str],
+ targets: set[tuple[str, str]],
+ ) -> None:
+ registry = self._resolver.function_registry
+ for target_type, target_qn in sorted(targets):
+ for variant in registry.variants(target_qn):
+ self.ingestor.ensure_relationship_batch(
+ caller_spec,
+ cs.RelationshipType.CALLS,
+ (target_type, cs.KEY_QUALIFIED_NAME, variant),
+ )
- path_parts.reverse()
+ def _cpp_member_init_class_qns(self, caller_node: Node, module_qn: str) -> set[str]:
+ # Class qns the ctor's member initializer list names explicitly;
+ # those base ctors are already emitted by the member-init pass.
+ named: set[str] = set()
+ for init_list in caller_node.children:
+ if init_list.type != cs.CppNodeType.FIELD_INITIALIZER_LIST:
+ continue
+ for initializer in init_list.named_children:
+ if initializer.type != cs.CppNodeType.FIELD_INITIALIZER:
+ continue
+ name = self._cpp_member_init_head_name(initializer)
+ if name and (
+ resolved := self._resolver._resolve_type_to_class_qn(
+ name, module_qn
+ )
+ ):
+ named.add(resolved)
+ return named
+
+ def _ingest_cpp_declaration_ctor_calls(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ ) -> None:
+ # A declaration-shaped stack construction has no call node, so its
+ # ctor (and end-of-lifetime dtor) got no edge and reported dead
+ # (issue #871). Two shapes: `Point origin(10, 10)` (an
+ # init_declarator whose argument list is a direct child) and the
+ # most-vexing-parse misparse `FlutterWindow window(project);` (a
+ # "function declaration" whose arguments are bare in-scope locals).
+ # Revive-only: nothing is emitted unless the declared type resolves
+ # to a registered first-party class. Lambda bodies are walked
+ # because their calls flat-attribute to the enclosing caller (the
+ # general call pass does the same); local-class scopes are skipped,
+ # their method bodies being genuinely separate callers. The walk
+ # descends into a declaration's own children too: a lambda bound by
+ # `auto g = [](...) { ... }` nests inside one.
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in cs.CPP_COMPOUND_TYPES:
+ continue
+ if node.type != cs.CppNodeType.DECLARATION:
+ stack.extend(node.children)
+ continue
+ stack.extend(node.children)
+ if not self._cpp_declaration_is_construction(node):
+ continue
+ type_name = self._cpp_declaration_type_name(node)
+ class_qn = (
+ self._resolver._resolve_type_to_class_qn(type_name, module_qn)
+ if type_name
+ else None
+ )
+ if class_qn is not None:
+ self._emit_cpp_construction_edges(caller_spec, class_qn)
+
+ @staticmethod
+ def _cpp_declaration_is_construction(node: Node) -> bool:
+ # `Point origin(10, 10)`: an init_declarator carrying a direct
+ # argument list; otherwise the most-vexing-parse evidence test.
+ for declarator in node.children_by_field_name(cs.FIELD_DECLARATOR):
+ if declarator.type == cs.CppNodeType.INIT_DECLARATOR and any(
+ child.type == cs.TS_ARGUMENT_LIST for child in declarator.children
+ ):
+ return True
+ return cpp_utils.is_cpp_vexing_parse_construction(node)
+
+ def _emit_cpp_construction_edges(
+ self, caller_spec: tuple[str, str, str], class_qn: str
+ ) -> None:
+ registry = self._resolver.function_registry
+ for class_variant in registry.variants(class_qn):
+ variant_type = registry.get(class_variant)
+ if variant_type is not None and variant_type != NodeType.CLASS:
+ continue
+ self.ingestor.ensure_relationship_batch(
+ caller_spec,
+ cs.RelationshipType.INSTANTIATES,
+ (cs.NodeLabel.CLASS, cs.KEY_QUALIFIED_NAME, class_variant),
+ )
+ self._emit_cpp_ctor_calls(caller_spec, class_qn)
+
+ @staticmethod
+ def _cpp_declaration_type_name(node: Node) -> str | None:
+ # The written type spelling, normalized like a member-init head:
+ # registered class qns are unspecialized and dot-separated, so cut
+ # at the first `<` and normalize `::`.
+ type_node = node.child_by_field_name(cs.FIELD_TYPE)
+ if type_node is None or type_node.text is None:
+ return None
+ name = type_node.text.decode(cs.ENCODING_UTF8).split(cs.CHAR_ANGLE_OPEN, 1)[0]
+ return name.replace(cs.SEPARATOR_DOUBLE_COLON, cs.SEPARATOR_DOT) or None
+
+ def _emit_cpp_ctor_calls(
+ self, caller_spec: tuple[str, str, str], class_qn: str
+ ) -> None:
+ # Construction runs a ctor now and the dtor at end of lifetime;
+ # neither has a call node, so both get the redirect. sorted(): the
+ # target label is a hash-randomized StrEnum, so sort for determinism.
+ registry = self._resolver.function_registry
+ targets = self._resolver.java_constructor_targets(
+ class_qn
+ ) | self._resolver.cpp_destructor_targets(class_qn)
+ for target_type, target_qn in sorted(targets):
+ for variant in registry.variants(target_qn):
+ self.ingestor.ensure_relationship_batch(
+ caller_spec,
+ cs.RelationshipType.CALLS,
+ (target_type, cs.KEY_QUALIFIED_NAME, variant),
+ )
+
+ @staticmethod
+ def _cpp_member_init_head_name(initializer: Node) -> str | None:
+ # The head is the initializer's first named child: a plain
+ # field_identifier (`buffer`), a template_method (`base`), or a
+ # qualified_identifier (`ns::other`, `ns::base` -- the
+ # qualified node CONTAINS the template one, so branching on the
+ # outer type cannot strip the specialization args). The raw text
+ # carries the written spelling in every shape; registered class qns
+ # are unspecialized and dot-separated, so cut at the first `<` and
+ # normalize `::` (PR #792 review: `ns::base(g)` resolved
+ # nothing because the args leaked into the lookup).
+ head = next(iter(initializer.named_children), None)
+ if (
+ head is None
+ or head.type
+ not in (
+ cs.CppNodeType.FIELD_IDENTIFIER,
+ cs.CppNodeType.TEMPLATE_METHOD,
+ cs.CppNodeType.QUALIFIED_IDENTIFIER,
+ )
+ or head.text is None
+ ):
+ return None
+ name = head.text.decode(cs.ENCODING_UTF8).split(cs.CHAR_ANGLE_OPEN, 1)[0]
+ return name.replace(cs.SEPARATOR_DOUBLE_COLON, cs.SEPARATOR_DOT) or None
+
+ def _ingest_go_composite_function_references(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ boundary_types: frozenset[str],
+ ) -> None:
+ # A Go function placed as a value in a composite literal, a func map
+ # (`map[string]any{"rpad": rpad}`) or a func slice (`[]Handler{a, b}`), is
+ # a dispatch table invoked later by key, never by a visible call, so its
+ # entries look dead. Reference each from the enclosing scope. Go's literal
+ # shape differs from the JS/Py pair form: composite_literal > literal_value >
+ # {keyed_element(value=literal_element) | literal_element}, and the element
+ # wraps the bare identifier one level down, so unwrap before resolving.
+ resolve_func = self._resolver.resolve_function_call
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in boundary_types:
+ continue
+ if node.type == cs.TS_GO_LITERAL_VALUE:
+ for element in node.named_children:
+ value = (
+ element.child_by_field_name(cs.FIELD_VALUE)
+ if element.type == cs.TS_GO_KEYED_ELEMENT
+ else element
+ )
+ if value is None or value.type != cs.TS_GO_LITERAL_ELEMENT:
+ continue
+ inner = value.named_children[0] if value.named_children else None
+ if inner is None:
+ continue
+ self._emit_value_function_ref(
+ inner,
+ caller_spec,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ )
+ stack.extend(node.children)
+
+ def _emit_value_function_ref(
+ self,
+ node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ resolve_func,
+ ensure_rel,
+ ) -> None:
+ # A value cast for typing (`persistImpl as unknown as Persist`) is
+ # transparent for reference resolution, and `fn.bind(ctx)` /
+ # `fn.call(...)` / `fn.apply(...)` in value position (onError:
+ # handleError.bind(toast)) hands off `fn`; peel both to a fixpoint.
+ node = self._peel_bound_callable(node)
+ # Only a bare name / attribute / member-expression in value position names
+ # a function; a call, comprehension or literal is not a reference to a
+ # callable. Reuses the flow-arg ref types (identifier, Python attribute,
+ # Go selector, JS/TS member expression).
+ if node.type not in _FLOW_ARG_REF_TYPES:
+ return
+ self._emit_callback_edge(
+ caller_spec,
+ node,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ )
+
+ def _unwrap_ts_value(self, node: Node) -> Node:
+ # Peel TS casts AND parens, interleaved (`((s) => {...}) as SetState`),
+ # down to the wrapped value.
+ current = node
+ while current.type in _TS_BINDING_WRAPPER_TYPES:
+ inner = current.named_child(0)
+ if inner is None:
+ break
+ current = inner
+ return current
+
+ def _unwrap_ts_cast(self, node: Node) -> Node:
+ # Peel TS cast wrappers (`x as T`, `x satisfies T`, `x!`) to the wrapped
+ # value; they are transparent for reference resolution. The wrapped value
+ # is the first named child; casts nest (`x as unknown as T`), so loop.
+ current = node
+ while current.type in cs.TS_CAST_WRAPPER_TYPES:
+ inner = current.named_child(0)
+ if inner is None:
+ break
+ current = inner
+ return current
+
+ def _peel_bound_callable(self, node: Node, peel_parens: bool = False) -> Node:
+ # Iterate cast (and optionally paren) unwraps with the bound-call
+ # unwrap to a FIXPOINT: `(handler as any).bind(null)` interleaves a
+ # cast INSIDE the bind receiver and `h.bind(a).bind(b)` chains, so
+ # a single pass of either unwrap leaves a wrapper behind.
+ while True:
+ node = (
+ self._unwrap_ts_value(node)
+ if peel_parens
+ else self._unwrap_ts_cast(node)
+ )
+ bound = self._unwrap_bound_function(node)
+ if bound is None:
+ return node
+ node = bound
+
+ def _unwrap_bound_function(self, node: Node) -> Node | None:
+ # For `fn.bind(ctx)` (a call_expression whose function is `fn.bind`),
+ # return the bound function `fn` (the member object) so the value is
+ # referenced as `fn`, not the Function.prototype builtin. call/apply use
+ # the function the same way. Returns None when the node is not such a call.
+ if node.type != cs.TS_CALL_EXPRESSION:
+ return None
+ fn = node.child_by_field_name(cs.TS_FIELD_FUNCTION)
+ if fn is None or fn.type != cs.TS_MEMBER_EXPRESSION:
+ return None
+ prop = fn.child_by_field_name(cs.FIELD_PROPERTY)
+ if (
+ prop is None
+ or safe_decode_text(prop) not in cs.JS_FUNCTION_PROTOTYPE_METHODS
+ ):
+ return None
+ return fn.child_by_field_name(cs.FIELD_OBJECT)
+
+ def _emit_inline_value_function_ref(
+ self,
+ pair: Node,
+ value: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ ensure_rel,
+ ) -> None:
+ # An inline arrow/function-expression object value is registered by the
+ # definition pass under {enclosing_scope}.. An identifier key names it
+ # by the key (scope.onSuccess); a string-literal key ({'onSuccess': ...})
+ # has no property name, so it registers as scope.anonymous__
from
+ # the value's position. Reference every candidate actually registered
+ # (variants cover same-name duplicates in one scope).
+ registry = self._resolver.function_registry
+ scope_qn = caller_spec[2]
+ candidates = {
+ f"{scope_qn}{cs.SEPARATOR_DOT}{cs.PREFIX_ANONYMOUS}"
+ f"{value.start_point[0]}_{value.start_point[1]}"
+ }
+ key_node = pair.child_by_field_name(cs.FIELD_KEY)
+ if (
+ key_node is not None
+ and key_node.type in (cs.TS_PROPERTY_IDENTIFIER, cs.TS_IDENTIFIER)
+ and (key := safe_decode_text(key_node))
+ ):
+ candidates.add(f"{scope_qn}{cs.SEPARATOR_DOT}{key}")
+ # A value nested under ANOTHER pair-arrow (`{ onCreated: (s) => {
+ # s.setEvents({ compute: ... }) } }`) registers under the pair-key
+ # PATH (scope.onCreated.compute); prefix the ancestor pair keys so
+ # the candidate matches the def pass's qn.
+ if pair_path := self._ancestor_pair_key_path(pair):
+ candidates.add(
+ f"{scope_qn}{cs.SEPARATOR_DOT}{pair_path}{cs.SEPARATOR_DOT}{key}"
+ )
+ # A NAMED function expression value (`get: function getrouter() {...}`,
+ # express) registers by its OWN name; neither the key nor the position
+ # form matches it, so try the name under the scope and module-flat
+ # (where the def pass puts it).
+ name_node = value.child_by_field_name(cs.FIELD_NAME)
+ if name_node is not None and (own := safe_decode_text(name_node)):
+ candidates.add(f"{scope_qn}{cs.SEPARATOR_DOT}{own}")
+ candidates.add(f"{module_qn}{cs.SEPARATOR_DOT}{own}")
+ for candidate in candidates:
+ for target_qn in registry.variants(candidate):
+ if registry.get(target_qn) is None:
+ continue
+ ensure_rel(
+ caller_spec,
+ cs.RelationshipType.REFERENCES,
+ (cs.NodeLabel.FUNCTION, cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+
+ def _ancestor_pair_key_path(self, pair: Node) -> str | None:
+ # Dotted key path of the ENCLOSING pairs above this one (`compute` inside
+ # `onCreated: (s) => ...` -> "onCreated"), outermost first; None when the
+ # pair has no pair ancestors. Registry-guarded by the caller, so an
+ # over-collected path (ancestors above the scanning scope) just misses.
+ keys: list[str] = []
+ current = pair.parent
+ while current is not None:
+ if current.type == cs.TS_PY_PAIR:
+ key_node = current.child_by_field_name(cs.FIELD_KEY)
+ if (
+ key_node is not None
+ and key_node.type in (cs.TS_PROPERTY_IDENTIFIER, cs.TS_IDENTIFIER)
+ and (key := safe_decode_text(key_node))
+ ):
+ keys.append(key)
+ current = current.parent
+ if not keys:
+ return None
+ keys.reverse()
+ return cs.SEPARATOR_DOT.join(keys)
+
+ def _emit_shorthand_method_ref(
+ self,
+ method_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ ensure_rel,
+ ) -> None:
+ # The def pass registers a shorthand method by NAME at MODULE scope
+ # (persist's thenable `catch` -> `...middleware.persist.catch`), while
+ # this scan runs per enclosing caller; try both scopes, plus the
+ # position form used when the name is taken.
+ name_node = method_node.child_by_field_name(cs.FIELD_NAME)
+ registry = self._resolver.function_registry
+ scope_qn = caller_spec[2]
+ candidates = {
+ f"{scope_qn}{cs.SEPARATOR_DOT}{cs.PREFIX_ANONYMOUS}"
+ f"{method_node.start_point[0]}_{method_node.start_point[1]}"
+ }
+ if name_node is not None and (name := safe_decode_text(name_node)):
+ candidates.add(f"{scope_qn}{cs.SEPARATOR_DOT}{name}")
+ candidates.add(f"{module_qn}{cs.SEPARATOR_DOT}{name}")
+ for candidate in candidates:
+ for target_qn in registry.variants(candidate):
+ if registry.get(target_qn) is None:
+ continue
+ ensure_rel(
+ caller_spec,
+ cs.RelationshipType.REFERENCES,
+ (cs.NodeLabel.FUNCTION, cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+
+ def _ingest_default_param_references(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ caller_qn: str | None,
+ ) -> None:
+ params = caller_node.child_by_field_name(cs.FIELD_PARAMETERS)
+ if params is None:
+ return
+ resolve_func = self._resolver.resolve_function_call
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ for param in params.named_children:
+ # TS carries a param default in required_parameter's `value` field;
+ # plain JS wraps the param in an assignment_pattern whose default
+ # sits under `right`. Scan both forms.
+ value = param.child_by_field_name(
+ cs.FIELD_VALUE
+ ) or param.child_by_field_name(cs.FIELD_RIGHT)
+ if value is None:
+ continue
+ value = self._unwrap_ts_value(value)
+ if (
+ value.type in _ASSIGNMENT_RHS_REF_TYPES
+ or value.type in _INLINE_FUNC_VALUE_TYPES
+ ):
+ self._emit_callback_edge(
+ caller_spec,
+ value,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ )
+
+ def _ingest_inline_call_arg_references(
+ self,
+ call_node: Node,
+ caller_spec: tuple[str, str, str],
+ ensure_rel,
+ caller_qn: str | None,
+ module_qn: str | None = None,
+ ) -> None:
+ args = call_node.child_by_field_name(cs.FIELD_ARGUMENTS)
+ if args is None:
+ return
+ for arg in args.named_children:
+ value = self._unwrap_ts_value(arg)
+ if value.type in _INLINE_FUNC_VALUE_TYPES:
+ self._emit_inline_arg_function_ref(
+ value,
+ caller_spec,
+ ensure_rel,
+ caller_qn,
+ cs.RelationshipType.REFERENCES,
+ module_qn=module_qn,
+ )
+
+ def _emit_inline_arg_function_ref(
+ self,
+ arg_node: Node,
+ source_spec: tuple[str, str, str],
+ ensure_rel,
+ caller_qn: str | None = None,
+ rel_type: cs.RelationshipType = cs.RelationshipType.REFERENCES,
+ module_qn: str | None = None,
+ ) -> None:
+ # An inline arrow/function-expression call argument is registered by the
+ # definition pass as {enclosing_scope}.anonymous__
from its own
+ # start position. The anonymous node lives in the CALLER's scope, so build
+ # the candidate from caller_qn (source_spec[2] is the callee for the
+ # callable-param path). Registry guard skips unregistered names.
+ registry = self._resolver.function_registry
+ scope_qn = caller_qn or source_spec[2]
+ suffixes = [
+ f"{cs.SEPARATOR_DOT}{cs.PREFIX_ANONYMOUS}"
+ f"{arg_node.start_point[0]}_{arg_node.start_point[1]}"
+ ]
+ # A NAMED function expression argument (`this.on('mount', function
+ # onmount(parent) {...})`, express) registers by its own NAME; the
+ # position form never matches it, so try the name too, both under the
+ # caller scope and module-flat (where the def pass puts it).
+ name_node = arg_node.child_by_field_name(cs.FIELD_NAME)
+ named = safe_decode_text(name_node) if name_node is not None else None
+ if named:
+ suffixes.append(f"{cs.SEPARATOR_DOT}{named}")
+ # A duplicate-variant caller (a TS overload implementation registers as
+ # `useStore@27`) owns anons the def pass registers under the NATURAL qn
+ # (`useStore.anonymous_R_C`); try the variant-stripped scope too.
+ scopes = _scope_qn_candidates(scope_qn)
+ if named and module_qn is not None and module_qn not in scopes:
+ scopes = [*scopes, module_qn]
+ for scope in scopes:
+ for suffix in suffixes:
+ candidate = f"{scope}{suffix}"
+ for target_qn in registry.variants(candidate):
+ if registry.get(target_qn) is None:
+ continue
+ ensure_rel(
+ source_spec,
+ rel_type,
+ (cs.NodeLabel.FUNCTION, cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+
+ @staticmethod
+ def _flow_scope_boundaries(lang_config: LanguageSpec) -> frozenset[str]:
+ # A nested function/class scope; a scan of a function body must not descend
+ # into one, since its returns and local bindings belong to it, not the
+ # enclosing scope. The Python set is unioned in so Python keeps its exact
+ # prior behaviour (its decorated_definition wrapper is not a config type).
+ return (
+ _PY_SCOPE_BOUNDARY_TYPES
+ | frozenset(lang_config.function_node_types)
+ | frozenset(lang_config.class_node_types)
+ )
+
+ def _collect_returned_callables(
+ self,
+ caller_node: Node,
+ caller_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ boundary_types: frozenset[str],
+ ) -> None:
+ # Record which functions/closures this function may return, so a call site
+ # that binds and invokes the returned value (x = factory(); x(cb)) can flow
+ # cb into the returned closure. Only this scope's own return statements
+ # count; a nested function's returns belong to it.
+ registry = self._resolver.function_registry
+ resolve_func = self._resolver.resolve_function_call
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in boundary_types:
+ continue
+ if node.type == cs.TS_PY_RETURN_STATEMENT:
+ for returned in node.named_children:
+ child = self._unwrap_ts_cast(returned)
+ if child.type not in (cs.TS_PY_IDENTIFIER, cs.TS_PY_ATTRIBUTE):
+ continue
+ if not (name := safe_decode_text(child)):
+ continue
+ nested_qn = f"{caller_qn}{cs.SEPARATOR_DOT}{name}"
+ # A duplicated name (if/else twin definitions) returns
+ # whichever branch ran, so record EVERY twin; recording one
+ # leaves the others unreachable and falsely dead.
+ if nested_qn in registry:
+ self._returned_callables.setdefault(caller_qn, set()).update(
+ registry.variants(nested_qn)
+ )
+ elif (
+ resolved := resolve_func(
+ name, module_qn, local_var_types, class_context, caller_qn
+ )
+ ) is not None and resolved[0] in _CALLABLE_NODE_LABELS:
+ self._returned_callables.setdefault(caller_qn, set()).update(
+ registry.variants(resolved[1])
+ )
+ stack.extend(node.children)
+
+ def _build_factory_alias_map(
+ self,
+ caller_node: Node,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ boundary_types: frozenset[str],
+ caller_qn: str | None = None,
+ ) -> dict[str, str]:
+ # Map a local `x` to the function `factory` in `x = factory(...)`, so a
+ # later `x(cb)` can be traced through factory's returned closure. Handles
+ # each flow language's binding form (Python assignment, JS/TS
+ # variable_declarator, Go short_var_declaration, C++ init_declarator).
+ resolve_func = self._resolver.resolve_function_call
+ aliases: dict[str, str] = {}
+ stack: list[Node] = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in boundary_types:
+ continue
+ for var, fn_name in self._factory_bindings(node):
+ if (
+ resolved := resolve_func(
+ fn_name, module_qn, local_var_types, class_context, caller_qn
+ )
+ ) is not None:
+ aliases.setdefault(var, resolved[1])
+ stack.extend(node.children)
+ return aliases
+
+ def _factory_bindings(self, node: Node) -> list[tuple[str, str]]:
+ # Yield (local_name, called_function_name) for a `x = f(...)` binding node.
+ match node.type:
+ case cs.TS_PY_ASSIGNMENT:
+ return self._simple_factory_binding(
+ node, cs.TS_FIELD_LEFT, cs.TS_FIELD_RIGHT, cs.TS_PY_CALL
+ )
+ case cs.TS_VARIABLE_DECLARATOR:
+ return self._simple_factory_binding(
+ node, cs.TS_FIELD_NAME, cs.FIELD_VALUE, cs.TS_CALL_EXPRESSION
+ )
+ case cs.CppNodeType.INIT_DECLARATOR:
+ return self._simple_factory_binding(
+ node, cs.TS_FIELD_DECLARATOR, cs.FIELD_VALUE, cs.TS_CALL_EXPRESSION
+ )
+ case cs.TS_GO_SHORT_VAR_DECLARATION:
+ return self._go_factory_bindings(node)
+ case _:
+ return []
+
+ def _simple_factory_binding(
+ self, node: Node, name_field: str, value_field: str, call_type: str
+ ) -> list[tuple[str, str]]:
+ left = node.child_by_field_name(name_field)
+ right = node.child_by_field_name(value_field)
+ # `const x = factory(...) as T` wraps the call in a cast; unwrap so the
+ # factory binding is still recognised.
+ if right is not None:
+ right = self._unwrap_ts_cast(right)
+ if (
+ left is not None
+ and left.type == cs.TS_PY_IDENTIFIER
+ and right is not None
+ and right.type == call_type
+ and (var := safe_decode_text(left))
+ and (fn := right.child_by_field_name(cs.TS_FIELD_FUNCTION)) is not None
+ and (fn_name := safe_decode_text(fn))
+ ):
+ return [(var, fn_name)]
+ return []
+
+ def _go_factory_bindings(self, node: Node) -> list[tuple[str, str]]:
+ # Go `a, b := f(), g()`: pair each left identifier with the call in the
+ # same position of the right expression list.
+ left = node.child_by_field_name(cs.TS_FIELD_LEFT)
+ right = node.child_by_field_name(cs.TS_FIELD_RIGHT)
+ if left is None or right is None:
+ return []
+ names = [c for c in left.named_children if c.type == cs.TS_PY_IDENTIFIER]
+ calls = [c for c in right.named_children if c.type == cs.TS_CALL_EXPRESSION]
+ bindings: list[tuple[str, str]] = []
+ for name_node, call in zip(names, calls):
+ if (
+ (var := safe_decode_text(name_node))
+ and (fn := call.child_by_field_name(cs.TS_FIELD_FUNCTION)) is not None
+ and (fn_name := safe_decode_text(fn))
+ ):
+ bindings.append((var, fn_name))
+ return bindings
+
+ def _resolve_callback_qn(
+ self,
+ node: Node,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ caller_qn: str | None = None,
+ ) -> str | None:
+ if node.type not in (cs.TS_PY_IDENTIFIER, cs.TS_PY_ATTRIBUTE):
+ return None
+ if not (text := safe_decode_text(node)):
+ return None
+ resolved = self._resolver.resolve_function_call(
+ text, module_qn, local_var_types, class_context, caller_qn
+ )
+ if resolved is None or resolved[0] not in _CALLABLE_NODE_LABELS:
+ return None
+ return resolved[1]
+
+ def _record_factory_call(
+ self,
+ call_node: Node,
+ scope_qn: str,
+ factory_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ ) -> None:
+ positional, keyword = self._parse_call_arguments(call_node)
+ pos_qns = tuple(
+ self._resolve_callback_qn(
+ n, module_qn, local_var_types, class_context, scope_qn
+ )
+ or ""
+ for n in positional
+ )
+ kw_qns = tuple(
+ (name, qn)
+ for name, value in keyword.items()
+ if (
+ qn := self._resolve_callback_qn(
+ value, module_qn, local_var_types, class_context, scope_qn
+ )
+ )
+ )
+ if any(pos_qns) or kw_qns:
+ self._factory_calls.append(
+ _FactoryCall(scope_qn, factory_qn, pos_qns, kw_qns)
+ )
+
+ def _parse_call_arguments(
+ self, call_node: Node
+ ) -> tuple[list[Node], dict[str, Node]]:
+ positional: list[Node] = []
+ keyword: dict[str, Node] = {}
+ args_node = _find_call_arguments_node(call_node)
+ if args_node is None:
+ return positional, keyword
+ for child in args_node.named_children:
+ # C# wraps every argument expression in an `argument` node (which
+ # may carry a ref/out modifier or a name: colon); Dart wraps
+ # positional values the same way. Unwrap to the expression itself,
+ # its LAST named child, so downstream reference-type checks see
+ # the identifier, not the wrapper.
+ if (
+ child.type in (cs.TS_CSHARP_ARGUMENT, cs.TS_DART_ARGUMENT)
+ and child.named_children
+ ):
+ child = child.named_children[-1]
+ if child.type == cs.TS_DART_NAMED_ARGUMENT:
+ _add_dart_named_argument(child, keyword)
+ elif child.type == cs.TS_PY_KEYWORD_ARGUMENT:
+ _add_py_keyword_argument(child, keyword)
+ else:
+ positional.append(child)
+ return positional, keyword
+
+ def _emit_callback_edge(
+ self,
+ source_spec: tuple[str, str, str],
+ arg_node: Node,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ resolve_func,
+ ensure_rel,
+ caller_qn: str | None = None,
+ rel_type: cs.RelationshipType = cs.RelationshipType.CALLS,
+ language: cs.SupportedLanguage | None = None,
+ ) -> None:
+ # A TS cast (`handler as any`, `fn satisfies T`, `cb!`) is transparent
+ # for reference resolution, and `fn.bind(ctx)` / `.call` / `.apply` in
+ # argument position (addEventListener("click", handler.bind(this)),
+ # django admin's inlines.js) hands off `fn` while the call itself
+ # resolves to the Function.prototype builtin; peel both to a fixpoint.
+ arg_node = self._peel_bound_callable(arg_node)
+ # An arrow/function-expression passed DIRECTLY as a call argument
+ # (useCallback(() => {}), setTimeout(() => {}), arr.map(x => ...)) is
+ # registered anonymously in the enclosing scope but named after no
+ # identifier, so resolve_func cannot find it. The call consumes it, so
+ # reference it by position the same way inline object-literal values are.
+ if arg_node.type in _INLINE_FUNC_VALUE_TYPES:
+ self._emit_inline_arg_function_ref(
+ arg_node, source_spec, ensure_rel, caller_qn, rel_type
+ )
+ return
+ if not (arg_text := safe_decode_text(arg_node)):
+ return
+ if language == cs.SupportedLanguage.CSHARP:
+ # `Callback` passes the method group with explicit type
+ # arguments; methods register generic-free, so strip them.
+ arg_text = arg_text.split(cs.CHAR_ANGLE_OPEN, 1)[0]
+ # A bare method-group name binds the ENCLOSING type's method
+ # group; reference the whole overload family (the delegate
+ # type that selects one overload is invisible to syntax, and
+ # the trie's lexicographic pick can even land on a sibling
+ # class, Polly's EmptyAction). Falls through when the enclosing
+ # type has no such member.
+ if cs.SEPARATOR_DOT not in arg_text:
+ engine = self._resolver.type_inference.csharp_type_inference
+ # An in-scope LOCAL FUNCTION shadows any same-name member
+ # (C# scoping) and the trie cannot see it at all: reference
+ # the local group first (Serilog's CreateLogger passes its
+ # Dispose/DisposeAsync locals to the Logger ctor).
+ if locals_group := engine.csharp_local_function_group(
+ arg_text, caller_qn, module_qn
+ ):
+ for target_qn in locals_group:
+ ensure_rel(
+ source_spec,
+ rel_type,
+ (
+ cs.NodeLabel.FUNCTION,
+ cs.KEY_QUALIFIED_NAME,
+ target_qn,
+ ),
+ )
+ return
+ if family := engine.csharp_method_group_family(arg_text, caller_qn):
+ for target_qn in family:
+ ensure_rel(
+ source_spec,
+ rel_type,
+ (cs.NodeLabel.METHOD, cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+ return
+ if not (
+ resolved := resolve_func(
+ arg_text, module_qn, local_var_types, class_context, caller_qn
+ )
+ ):
+ return
+ res_type, res_qn = resolved
+ registry = self._resolver.function_registry
+ if res_type == cs.NodeLabel.CLASS:
+ init_qn = f"{res_qn}{cs.SEPARATOR_DOT}{cs.PY_METHOD_INIT}"
+ if init_qn not in registry:
+ return
+ res_type = cs.NodeLabel.METHOD
+ res_qn = init_qn
+ # Only callables are meaningful callback/reference targets: a value can
+ # resolve to an Interface/Type node (`selector = identity as Selector`
+ # resolves the cast's TYPE name in some paths), and emitting that
+ # produces schema-invalid edges.
+ if res_type not in (cs.NodeLabel.FUNCTION, cs.NodeLabel.METHOD):
+ return
+ # A Dart getter in argument position is a VALUE READ, not a tear-off:
+ # the getter-read pass owns those edges (with shadow handling), so
+ # emitting here would fabricate liveness for a local that hides the
+ # getter (issue #869).
+ if language == cs.SupportedLanguage.DART and registry.is_property(res_qn):
+ return
+ for target_qn in registry.variants(res_qn):
+ ensure_rel(
+ source_spec,
+ rel_type,
+ (res_type, cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+
+ def _ingest_callable_param_calls(
+ self,
+ call_node: Node,
+ callee_type: str,
+ callee_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ resolve_func,
+ ensure_rel,
+ caller_qn: str | None = None,
+ ) -> None:
+ if not (params := self._resolver.function_registry.callable_params(callee_qn)):
+ return
+ positional, keyword = self._parse_call_arguments(call_node)
+ source_spec = (callee_type, cs.KEY_QUALIFIED_NAME, callee_qn)
+ for param_name, index in params.items():
+ arg_node = keyword.get(param_name)
+ if arg_node is None and index < len(positional):
+ arg_node = positional[index]
+ if arg_node is not None:
+ self._emit_callback_edge(
+ source_spec,
+ arg_node,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ )
+
+ def _collect_callable_flow(
+ self,
+ call_node: Node,
+ callee_qn: str,
+ caller_qn: str,
+ caller_params: frozenset[str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ ) -> None:
+ # Record, for each call-site argument that names a callable, whether it is a
+ # concrete function or a parameter of the caller (a pass-through). The
+ # fixpoint in finalize propagates concretes through pass-through params to
+ # the functions that actually invoke them.
+ positional, keyword = self._parse_call_arguments(call_node)
+ items: list[tuple[int, str, Node]] = [
+ (index, "", node) for index, node in enumerate(positional)
+ ]
+ items.extend((-1, name, node) for name, node in keyword.items())
+ callable_labels = (
+ cs.NodeLabel.FUNCTION,
+ cs.NodeLabel.METHOD,
+ cs.NodeLabel.CLASS,
+ )
+ for position, keyword_name, raw_arg in items:
+ # `f(callback as any)` casts the argument; unwrap so a cast callable
+ # argument still flows.
+ # `outer(handler.bind(this))` forwards the bound handler through
+ # a pass-through param; peel .bind like the direct-argument path
+ # or the propagated flow edge is lost.
+ arg_node = self._peel_bound_callable(raw_arg)
+ if arg_node.type not in _FLOW_ARG_REF_TYPES:
+ continue
+ arg_text = safe_decode_text(arg_node)
+ if not arg_text:
+ continue
+ if arg_node.type == cs.TS_PY_IDENTIFIER and arg_text in caller_params:
+ self._flow_args.append(
+ _CallableFlowArg(
+ callee_qn, position, keyword_name, "", caller_qn, arg_text
+ )
+ )
+ continue
+ resolved = self._resolver.resolve_function_call(
+ arg_text, module_qn, local_var_types, class_context, caller_qn
+ )
+ if resolved is not None and resolved[0] in callable_labels:
+ self._flow_args.append(
+ _CallableFlowArg(
+ callee_qn, position, keyword_name, resolved[1], "", ""
+ )
+ )
+
+ def finalize_flow(self) -> None:
+ # Resolve deferred FLOWS_TO return-taint once every function body has
+ # been walked, so a callee processed after its caller still contributes
+ # its return edge and resource flow (issue #712).
+ self._flow_processor.finalize()
+
+ def finalize_callable_param_flow(self) -> None:
+ # Resolve the recorded call-site argument bindings to a fixpoint and emit a
+ # CALLS edge from every function that invokes a callable parameter to each
+ # concrete function that can reach it (directly or via pass-through params).
+ registry = self._resolver.function_registry
+ seeds: dict[tuple[str, str], set[str]] = defaultdict(set)
+ edges: dict[tuple[str, str], set[tuple[str, str]]] = defaultdict(set)
+ for arg in self._flow_args:
+ if arg.keyword:
+ param_name = arg.keyword
+ else:
+ callee_params = self._flow_param_names.get(arg.callee_qn)
+ if callee_params is None or not (
+ 0 <= arg.position < len(callee_params)
+ ):
+ continue
+ param_name = callee_params[arg.position]
+ slot = (arg.callee_qn, param_name)
+ if arg.source_concrete:
+ seeds[slot].add(arg.source_concrete)
+ else:
+ edges[slot].add((arg.source_caller, arg.source_param))
+
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ # A nested closure a function returns is reachable whenever that function
+ # is reached (created and handed back as the return value). Nested
+ # functions are no longer roots, so this producer edge keeps a genuinely
+ # used closure (a returned decorator/formatter) live without reviving the
+ # closures of an unreachable outer function.
+ for producer_qn, returned in self._returned_callables.items():
+ producer_type = registry.get(producer_qn)
+ if producer_type is None:
+ continue
+ prefix = f"{producer_qn}{cs.SEPARATOR_DOT}"
+ producer_spec = (producer_type, cs.KEY_QUALIFIED_NAME, producer_qn)
+ for closure_qn in returned:
+ if not closure_qn.startswith(prefix):
+ continue
+ closure_type = registry.get(closure_qn)
+ if closure_type is None:
+ continue
+ ensure_rel(
+ producer_spec,
+ cs.RelationshipType.CALLS,
+ (closure_type, cs.KEY_QUALIFIED_NAME, closure_qn),
+ )
+
+ for fc in self._factory_calls:
+ for closure_qn in self._returned_callables.get(fc.factory_qn, ()):
+ # The returned closure runs when the alias is called, so it is
+ # reachable from the enclosing scope.
+ closure_type = registry.get(closure_qn)
+ if closure_type is None:
+ continue
+ scope_type = registry.get(fc.scope_qn) or cs.NodeLabel.MODULE
+ ensure_rel(
+ (scope_type, cs.KEY_QUALIFIED_NAME, fc.scope_qn),
+ cs.RelationshipType.CALLS,
+ (closure_type, cs.KEY_QUALIFIED_NAME, closure_qn),
+ )
+ # Each argument the closure receives seeds its callable parameter,
+ # so the callback is reached wherever the closure invokes it.
+ closure_params = self._flow_param_names.get(closure_qn)
+ for index, callback_qn in enumerate(fc.positional):
+ if (
+ callback_qn
+ and closure_params is not None
+ and index < len(closure_params)
+ ):
+ seeds[(closure_qn, closure_params[index])].add(callback_qn)
+ for keyword_name, callback_qn in fc.keyword:
+ seeds[(closure_qn, keyword_name)].add(callback_qn)
+
+ bindings: dict[tuple[str, str], set[str]] = {
+ k: set(v) for k, v in seeds.items()
+ }
+ for slot in edges:
+ bindings.setdefault(slot, set())
+ changed = True
+ while changed:
+ changed = False
+ for slot, sources in edges.items():
+ for source in sources:
+ if (reachable := bindings.get(source)) and not reachable.issubset(
+ bindings[slot]
+ ):
+ bindings[slot] |= reachable
+ changed = True
+
+ for func_qn, invoked in (
+ (qn, registry.callable_params(qn)) for qn in self._flow_param_names
+ ):
+ if not invoked or (func_type := registry.get(func_qn)) is None:
+ continue
+ source_spec = (func_type, cs.KEY_QUALIFIED_NAME, func_qn)
+ for param_name in invoked:
+ for target_qn in bindings.get((func_qn, param_name), ()):
+ target_type = registry.get(target_qn)
+ if target_type is None:
+ continue
+ for variant in registry.variants(target_qn):
+ ensure_rel(
+ source_spec,
+ cs.RelationshipType.CALLS,
+ (target_type, cs.KEY_QUALIFIED_NAME, variant),
+ )
+
+ def _ingest_callable_field_calls(
+ self,
+ call_name: str,
+ caller_spec: tuple[str, str, str],
+ local_var_types: dict[str, str] | None,
+ ensure_rel,
+ ) -> None:
+ recv, sep, field = call_name.rpartition(cs.SEPARATOR_DOT)
+ if not sep:
+ return
+ recv_type = local_var_types.get(recv) if local_var_types else None
+ targets = self._resolver.callable_field_targets(field, recv_type)
+ if not targets:
+ return
+ registry = self._resolver.function_registry
+ for target_qn in targets:
+ if target_qn in registry:
+ ensure_rel(
+ caller_spec,
+ cs.RelationshipType.CALLS,
+ (registry[target_qn], cs.KEY_QUALIFIED_NAME, target_qn),
+ )
+
+ def _ingest_higher_order_builtin_calls(
+ self,
+ call_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ resolve_func,
+ ensure_rel,
+ caller_qn: str | None = None,
+ ) -> None:
+ positional, keyword = self._parse_call_arguments(call_node)
+ for arg_node in (*positional, *keyword.values()):
+ self._emit_callback_edge(
+ caller_spec,
+ arg_node,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ )
+
+ def _ingest_argument_function_references(
+ self,
+ call_node: Node,
+ caller_spec: tuple[str, str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ resolve_func,
+ ensure_rel,
+ caller_qn: str | None = None,
+ rel_type: cs.RelationshipType = cs.RelationshipType.CALLS,
+ language: cs.SupportedLanguage | None = None,
+ ) -> None:
+ # A function/method passed as an argument is a first-class value the
+ # callee may invoke (external framework) or store for later dynamic
+ # dispatch (first-party plumbing); either way the passing scope holds a
+ # live reference, so emit an edge to keep the callback reachable.
+ # External/builtin callees keep the CALLS edge; first-party callees
+ # record the pass as REFERENCES (the precise invocation edge, if any,
+ # comes from callable-param flow).
+ positional, keyword = self._parse_call_arguments(call_node)
+ for arg_node in (*positional, *keyword.values()):
+ # A container-literal or conditional argument
+ # (`validators=[_simple_domain_name_validator]`, django's Site;
+ # `... if single else local_setter_noop`, its SQLCompiler) hides
+ # the passed functions one level down; each expanded value is a
+ # first-class reference exactly like a bare-name argument.
+ for value_node in self._expand_py_first_class_values(arg_node, language):
+ self._emit_callback_edge(
+ caller_spec,
+ value_node,
+ module_qn,
+ local_var_types,
+ class_context,
+ resolve_func,
+ ensure_rel,
+ caller_qn,
+ rel_type,
+ language,
+ )
+
+ def _build_local_alias_map(
+ self, caller_node: Node, lang_config: LanguageSpec, module_qn: str
+ ) -> dict[str, str]:
+ identifier = cs.TS_PY_IDENTIFIER
+ attribute = cs.TS_PY_ATTRIBUTE
+ assignment = cs.TS_PY_ASSIGNMENT
+ left_field = cs.TS_FIELD_LEFT
+ right_field = cs.TS_FIELD_RIGHT
+ function_types = lang_config.function_node_types
+ class_types = lang_config.class_node_types
+ aliases: dict[str, str] = {}
+ stack = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ node_type = node.type
+ if node_type in function_types or node_type in class_types:
+ continue
+ if node_type == assignment:
+ left = node.child_by_field_name(left_field)
+ right = node.child_by_field_name(right_field)
+ if (
+ left is not None
+ and left.type == identifier
+ and (left_text := left.text) is not None
+ and right is not None
+ and (
+ target := self._alias_reference_text(
+ right, identifier, attribute, module_qn
+ )
+ )
+ is not None
+ ):
+ aliases.setdefault(left_text.decode(cs.ENCODING_UTF8), target)
+ stack.extend(node.children)
+ return aliases
+
+ def _alias_reference_text(
+ self, right: Node, identifier: str, attribute: str, module_qn: str
+ ) -> str | None:
+ # An alias rhs is a plain name/attribute, a conditional that picks one
+ # (resolve_builtin_call if is_js_ts else None), or getattr(recv, name)
+ # dynamic dispatch. Take the name/attribute branch (consequence or
+ # alternative, never the condition) or build recv. for getattr.
+ # A TS cast (`y as T`) is transparent; unwrap so `x = y as T` still aliases.
+ right = self._unwrap_ts_cast(right)
+ if right.type in (identifier, attribute):
+ return right.text.decode(cs.ENCODING_UTF8) if right.text else None
+ if right.type == cs.TS_PY_CONDITIONAL_EXPRESSION and right.named_children:
+ for branch in (right.named_children[0], right.named_children[-1]):
+ if branch.type in (identifier, attribute) and branch.text:
+ return branch.text.decode(cs.ENCODING_UTF8)
+ if right.type == cs.TS_PY_CALL:
+ return self._getattr_reference_text(right, identifier, attribute, module_qn)
+ return None
+
+ def _getattr_reference_text(
+ self, call: Node, identifier: str, attribute: str, module_qn: str
+ ) -> str | None:
+ func = call.child_by_field_name(cs.TS_FIELD_FUNCTION)
+ args = call.child_by_field_name(cs.FIELD_ARGUMENTS)
+ if (
+ func is None
+ or safe_decode_text(func) != cs.PY_BUILTIN_GETATTR
+ or args is None
+ or len(args.named_children) < 2
+ ):
+ return None
+ receiver, name_node = args.named_children[0], args.named_children[1]
+ if receiver.type not in (identifier, attribute):
+ return None
+ if (name := self._resolve_str_const(name_node, module_qn)) is None:
+ return None
+ return f"{safe_decode_text(receiver)}{cs.SEPARATOR_DOT}{name}"
+
+ def _resolve_str_const(self, node: Node, module_qn: str) -> str | None:
+ # Resolve a getattr name argument to its string value: a string literal
+ # directly, or a module-level constant (cs.METHOD_X / METHOD_X) read from
+ # the defining module's AST.
+ if node.type == cs.TS_PY_STRING:
+ content = next(
+ (c for c in node.children if c.type == cs.TS_PY_STRING_CONTENT), None
+ )
+ return safe_decode_text(content) if content is not None else None
+ if node.type not in (cs.TS_PY_IDENTIFIER, cs.TS_PY_ATTRIBUTE):
+ return None
+ name_text = safe_decode_text(node)
+ if not name_text:
+ return None
+ import_map = self._resolver.import_processor.import_mapping.get(module_qn, {})
+ prefix, _, const_name = name_text.rpartition(cs.SEPARATOR_DOT)
+ if not prefix:
+ mapped = import_map.get(const_name)
+ const_module_qn = (
+ mapped.rsplit(cs.SEPARATOR_DOT, 1)[0] if mapped else module_qn
+ )
+ elif (mapped_module := import_map.get(prefix)) is not None:
+ const_module_qn = mapped_module
+ else:
+ const_module_qn = prefix
+ return self._module_string_constant(const_module_qn, const_name)
+
+ def _module_string_constant(self, module_qn: str, const_name: str) -> str | None:
+ type_inference = self._resolver.type_inference
+ file_path = type_inference.module_qn_to_file_path.get(module_qn)
+ if file_path is None or not (entry := type_inference.ast_cache.load(file_path)):
+ return None
+ root_node, _ = entry
+ for child in root_node.children:
+ if child.type != cs.TS_PY_EXPRESSION_STATEMENT or not child.children:
+ continue
+ assignment = child.children[0]
+ if assignment.type != cs.TS_PY_ASSIGNMENT:
+ continue
+ left = assignment.child_by_field_name(cs.TS_FIELD_LEFT)
+ right = assignment.child_by_field_name(cs.TS_FIELD_RIGHT)
+ if (
+ left is not None
+ and left.type == cs.TS_PY_IDENTIFIER
+ and safe_decode_text(left) == const_name
+ and right is not None
+ and right.type == cs.TS_PY_STRING
+ ):
+ return self._resolve_str_const(right, module_qn)
+ return None
+
+ def _ingest_property_accesses(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ caller_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ lang_config: LanguageSpec,
+ prop_names: set[str],
+ ) -> None:
+ # Accessing an @property getter invokes the getter method at runtime, but
+ # tree-sitter sees a plain attribute, not a call. Resolve attribute
+ # accesses whose tail names a known property and emit a CALLS edge to the
+ # getter (skipping the attribute that is itself a call's function, already
+ # resolved by the call path above).
+ resolver = self._resolver
+ resolve_func = resolver.resolve_function_call
+ registry = resolver.function_registry
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ calls_rel = cs.RelationshipType.CALLS
+ qn_key = cs.KEY_QUALIFIED_NAME
+ method_label = cs.NodeLabel.METHOD
+ attr_type = cs.TS_PY_ATTRIBUTE
+ call_type = cs.TS_PY_CALL
+ func_field = cs.TS_FIELD_FUNCTION
+ function_types = lang_config.function_node_types
+ class_types = lang_config.class_node_types
+ seen: set[str] = set()
+
+ stack = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ node_type = node.type
+ if node_type in function_types or node_type in class_types:
+ continue
+ if node_type == attr_type and (text := node.text) is not None:
+ attr_text = text.decode(cs.ENCODING_UTF8)
+ if attr_text.rsplit(cs.SEPARATOR_DOT, 1)[-1] in prop_names:
+ parent = node.parent
+ is_call_target = (
+ parent is not None
+ and parent.type == call_type
+ and parent.child_by_field_name(func_field) is node
+ )
+ if not is_call_target and (
+ callee_info := resolve_func(
+ attr_text,
+ module_qn,
+ local_var_types,
+ class_context,
+ caller_qn,
+ )
+ ):
+ callee_qn = callee_info[1]
+ if (
+ registry.is_property(callee_qn)
+ and callee_qn != caller_qn
+ and callee_qn not in seen
+ ):
+ seen.add(callee_qn)
+ for target_qn in registry.variants(callee_qn):
+ ensure_rel(
+ caller_spec,
+ calls_rel,
+ (method_label, qn_key, target_qn),
+ )
+ stack.extend(node.children)
+
+ def _ingest_dart_getter_reads(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ caller_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ lang_config: LanguageSpec,
+ prop_names: set[str],
+ ) -> None:
+ # Emit a REFERENCES edge (a read is not an invocation; the call graph
+ # stays invocation-only) from the caller to each getter it reads:
+ # bare identifiers resolve against the enclosing class (implicit
+ # this), member selectors reassemble their receiver chain and resolve
+ # through receiver typing. Registry-guarded: only resolutions landing
+ # on a MARKED property emit, so unrelated same-name locals or
+ # functions never fabricate an edge. Nested closures are walked (a
+ # Dart lambda's reads attribute to the enclosing method, matching the
+ # flat-attribute call pass), so their parameters join the shadow set.
+ class_types = lang_config.class_node_types
+ seen: set[str] = set()
+
+ # The grammar splits a definition into a signature node and a SIBLING
+ # function_body: the signature holds no reads, so walk the body.
+ body = dart_utils.dart_body_node(caller_node)
+ walk_root = body if body is not None else caller_node
+ # The bodiless caller is the MODULE pass: FIELD INITIALIZERS read
+ # getters outside any method body (`late final body =
+ # _createFont(contentFont, ..)`), so each class subtree is walked by
+ # a per-class sub-pass carrying the OWNING class as resolution
+ # context. Method callers keep skipping nested classes instead.
+ is_module_scope = body is None
+ shadow_cell: list[dict[str, list[tuple[int, int]]]] = []
+
+ def shadow_spans() -> dict[str, list[tuple[int, int]]]:
+ # Built lazily on the first bare read; most callers have none.
+ if not shadow_cell:
+ shadow_cell.append(self._dart_shadow_spans(caller_node, walk_root))
+ return shadow_cell[0]
+
+ stack = list(walk_root.children)
+ while stack:
+ node = stack.pop()
+ if node.type in class_types:
+ # Class subtrees (field initializers) are scanned once at
+ # FILE level by _ingest_dart_class_initializer_reads.
+ continue
+ if is_module_scope and _dart_is_owned_function_body(node):
+ continue
+ read_name = self._dart_read_name(node, prop_names, shadow_spans)
+ if read_name:
+ self._emit_dart_property_read(
+ read_name,
+ caller_spec,
+ caller_qn,
+ module_qn,
+ local_var_types,
+ class_context,
+ seen,
+ )
+ stack.extend(node.children)
+
+ def _ingest_dart_class_initializer_reads(
+ self,
+ class_node: Node,
+ caller_spec: tuple[str, str, str],
+ caller_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ lang_config: LanguageSpec,
+ prop_names: set[str],
+ parent_qn: str | None = None,
+ ) -> None:
+ # Field initializers run in the class's OWN scope: bare reads resolve
+ # against the owning class (two classes may share a getter name), and
+ # each class gets its own dedup set. A closure inside an initializer
+ # belongs to no method pass, so its body is walked here; only
+ # method/constructor bodies (owned by a class-body signature) have
+ # their own pass. Dart forbids nested class declarations, but the
+ # defensive recursion still threads the enclosing context through.
+ class_types = lang_config.class_node_types
+ name_node = next(
+ (
+ child
+ for child in class_node.named_children
+ if child.type == cs.TS_DART_IDENTIFIER
+ ),
+ None,
+ )
+ class_name = safe_decode_text(name_node) if name_node is not None else None
+ owner_qn = parent_qn if parent_qn is not None else module_qn
+ class_ctx = f"{owner_qn}{cs.SEPARATOR_DOT}{class_name}" if class_name else None
+ seen: set[str] = set()
+ shadow_cell: list[dict[str, list[tuple[int, int]]]] = []
+
+ def shadow_spans() -> dict[str, list[tuple[int, int]]]:
+ # Initializer closures declare parameters; their spans confine
+ # any shadowing to the closure itself. Member signatures and
+ # owned bodies stay out: a method parameter scopes its own body,
+ # which this walk never reads.
+ if not shadow_cell:
+ shadow_cell.append(
+ self._dart_shadow_spans(
+ class_node, class_node, skip_owned_members=True
+ )
+ )
+ return shadow_cell[0]
+
+ stack = list(class_node.children)
+ while stack:
+ node = stack.pop()
+ if node.type in class_types:
+ self._ingest_dart_class_initializer_reads(
+ node,
+ caller_spec,
+ caller_qn,
+ module_qn,
+ local_var_types,
+ lang_config,
+ prop_names,
+ parent_qn=class_ctx,
+ )
+ continue
+ if _dart_is_owned_function_body(node):
+ continue
+ read_name = self._dart_read_name(node, prop_names, shadow_spans)
+ if read_name:
+ self._emit_dart_property_read(
+ read_name,
+ caller_spec,
+ caller_qn,
+ module_qn,
+ local_var_types,
+ class_ctx,
+ seen,
+ )
+ stack.extend(node.children)
+
+ def _dart_read_name(
+ self,
+ node: Node,
+ prop_names: set[str],
+ shadow_spans: Callable[[], dict[str, list[tuple[int, int]]]],
+ ) -> str | None:
+ # The dotted name this node READS, or None: member selectors and
+ # cascades reassemble their receiver chain; a bare identifier counts
+ # only when no in-scope local/parameter shadows it.
+ node_type = node.type
+ if node_type == cs.TS_DART_SELECTOR:
+ read_name = dart_utils.dart_member_read_name(node)
+ elif node_type == cs.TS_DART_CASCADE_SECTION:
+ read_name = dart_utils.dart_cascade_read_name(node)
+ elif node_type == cs.TS_DART_IDENTIFIER and _dart_is_bare_read(node):
+ read_name = self._dart_unshadowed_name(node, prop_names, shadow_spans)
+ else:
+ return None
+ if read_name and read_name.rsplit(cs.SEPARATOR_DOT, 1)[-1] in prop_names:
+ return read_name
+ return None
+
+ def _dart_unshadowed_name(
+ self,
+ node: Node,
+ prop_names: set[str],
+ shadow_spans: Callable[[], dict[str, list[tuple[int, int]]]],
+ ) -> str | None:
+ name = safe_decode_text(node)
+ if not name or name not in prop_names:
+ return None
+ pos = node.start_byte
+ if any(lo <= pos < hi for lo, hi in shadow_spans().get(name, ())):
+ return None
+ return name
+
+ def _emit_dart_property_read(
+ self,
+ read_name: str,
+ caller_spec: tuple[str, str, str],
+ caller_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ seen: set[str],
+ ) -> None:
+ if read_name in seen:
+ return
+ registry = self._resolver.function_registry
+ res_qn: str | None = None
+ # A bare read inside a class binds the OWNING class's member first:
+ # the trie fallback is name-global and would pick a same-named getter
+ # from whichever class sorts first (a module-caller's caller_qn
+ # carries no class to guide it).
+ if class_context is not None and cs.SEPARATOR_DOT not in read_name:
+ candidate = f"{class_context}{cs.SEPARATOR_DOT}{read_name}"
+ if registry.get(candidate) is not None:
+ res_qn = candidate
+ if res_qn is None:
+ resolved = self._resolver.resolve_function_call(
+ read_name, module_qn, local_var_types, class_context, caller_qn
+ )
+ if not resolved:
+ return
+ res_qn = resolved[1]
+ if not registry.is_property(res_qn) or res_qn == caller_qn:
+ return
+ seen.add(read_name)
+ self.ingestor.ensure_relationship_batch(
+ caller_spec,
+ cs.RelationshipType.REFERENCES,
+ (cs.NodeLabel.METHOD, cs.KEY_QUALIFIED_NAME, res_qn),
+ )
+
+ def _dart_shadow_spans(
+ self,
+ caller_node: Node,
+ walk_root: Node,
+ skip_owned_members: bool = False,
+ ) -> dict[str, list[tuple[int, int]]]:
+ # {declared name: [byte span of each declaring SCOPE]} for parameters
+ # and locals. A declaration shadows a same-name getter only for bare
+ # reads INSIDE its declaring scope: a closure's parameter must not
+ # suppress the enclosing method's read (cf. _csharp_shadow_spans).
+ # Method parameters live in the SIGNATURE, outside the walked body,
+ # so their scope walk falls through to the whole body.
+ spans: dict[str, list[tuple[int, int]]] = {}
+ stack = list(caller_node.children)
+ if walk_root is not caller_node:
+ stack.extend(walk_root.children)
+ while stack:
+ node = stack.pop()
+ # For the class-initializer pass, member signatures and owned
+ # bodies are invisible to the READ walk, so their parameters and
+ # locals must not join the shadow map either: a method parameter
+ # scopes its own body, never a sibling field initializer.
+ if skip_owned_members and (
+ node.type in cs.DART_SIGNATURE_TYPES
+ or _dart_is_owned_function_body(node)
+ ):
+ continue
+ for name in _dart_declared_names(node):
+ spans.setdefault(name, []).append(_dart_scope_span(node, walk_root))
+ stack.extend(node.children)
+ return spans
+
+ def _csharp_read_identifier(self, receiver: Node | None) -> str | None:
+ # The identifier actually being READ in receiver position: unwrap
+ # parens, a cast's VALUE (`((IDictionary<...>)WrappedDictionary)`),
+ # and a null-forgiving postfix (`s!`) down to a bare identifier.
+ while receiver is not None:
+ if receiver.type == cs.TS_PARENTHESIZED_EXPRESSION:
+ receiver = (
+ receiver.named_children[0] if receiver.named_children else None
+ )
+ continue
+ if receiver.type == cs.TS_CSHARP_CAST_EXPRESSION:
+ receiver = receiver.child_by_field_name(cs.FIELD_VALUE)
+ continue
+ if receiver.type == cs.TS_CSHARP_POSTFIX_UNARY_EXPRESSION:
+ receiver = (
+ receiver.named_children[0] if receiver.named_children else None
+ )
+ continue
+ break
+ if receiver is not None and receiver.type == cs.TS_CSHARP_IDENTIFIER:
+ return safe_decode_text(receiver)
+ return None
+
+ def _csharp_shadow_spans(
+ self,
+ caller_node: Node,
+ function_types: tuple[str, ...],
+ class_types: tuple[str, ...],
+ ) -> dict[str, list[tuple[int, int]]]:
+ # {declared name: [byte span of each declaring SCOPE]} for every
+ # declaration in the scopes the READ WALK descends into (locals
+ # incl. untyped `var x = 3`, parameters, and a simple lambda's
+ # bare implicit_parameter). A declaration shadows a same-name
+ # property only for reads INSIDE its declaring scope: a lambda
+ # param or a sibling block's local must not suppress an outer
+ # read, while an in-lambda shadowed read must not fabricate a
+ # property reference. The two walks skip the SAME nested
+ # function/class scopes (each has its own pass).
+ def scope_span(decl: Node) -> tuple[int, int]:
+ anc = decl.parent
+ while anc is not None and anc != caller_node:
+ if anc.type in (cs.TS_CSHARP_BLOCK, cs.TS_CSHARP_LAMBDA_EXPRESSION):
+ return (anc.start_byte, anc.end_byte)
+ anc = anc.parent
+ return (caller_node.start_byte, caller_node.end_byte)
+
+ spans: dict[str, list[tuple[int, int]]] = {}
+ stack = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ node_type = node.type
+ if node_type in function_types or node_type in class_types:
+ continue
+ name = None
+ if node_type in (cs.TS_CSHARP_VARIABLE_DECLARATOR, cs.TS_CSHARP_PARAMETER):
+ name = safe_decode_text(node.child_by_field_name(cs.FIELD_NAME))
+ elif node_type == cs.TS_CSHARP_IMPLICIT_PARAMETER:
+ name = safe_decode_text(node)
+ if name:
+ spans.setdefault(name, []).append(scope_span(node))
+ stack.extend(node.children)
+ return spans
+
+ def _ingest_csharp_property_reads(
+ self,
+ caller_node: Node,
+ caller_spec: tuple[str, str, str],
+ caller_qn: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ lang_config: LanguageSpec,
+ prop_names: set[str],
+ ) -> None:
+ # Emit a REFERENCES edge (a read is not an invocation; the call
+ # graph stays invocation-only) from the caller to each property of
+ # its enclosing type read in receiver position. Registry-guarded via
+ # resolve_property_read, which accepts only marked properties.
+ engine = self._resolver.type_inference.csharp_type_inference
+ ensure_rel = self.ingestor.ensure_relationship_batch
+ refs_rel = cs.RelationshipType.REFERENCES
+ qn_key = cs.KEY_QUALIFIED_NAME
+ method_label = cs.NodeLabel.METHOD
+ function_types = lang_config.function_node_types
+ class_types = lang_config.class_node_types
+ shadowed: dict[str, list[tuple[int, int]]] | None = None
+ seen: set[str] = set()
+
+ def try_emit(name: str | None, read_node: Node, this_read: bool) -> None:
+ nonlocal shadowed
+ if not name or name not in prop_names or name in seen:
+ return
+ if shadowed is None:
+ shadowed = self._csharp_shadow_spans(
+ caller_node, function_types, class_types
+ )
+ pos = read_node.start_byte
+ is_shadowed = any(lo <= pos < hi for lo, hi in shadowed.get(name, ()))
+ if (this_read or not is_shadowed) and (
+ prop_qn := engine.resolve_property_read(name, caller_qn)
+ ):
+ seen.add(name)
+ if prop_qn != caller_qn:
+ ensure_rel(caller_spec, refs_rel, (method_label, qn_key, prop_qn))
+
+ stack = list(caller_node.children)
+ while stack:
+ node = stack.pop()
+ node_type = node.type
+ if node_type in function_types or node_type in class_types:
+ continue
+ if node_type == cs.TS_CSHARP_MEMBER_ACCESS_EXPRESSION:
+ receiver = node.child_by_field_name(cs.TS_CSHARP_FIELD_EXPRESSION)
+ # `this.Size` is ALWAYS the member (a local can never
+ # shadow a this-qualified read), so the read is the NAME
+ # field and the shadow check does not apply.
+ this_read = receiver is not None and receiver.type == cs.TS_CSHARP_THIS
+ if this_read:
+ try_emit(
+ safe_decode_text(node.child_by_field_name(cs.FIELD_NAME)),
+ node,
+ True,
+ )
+ else:
+ recv_name = self._csharp_read_identifier(receiver)
+ try_emit(recv_name, node, False)
+ # The NAME field can be a property of the RECEIVER's
+ # type: `Cfg.Value` (static, class-name receiver),
+ # `w.Inner` (instance, local of inferred type), or
+ # `N.Cfg.Value` (namespace/type-qualified: a dotted
+ # receiver of all-PascalCase segments names a type, a
+ # camelCase head is an expression chain and is skipped).
+ # An unresolvable receiver yields nothing, so unrelated
+ # chains never fabricate an edge.
+ member = safe_decode_text(node.child_by_field_name(cs.FIELD_NAME))
+ recv_type = None
+ if recv_name:
+ recv_type = (local_var_types or {}).get(recv_name, recv_name)
+ elif (
+ receiver is not None
+ and receiver.type == cs.TS_CSHARP_MEMBER_ACCESS_EXPRESSION
+ ):
+ dotted = safe_decode_text(receiver)
+ if dotted and all(
+ seg[:1].isupper() for seg in dotted.split(cs.SEPARATOR_DOT)
+ ):
+ recv_type = dotted
+ if recv_type and member and member in prop_names:
+ prop_qn = engine.resolve_member_property_read(
+ recv_type, member, module_qn
+ )
+ if (
+ prop_qn is not None
+ and prop_qn != caller_qn
+ and prop_qn not in seen
+ ):
+ seen.add(prop_qn)
+ ensure_rel(
+ caller_spec, refs_rel, (method_label, qn_key, prop_qn)
+ )
+ elif node_type == cs.TS_CSHARP_IDENTIFIER:
+ # A bare identifier expression (`return Size;`, `Use(Size)`,
+ # `var n = Size;`) reads the getter just the same. NOT a
+ # read: any member-access position (receiver/name handled
+ # above), a parent's NAME field (a declarator/parameter's
+ # own name, a named-argument label `Use(Size: 3)`), or a
+ # parent's TYPE field (`new Size()`, `(Size)x`).
+ # `==`, not `is`: child_by_field_name returns a fresh Node
+ # wrapper each call, so identity never matches.
+ parent = node.parent
+ if (
+ parent is not None
+ and parent.type != cs.TS_CSHARP_MEMBER_ACCESS_EXPRESSION
+ and parent.child_by_field_name(cs.FIELD_NAME) != node
+ and parent.child_by_field_name(cs.FIELD_TYPE) != node
+ ):
+ try_emit(safe_decode_text(node), node, False)
+ stack.extend(node.children)
+
+ def _build_nested_qualified_name(
+ self,
+ func_node: Node,
+ module_qn: str,
+ func_name: str,
+ lang_config: LanguageSpec,
+ ) -> str | None:
+ path_parts: list[str] = []
+ current = func_node.parent
+
+ if not isinstance(current, Node):
+ logger.warning(
+ ls.CALL_UNEXPECTED_PARENT, node=func_node, parent_type=type(current)
+ )
+ return None
+
+ while current and current.type not in lang_config.module_node_types:
+ if current.type in lang_config.function_node_types:
+ name_node = current.child_by_field_name(cs.FIELD_NAME)
+ if name_node is not None:
+ if name_node.text is not None:
+ path_parts.append(name_node.text.decode(cs.ENCODING_UTF8))
+ # A JS/TS arrow-const ancestor (`getQueryString = () => {...}`) has
+ # no `name` field (the name lives on the parent declarator), so it
+ # would be dropped, flattening a nested callee's qn (request.encodePair
+ # instead of request.getQueryString.encodePair). Recover the binding
+ # name the definition pass used so the qns agree; else the callee's
+ # own inline-arg/object callbacks never match and report dead.
+ elif lang_config.language in _JS_TS_LANGUAGES and (
+ binding := self._js_ts_arrow_binding_name(current)
+ ):
+ path_parts.append(binding)
+ elif current.type in lang_config.class_node_types:
+ return None
+
+ current = current.parent
+
+ path_parts.reverse()
if path_parts:
return f"{module_qn}{cs.SEPARATOR_DOT}{cs.SEPARATOR_DOT.join(path_parts)}{cs.SEPARATOR_DOT}{func_name}"
return f"{module_qn}{cs.SEPARATOR_DOT}{func_name}"
+ def _js_ts_arrow_binding_name(self, func_node: Node) -> str | None:
+ # An arrow / function expression has no `name` field, so the call pass
+ # skipped it and never processed its body's calls. Recover the binding
+ # name for the two named forms whose value IS the arrow: a module/local
+ # `const f = () => ...` (variable_declarator) and a class field
+ # `helper = () => ...` (public_field_definition). The body's calls then
+ # attribute to the qn the definition pass registered. Anonymous /
+ # destructured arrows stay unnamed (skipped).
+ if func_node.type not in (cs.TS_ARROW_FUNCTION, cs.TS_FUNCTION_EXPRESSION):
+ return None
+ # The arrow may be bound THROUGH transparent wrappers: parens and TS
+ # casts (`export const createStore = ((s) => ...) as CreateStore`,
+ # zustand's public-API shape). The def pass unwraps these when naming the
+ # function, so the call pass must climb them too or the arrow looks
+ # anonymous and its body's calls drop at module scope.
+ node = func_node
+ parent = node.parent
+ while parent is not None and parent.type in _TS_BINDING_WRAPPER_TYPES:
+ node = parent
+ parent = node.parent
+ if parent is None:
+ return None
+ # node must be the parent's value/initializer for both forms
+ # (variable_declarator and public_field_definition), so one value check
+ # covers both. `==` not `is`: py-tree-sitter returns a fresh Node wrapper
+ # per access, so identity comparison always fails (Node `==` compares id).
+ if parent.child_by_field_name(cs.FIELD_VALUE) != node:
+ return None
+ name_node = parent.child_by_field_name(cs.FIELD_NAME)
+ if name_node is None or name_node.type not in (
+ cs.TS_IDENTIFIER,
+ cs.TS_PROPERTY_IDENTIFIER,
+ ):
+ return None
+ return safe_decode_text(name_node)
+
+ def _attributable_func_nodes(
+ self, func_nodes: list[Node], language: cs.SupportedLanguage
+ ) -> list[Node]:
+ # The func nodes that will get their own caller node: named functions
+ # plus arrows/function-expressions bound to a name. An anonymous arrow
+ # passed directly as an argument (`hooks.tap(name, (x) => {...})`) has
+ # neither, so the call loop skips it. Its calls must therefore NOT be
+ # excluded from the enclosing named scope by _calls_owned_by; otherwise
+ # they attribute to nothing and drop (webpack is saturated with this
+ # callback pattern). Returning only attributable nodes as the exclusion
+ # set lets an anonymous arrow's calls bubble up to the nearest named
+ # function/method, matching where the oracle attributes them.
+ if language == cs.SupportedLanguage.RUST:
+ # A Rust closure (`expire.map(|_| state.next_expiration())`) is unnamed,
+ # so the call loop skips it (no caller node); like JS/TS anon arrows its
+ # calls must bubble to the enclosing fn/method (which holds the captured
+ # locals' types) instead of being excluded and dropped. Named nested
+ # `fn`s stay attributable and keep their own calls.
+ return [n for n in func_nodes if n.type != cs.TS_RS_CLOSURE_EXPRESSION]
+ if language not in _JS_TS_LANGUAGES:
+ return func_nodes
+ return [
+ n
+ for n in func_nodes
+ if self._get_node_name(n) or self._js_ts_arrow_binding_name(n)
+ ]
+
+ def _is_unowned_js_scope(self, node: Node) -> bool:
+ # An anonymous arrow/function-expression that gets no caller node of its
+ # own (no name, no binding name): a `.map()`/`cell`/forwardRef callback.
+ # Its calls bubble up to the enclosing named scope (_attributable_func_nodes),
+ # and so must its JSX references: the enclosing JSX walk continues through it.
+ if node.type not in (cs.TS_ARROW_FUNCTION, cs.TS_FUNCTION_EXPRESSION):
+ return False
+ return not (self._get_node_name(node) or self._js_ts_arrow_binding_name(node))
+
def _is_method(self, func_node: Node, lang_config: LanguageSpec) -> bool:
return is_method_node(func_node, lang_config)
diff --git a/codebase_rag/parsers/call_resolver.py b/codebase_rag/parsers/call_resolver.py
index 322a583a3..117cee57f 100644
--- a/codebase_rag/parsers/call_resolver.py
+++ b/codebase_rag/parsers/call_resolver.py
@@ -1,6 +1,7 @@
from __future__ import annotations
import re
+from collections import defaultdict, deque
from loguru import logger
from tree_sitter import Node
@@ -11,30 +12,238 @@
from .import_processor import ImportProcessor
from .py import resolve_class_name
from .type_inference import TypeInferenceEngine
+from .utils import follow_reexports
+
+_SEPARATOR_PATTERN = re.compile(r"[.:]|::")
+_SEARCH_NAME_CACHE: dict[str, str] = {}
+_CHAINED_METHOD_PATTERN = re.compile(r"\.([^.()]+)$")
+_QN_SPLIT_CACHE: dict[str, tuple[list[str], int]] = {}
+_CHAIN_OPEN_BRACKETS = "([{"
+_CHAIN_CLOSE_BRACKETS = ")]}"
+# Node labels a Rust receiver type name may resolve to: a struct (Class), an
+# enum, a type alias, or a trait (Interface, when the receiver is typed to a
+# `dyn`/`impl` trait or a trait-returning factory); all can carry methods.
+_RS_TYPE_NODE_TYPES = frozenset(
+ {NodeType.CLASS, NodeType.ENUM, NodeType.TYPE, NodeType.INTERFACE}
+)
+
+
+def _split_receiver_chain(expr: str) -> list[str]:
+ # Split a receiver chain (`c.Find(1.5).Root`) on the `.` separators between
+ # hops only, never on a `.` inside call arguments, an index, or a generic
+ # (`1.5`, `x.y` args, `List`), which a naive str.split would mangle.
+ parts: list[str] = []
+ depth = 0
+ current: list[str] = []
+ for char in expr:
+ if char in _CHAIN_OPEN_BRACKETS:
+ depth += 1
+ elif char in _CHAIN_CLOSE_BRACKETS:
+ depth = max(0, depth - 1)
+ if char == cs.SEPARATOR_DOT and depth == 0:
+ parts.append("".join(current))
+ current = []
+ else:
+ current.append(char)
+ parts.append("".join(current))
+ return parts
class CallResolver:
+ __slots__ = (
+ "function_registry",
+ "import_processor",
+ "type_inference",
+ "class_inheritance",
+ "type_aliases",
+ "interface_implementers",
+ "_interface_impl_cache",
+ "_simple_resolution_cache",
+ "_wildcard_cache",
+ "_protocol_impl_cache",
+ "_field_bindings",
+ "_field_to_classes",
+ "_subclass_map_cache",
+ "_protocol_classes_cache",
+ "_struct_impl_cache",
+ "_ctor_params",
+ "_ctor_param_attrs",
+ "_pending_field_bindings",
+ )
+
def __init__(
self,
function_registry: FunctionRegistryTrieProtocol,
import_processor: ImportProcessor,
type_inference: TypeInferenceEngine,
class_inheritance: dict[str, list[str]],
+ type_aliases: dict[str, str] | None = None,
+ interface_implementers: dict[str, set[str]] | None = None,
) -> None:
self.function_registry = function_registry
self.import_processor = import_processor
self.type_inference = type_inference
self.class_inheritance = class_inheritance
+ # {interface_qn: [implementer_qns]} (shared ref, populated during
+ # ingestion). Used to redirect an interface-typed call to the single
+ # concrete implementer's method (call-graph accuracy; single-impl only).
+ self.interface_implementers = (
+ interface_implementers if interface_implementers is not None else {}
+ )
+ self._interface_impl_cache: dict[str, str] | None = None
+ # C++ typedef/using alias -> underlying bare type, consulted when a
+ # receiver type name is mapped to a class (empty for other languages).
+ self.type_aliases = type_aliases if type_aliases is not None else {}
+ self._simple_resolution_cache: dict[
+ tuple[str, str], tuple[str, str] | None
+ ] = {}
+ self._wildcard_cache: dict[int, list[tuple[str, str]]] = {}
+ self._protocol_impl_cache: dict[str, str] | None = None
+ self._field_bindings: dict[tuple[str, str], set[str]] = {}
+ self._field_to_classes: dict[str, set[str]] = {}
+ self._subclass_map_cache: dict[str, set[str]] | None = None
+ self._protocol_classes_cache: set[str] | None = None
+ self._struct_impl_cache: dict[str, set[str]] = {}
+ # Ordered constructor parameter names per class (explicit __init__
+ # params, or annotated class-body fields for NamedTuple/dataclass),
+ # plus the param -> stored-attribute renames found in __init__ bodies
+ # (self.ctx_factory = create_context). Construction-site bindings are
+ # held PENDING until every file's ctor metadata is collected, since a
+ # site may be scanned before its class's file.
+ self._ctor_params: dict[str, tuple[str, ...]] = {}
+ self._ctor_param_attrs: dict[tuple[str, str], str] = {}
+ self._pending_field_bindings: list[tuple[str, int | str, str]] = []
+
+ def record_ctor_params(self, class_qn: str, params: tuple[str, ...]) -> None:
+ self._ctor_params[class_qn] = params
+
+ def record_ctor_param_attr(self, class_qn: str, param: str, attr: str) -> None:
+ self._ctor_param_attrs[(class_qn, param)] = attr
+
+ def record_pending_field_binding(
+ self, class_qn: str, key: int | str, func_qn: str
+ ) -> None:
+ # key: keyword name, or positional index awaiting the ctor param order.
+ self._pending_field_bindings.append((class_qn, key, func_qn))
+
+ def finalize_field_bindings(self) -> None:
+ # Resolve pendings now that every class's ctor metadata is known. A
+ # subclass without its own __init__ inherits the base's params and
+ # field, so both a positional index and a keyword name resolve against
+ # the nearest self-or-ancestor that owns the ctor param, and the
+ # binding is recorded under THAT owner (where the field lives) so an
+ # inherited self.handler(), typed to the base, still matches.
+ for class_qn, key, func_qn in self._pending_field_bindings:
+ if isinstance(key, int):
+ owner_qn, params = self._ctor_params_owner(class_qn)
+ if key >= len(params):
+ continue
+ param = params[key]
+ else:
+ param = key
+ owner_qn = self._ctor_param_owner(class_qn, param)
+ field = self._ctor_param_attrs.get((owner_qn, param), param)
+ self.record_callable_field_binding(owner_qn, field, func_qn)
+ self._pending_field_bindings.clear()
+
+ def _ctor_params_owner(self, class_qn: str) -> tuple[str, tuple[str, ...]]:
+ # Nearest self-or-ancestor with a non-empty recorded ctor param list
+ # (a subclass with no __init__ has an empty list, so keep walking).
+ for ancestor in self._mro(class_qn):
+ if params := self._ctor_params.get(ancestor):
+ return ancestor, params
+ return class_qn, self._ctor_params.get(class_qn, ())
+
+ def _ctor_param_owner(self, class_qn: str, param: str) -> str:
+ # Nearest self-or-ancestor whose ctor declares `param`, so an
+ # inherited keyword binding attaches to the class that owns the field.
+ for ancestor in self._mro(class_qn):
+ if param in self._ctor_params.get(ancestor, ()):
+ return ancestor
+ return class_qn
+
+ def _mro(self, class_qn: str) -> list[str]:
+ # BFS over the inheritance graph, self first; guards cycles.
+ seen: set[str] = set()
+ order: list[str] = []
+ queue: deque[str] = deque([class_qn])
+ while queue:
+ cur = queue.popleft()
+ if cur in seen:
+ continue
+ seen.add(cur)
+ order.append(cur)
+ queue.extend(self.class_inheritance.get(cur, []))
+ return order
+
+ def record_callable_field_binding(
+ self, class_qn: str, field: str, func_qn: str
+ ) -> None:
+ # A NamedTuple/dataclass field holding a function reference: every
+ # function bound to it at any construction site is a possible callee
+ # when the field is invoked. Recording all of them is a sound call
+ # graph (each runs for its own configuration), so recall is complete.
+ self._field_bindings.setdefault((class_qn, field), set()).add(func_qn)
+ self._field_to_classes.setdefault(field, set()).add(class_qn)
+
+ def callable_field_targets(
+ self, field: str, recv_type: str | None = None
+ ) -> set[str]:
+ classes = self._field_to_classes.get(field)
+ if not classes:
+ return set()
+ if recv_type:
+ simple = recv_type.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ matched = [
+ qn
+ for qn in classes
+ if qn == recv_type or qn.rsplit(cs.SEPARATOR_DOT, 1)[-1] == simple
+ ]
+ if len(matched) == 1:
+ return self._field_bindings.get((matched[0], field), set())
+ # Receiver type unknown or ambiguous: only resolve when exactly one
+ # class declares this callable field, so the targets are unambiguous.
+ if len(classes) == 1:
+ return self._field_bindings.get((next(iter(classes)), field), set())
+ return set()
def _resolve_class_qn_from_type(
self, var_type: str, import_map: dict[str, str], module_qn: str
) -> str:
+ var_type = self._strip_optional(var_type)
+ if cs.SEPARATOR_DOUBLE_COLON in var_type:
+ return self._resolve_rust_class_qn(var_type)
if cs.SEPARATOR_DOT in var_type:
- return var_type
+ return self._follow_reexports(var_type)
if var_type in import_map:
- return import_map[var_type]
+ # A Rust import target is a raw `::`-path (`crate::parse::Parse`) that
+ # is not a registry qn; resolve it to the real type node so both
+ # local-type dispatch and the external-receiver guard see it as
+ # first-party (else its trie fallback is wrongly suppressed).
+ target = import_map[var_type]
+ if cs.SEPARATOR_DOUBLE_COLON in target:
+ return self._resolve_rust_class_qn(target)
+ return self._follow_reexports(target)
return self._resolve_class_name(var_type, module_qn) or ""
+ def _strip_optional(self, var_type: str) -> str:
+ # An Optional annotation (X | None) names a single concrete class; reduce it
+ # so attribute/operator resolution can find that class. Genuine multi-type
+ # unions stay unresolved (ambiguous).
+ if cs.PY_UNION_SEPARATOR not in var_type:
+ return var_type
+ non_none = [
+ member
+ for part in var_type.split(cs.PY_UNION_SEPARATOR)
+ if (member := part.strip()) and member != cs.PY_NONE
+ ]
+ return non_none[0] if len(non_none) == 1 else var_type
+
+ def _follow_reexports(self, class_qn: str) -> str:
+ return follow_reexports(
+ class_qn, self.import_processor.import_mapping, self.function_registry
+ )
+
def _try_resolve_method(
self, class_qn: str, method_name: str, separator: str = cs.SEPARATOR_DOT
) -> tuple[str, str] | None:
@@ -49,7 +258,426 @@ def resolve_function_call(
module_qn: str,
local_var_types: dict[str, str] | None = None,
class_context: str | None = None,
+ caller_qn: str | None = None,
+ language: cs.SupportedLanguage | None = None,
+ ) -> tuple[str, str] | None:
+ return self._redirect_protocol_method(
+ self._resolve_function_call(
+ call_name,
+ module_qn,
+ local_var_types,
+ class_context,
+ caller_qn,
+ language,
+ )
+ )
+
+ def _resolve_js_prototype_sibling(
+ self,
+ call_name: str,
+ caller_qn: str | None,
+ language: cs.SupportedLanguage | None,
+ ) -> tuple[str, str] | None:
+ # Only a two-part `this.m` call in a JS/TS caller qualifies; the caller's
+ # parent scope (module.Date for Date.prototype.strftime) is where the
+ # prototype siblings were registered. A module-level caller degrades to
+ # the same module-method lookup the CommonJS fallback performs.
+ # A dotless caller_qn has no parent scope: rsplit would return the
+ # caller itself and the lookup would land on the caller's own NESTED
+ # function, which `this.m` never names.
+ if language not in cs.JS_TS_LANGUAGES or not caller_qn:
+ return None
+ if cs.SEPARATOR_DOT not in caller_qn:
+ return None
+ if not call_name.startswith(cs.JS_THIS_CALL_PREFIX):
+ return None
+ method_name = call_name[len(cs.JS_THIS_CALL_PREFIX) :]
+ if not method_name or cs.SEPARATOR_DOT in method_name:
+ return None
+ parent_scope = caller_qn.rsplit(cs.SEPARATOR_DOT, 1)[0]
+ method_qn = f"{parent_scope}{cs.SEPARATOR_DOT}{method_name}"
+ if func_type := self.function_registry.get(method_qn):
+ return func_type, method_qn
+ return None
+
+ def _resolve_enclosing_scope(
+ self, call_name: str, caller_qn: str | None, module_qn: str
+ ) -> tuple[str, str] | None:
+ # Python LEGB: a bare name defined in the caller's own body or an enclosing
+ # FUNCTION scope (a nested def) shadows module-level and same-named nested
+ # defs in sibling scopes. The module-keyed trie fallback cannot tell two
+ # sibling `traverse` defs apart, so resolve against the caller's scope chain
+ # first. Walk up only through function/method scopes (each ancestor must be
+ # in the registry); a class or the module boundary stops the walk, because
+ # class scope is NOT part of a method's name-lookup chain in Python.
+ if not caller_qn or cs.SEPARATOR_DOT in call_name:
+ return None
+ scope = caller_qn
+ while True:
+ candidate = f"{scope}{cs.SEPARATOR_DOT}{call_name}"
+ if candidate in self.function_registry:
+ return self.function_registry[candidate], candidate
+ # A duplicate-variant caller (click's real `command` registers as
+ # `command@168` behind its @t.overload stubs) owns nested defs the
+ # def pass registers under the NATURAL qn (`command.decorator`);
+ # probe the variant-stripped scope too, or the call falls to the
+ # module trie and mis-binds to a sibling's same-named nested.
+ last = scope.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ if cs.DUP_QN_MARKER in last:
+ natural_scope = (
+ scope[: len(scope) - len(last)] + last.split(cs.DUP_QN_MARKER, 1)[0]
+ )
+ natural_candidate = f"{natural_scope}{cs.SEPARATOR_DOT}{call_name}"
+ if natural_candidate in self.function_registry:
+ return (
+ self.function_registry[natural_candidate],
+ natural_candidate,
+ )
+ if cs.SEPARATOR_DOT not in scope:
+ return None
+ parent = scope.rsplit(cs.SEPARATOR_DOT, 1)[0]
+ if parent == module_qn or parent not in self.function_registry:
+ return None
+ scope = parent
+
+ def _protocol_impl_map(self) -> dict[str, str]:
+ # A Protocol stub never runs; the concrete implementer does. Map each
+ # XxxProtocol to a unique non-Protocol class named Xxx (the suffix
+ # convention disambiguates the real impl from test mocks or other
+ # structural conformers, which structural matching alone cannot).
+ if self._protocol_impl_cache is not None:
+ return self._protocol_impl_cache
+ sep = cs.SEPARATOR_DOT
+ protocols: set[str] = set()
+ classes_by_simple: dict[str, list[str]] = defaultdict(list)
+ for qn, bases in self.class_inheritance.items():
+ classes_by_simple[qn.rsplit(sep, 1)[-1]].append(qn)
+ if any(base.rsplit(sep, 1)[-1] == cs.PY_PROTOCOL for base in bases):
+ protocols.add(qn)
+ impl: dict[str, str] = {}
+ for protocol_qn in protocols:
+ simple = protocol_qn.rsplit(sep, 1)[-1]
+ if simple == cs.PY_PROTOCOL or not simple.endswith(cs.PY_PROTOCOL):
+ continue
+ base_name = simple[: -len(cs.PY_PROTOCOL)]
+ candidates = [
+ qn for qn in classes_by_simple.get(base_name, []) if qn not in protocols
+ ]
+ if len(candidates) == 1:
+ impl[protocol_qn] = candidates[0]
+ self._protocol_impl_cache = impl
+ return impl
+
+ def _protocol_classes(self) -> set[str]:
+ if self._protocol_classes_cache is None:
+ sep = cs.SEPARATOR_DOT
+ self._protocol_classes_cache = {
+ qn
+ for qn, bases in self.class_inheritance.items()
+ if any(base.rsplit(sep, 1)[-1] == cs.PY_PROTOCOL for base in bases)
+ }
+ return self._protocol_classes_cache
+
+ def protocol_dispatch_targets(self, callee_qn: str) -> set[tuple[str, str]]:
+ # A call resolved to a Protocol stub method (P.M) never runs the stub: the
+ # runtime receiver is some conformer, so the sound call graph emits an edge
+ # to M on every non-Protocol class that defines it. Gating on the resolved
+ # target being a Protocol method keeps this from firing on ordinary calls.
+ class_qn, sep, method_name = callee_qn.rpartition(cs.SEPARATOR_DOT)
+ if not sep or class_qn not in self._protocol_classes():
+ return set()
+ protocols = self._protocol_classes()
+ targets: set[tuple[str, str]] = set()
+ for qn in self.function_registry.find_ending_with(method_name):
+ definer, dot, name = qn.rpartition(cs.SEPARATOR_DOT)
+ if dot and name == method_name and definer not in protocols:
+ targets.add((self.function_registry[qn], qn))
+ return targets
+
+ def self_dispatch_targets(
+ self, class_context: str, method_name: str
+ ) -> set[tuple[str, str]]:
+ # self.M()/cls.M() statically targets the enclosing class's own or inherited
+ # M, and dynamically dispatches to any concrete subclass override of M. Anchor
+ # on the ENCLOSING class (not the resolved callee, which the trie may pick as
+ # an arbitrary sibling override when M is abstract with several overrides) and
+ # emit an edge to the enclosing-class method AND every concrete override, so an
+ # override or an abstract base reached only through a self-call is not reported
+ # dead.
+ if not class_context or not method_name:
+ return set()
+ targets: set[tuple[str, str]] = set()
+ # Skip abstract targets: an @abstractmethod stub never runs (the concrete
+ # override does), so it must not be a call target; a concrete sibling/impl
+ # wins. Abstract methods only "reached" polymorphically are handled as
+ # dead-code roots, not by a spurious CALLS edge.
+ if (base := self._try_resolve_method(class_context, method_name)) and (
+ not self.function_registry.is_abstract(base[1])
+ ):
+ targets.add(base)
+ for subclass_qn in self._concrete_subclasses(class_context):
+ override_qn = f"{subclass_qn}{cs.SEPARATOR_DOT}{method_name}"
+ if override_qn in self.function_registry and not (
+ self.function_registry.is_abstract(override_qn)
+ ):
+ targets.add((self.function_registry[override_qn], override_qn))
+ return targets
+
+ def js_member_twin_targets(self, callee_qn: str) -> set[tuple[str, str]]:
+ # `View.prototype.lookup = function lookup(...)` registers TWO nodes for
+ # one method: the prototype path's `View.lookup` and the fn-expr's
+ # own-name module-flat `view.lookup`. A call binds one twin and the
+ # other reports dead. Return the same-name twin(s) whose parent chain
+ # extends (or is extended by) the callee's parent, i.e. the same
+ # module's flat/member pair, so the caller can edge both (the
+ # duplicate-QN keep-both design). Never crosses modules.
+ parent_qn, sep, leaf = callee_qn.rpartition(cs.SEPARATOR_DOT)
+ if not sep:
+ return set()
+ twins: set[tuple[str, str]] = set()
+ for qn in self.function_registry.find_ending_with(leaf):
+ if qn == callee_qn:
+ continue
+ other_parent, d, other_leaf = qn.rpartition(cs.SEPARATOR_DOT)
+ if not d or other_leaf != leaf:
+ continue
+ if not (
+ other_parent.startswith(f"{parent_qn}{cs.SEPARATOR_DOT}")
+ or parent_qn.startswith(f"{other_parent}{cs.SEPARATOR_DOT}")
+ ):
+ continue
+ label = self.function_registry.get(qn)
+ if label in (cs.NodeLabel.FUNCTION, cs.NodeLabel.METHOD):
+ twins.add((label, qn))
+ return twins
+
+ def go_package_sibling_targets(self, callee_qn: str) -> set[tuple[str, str]]:
+ # Go package-level functions are package-scoped, but cgr keys each file as
+ # its own module (`pkgdir.file.name`). Two same-package functions with the
+ # same name can ONLY be mutually-exclusive build-tag variants (gin's
+ # `validate` under `//go:build !nomsgpack` vs `nomsgpack`); the compiler
+ # rejects duplicate top-level identifiers in a package otherwise. A bare call
+ # resolves to just one file's copy, orphaning the other build's copy; return
+ # every same-package same-name package-level sibling so no build variant is
+ # reported dead. Revive-only and precise: a same-name function in a DIFFERENT
+ # package (different directory) is a distinct function, never a variant, so
+ # the package-dir equality guard excludes it.
+ file_module_qn, sep, name = callee_qn.rpartition(cs.SEPARATOR_DOT)
+ if not sep:
+ return set()
+ pkg_dir, dsep, _file = file_module_qn.rpartition(cs.SEPARATOR_DOT)
+ if not dsep:
+ return set()
+ targets: set[tuple[str, str]] = set()
+ for qn in self.function_registry.find_ending_with(name):
+ label = self.function_registry.get(qn)
+ if qn == callee_qn or label != cs.NodeLabel.FUNCTION:
+ continue
+ other_module, d, other_name = qn.rpartition(cs.SEPARATOR_DOT)
+ if not d or other_name != name or other_module == file_module_qn:
+ continue
+ other_pkg, d2, other_file = other_module.rpartition(cs.SEPARATOR_DOT)
+ # Same directory is necessary but not sufficient for same-package: Go
+ # permits an external test package (`package p_test`) in a `_test.go` file
+ # sharing the directory. Production code can never call a function defined
+ # in a `_test.go` file, so exclude such siblings; else a genuinely
+ # test-only dead function would be masked as live.
+ if (
+ d2
+ and other_pkg == pkg_dir
+ and not other_file.endswith(cs.GO_TEST_FILE_SUFFIX)
+ ):
+ targets.add((label, qn))
+ return targets
+
+ def java_constructor_targets(self, class_qn: str) -> set[tuple[str, str]]:
+ # A Java constructor is registered as a method directly under its class whose
+ # simple name equals the class's simple name (`Foo.Foo(int)`). `new Foo(...)`
+ # resolves to the CLASS, so redirect a CALLS edge to each declared constructor
+ # (all overloads); argument-type overload selection is not attempted, which is
+ # unnecessary for reachability and never fabricates a call to a
+ # non-constructor. Only constructors DIRECTLY on the class match (a nested
+ # class's constructor has an extra qn segment and is excluded).
+ simple = class_qn.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ targets: set[tuple[str, str]] = set()
+ for qn, node_type in self.function_registry.find_with_prefix(class_qn):
+ head = qn.split(cs.CHAR_PAREN_OPEN, 1)[0]
+ parent, dot, mname = head.rpartition(cs.SEPARATOR_DOT)
+ if dot and parent == class_qn and mname == simple:
+ targets.add((node_type, qn))
+ return targets
+
+ def cpp_destructor_targets(self, class_qn: str) -> set[tuple[str, str]]:
+ # Constructing a C++ object guarantees `~X` runs at end of
+ # lifetime, but no call node ever names it, so construction sites
+ # redirect a CALLS edge to the class's destructor exactly as they
+ # do to its constructors (same direct-member gate as
+ # java_constructor_targets: a nested class's dtor has an extra qn
+ # segment and is excluded).
+ # Destroying an object runs its own dtor AND every ancestor dtor
+ # unconditionally (issue #892), so the whole INHERITS closure's
+ # declared dtors are targets. The full chain matters even when an
+ # intermediate dtor is declared: its definition may live outside
+ # the parsed source, in which case no caller pass ever emits its
+ # own base-dtor edge (Greptile, PR #894). A destructor cannot be
+ # overloaded, so each qn is one direct registry lookup (PR #799).
+ targets: set[tuple[str, str]] = set()
+ seen: set[str] = set()
+ stack = [class_qn]
+ while stack:
+ current = stack.pop()
+ if current in seen:
+ continue
+ seen.add(current)
+ simple = current.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ dtor_qn = f"{current}{cs.SEPARATOR_DOT}{cs.CPP_DESTRUCTOR_PREFIX}{simple}"
+ dtor_type = self.function_registry.get(dtor_qn)
+ if dtor_type is not None:
+ targets.add((dtor_type, dtor_qn))
+ stack.extend(self.class_inheritance.get(current, ()))
+ return targets
+
+ def cpp_braced_return_class(self, caller_qn: str, module_qn: str) -> str | None:
+ # The class constructed by a `return {...};` inside caller_qn: the
+ # caller's recorded return type, resolved to a registered class
+ # (None for primitives/auto/unrecorded).
+ return_type = self.type_inference.method_return_types.get(caller_qn)
+ if not return_type:
+ return None
+ return self._resolve_type_to_class_qn(return_type, module_qn)
+
+ def cpp_dispatch_targets(
+ self,
+ call_name: str,
+ local_var_types: dict[str, str] | None,
+ template_params: frozenset[str],
+ ) -> set[tuple[str, str]]:
+ # A C++ call through a template-parameter receiver (`sax->start_object()`
+ # inside a `template` fn) has no concrete receiver type, so
+ # precise resolution fails and the trie binds one arbitrary same-named method
+ # (or the external-type guard drops the edge), leaving every OTHER structural
+ # interface implementer reported dead (nlohmann/json's json_sax_* visitors).
+ # When the receiver does NOT resolve to a first-party class, fan the call out
+ # to the method on EVERY class that defines it. A receiver typed to a concrete
+ # first-party class dispatches precisely and is skipped, so this only adds
+ # edges the precise path could not place. Full-fallback by design: a
+ # template-parameter or unresolved receiver is indistinguishable from an
+ # external one by name, so std::-typed receivers also fan out.
+ parts = call_name.split(cs.SEPARATOR_DOT)
+ if len(parts) != 2:
+ return set()
+ object_name, method_name = parts
+ # Fan out only when the receiver is typed to a template PARAMETER (`SAX* sax`
+ # in a `template` fn), whose concrete type is the argument at
+ # each instantiation, unknowable statically, so every implementer is a
+ # possible target. A concrete type is left to the precise path (a first-party
+ # class dispatches exactly; an external `std::string` receiver must NOT be
+ # rebound to an unrelated first-party method). An untyped receiver is left to
+ # the single-best trie fallback; fanning every untyped call out to all
+ # same-named methods would flood the graph with false edges.
+ if (local_var_types or {}).get(object_name) not in template_params:
+ return set()
+ targets: set[tuple[str, str]] = set()
+ for qn in self.function_registry.find_ending_with(method_name):
+ definer, dot, name = qn.rpartition(cs.SEPARATOR_DOT)
+ if (
+ dot
+ and name == method_name
+ and self.function_registry[qn] == cs.NodeLabel.METHOD
+ ):
+ targets.add((self.function_registry[qn], qn))
+ return targets
+
+ def _interface_impl_map(self) -> dict[str, str]:
+ # Map an interface to its SOLE first-party implementer. A call typed to an
+ # interface resolves to the interface's own method declaration (the static
+ # callee); when the interface has exactly one implementer the concrete
+ # method is the one that runs, so ALSO edge it (call-graph accuracy).
+ # >1 implementer is ambiguous -> not mapped -> the call stays on the
+ # interface method alone (no precision risk, recall preserved).
+ if self._interface_impl_cache is None:
+ self._interface_impl_cache = {
+ interface_qn: next(iter(implementers))
+ for interface_qn, implementers in self.interface_implementers.items()
+ if len(implementers) == 1
+ }
+ return self._interface_impl_cache
+
+ def interface_sole_impl_targets(self, callee_qn: str) -> set[tuple[str, str]]:
+ # A callee that IS an interface/trait method (the receiver was typed to
+ # the interface -- a concrete receiver dispatches to the impl directly)
+ # with exactly one implementer also runs the concrete method, so return
+ # it for an additional CALLS edge. REPLACING the interface edge instead
+ # (the pre-#665-era redirect) orphaned the interface stub: OVERRIDES
+ # expansion only walks interface -> impl, so the stub's declaration
+ # (gson's FieldNamingStrategy.translateName) reported dead.
+ class_qn, sep, method_name = callee_qn.rpartition(cs.SEPARATOR_DOT)
+ if not sep:
+ return set()
+ impl_qn = self._interface_impl_map().get(class_qn)
+ if impl_qn is None:
+ return set()
+ if result := self._try_resolve_method(impl_qn, method_name):
+ return {result}
+ return set()
+
+ def _redirect_protocol_method(
+ self, result: tuple[str, str] | None
+ ) -> tuple[str, str] | None:
+ # Only Python Protocol stubs REPLACE the resolved target: a Protocol
+ # method body never runs (it is `...`), so the concrete method is the
+ # sole real callee. An interface/trait method is a live declaration the
+ # call depends on, so its sole-impl companion edge is ADDITIVE
+ # (interface_sole_impl_targets), never a replacement.
+ if result is None:
+ return result
+ class_qn, sep, method_name = result[1].rpartition(cs.SEPARATOR_DOT)
+ if not sep:
+ return result
+ impl_qn = self._protocol_impl_map().get(class_qn)
+ if impl_qn is None:
+ return result
+ redirected = f"{impl_qn}{cs.SEPARATOR_DOT}{method_name}"
+ if redirected in self.function_registry:
+ return self.function_registry[redirected], redirected
+ return result
+
+ def _resolve_function_call(
+ self,
+ call_name: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None = None,
+ class_context: str | None = None,
+ caller_qn: str | None = None,
+ language: cs.SupportedLanguage | None = None,
) -> tuple[str, str] | None:
+ # Enclosing-scope (nested def) lookup is caller-specific, so it must run
+ # before the module-keyed cache/trie, which would otherwise return a sibling
+ # scope's same-named nested function.
+ if result := self._resolve_enclosing_scope(call_name, caller_qn, module_qn):
+ return result
+
+ # `this.m()` inside a prototype-assigned function dispatches to a sibling
+ # method of the same prototype target (Date.prototype.strftime calling
+ # this.getTwoDigitMonth()); the caller's parent scope names that target.
+ # Caller-specific, so it too must run before the module-keyed cache; the
+ # CommonJS module-receiver fallback still applies when no sibling exists.
+ if result := self._resolve_js_prototype_sibling(call_name, caller_qn, language):
+ return result
+
+ # The cache is keyed by (call_name, module_qn) only, so a caller whose
+ # class carries extends-clause type arguments must bypass it: its
+ # member resolution is class-context-dependent (#875) and a sibling
+ # class's cached answer for the same name would be wrong.
+ use_cache = not local_var_types and not (
+ language == cs.SupportedLanguage.DART
+ and class_context in self.type_inference.dart_extends_type_args
+ )
+ if use_cache:
+ cache_key = (call_name, module_qn)
+ if cache_key in self._simple_resolution_cache:
+ return self._simple_resolution_cache[cache_key]
+
if result := self._try_resolve_iife(call_name, module_qn):
return result
@@ -57,17 +685,308 @@ def resolve_function_call(
return self._resolve_super_call(call_name, class_context)
if cs.SEPARATOR_DOT in call_name and self._is_method_chain(call_name):
- return self._resolve_chained_call(call_name, module_qn, local_var_types)
+ # A chained call resolves via return-type inference only; it does NOT
+ # fall through to the trie fallback, because a hop returning a container
+ # (`Kids() []Command`) or an unknown type must drop the edge rather than
+ # rebind the final method by bare name (a false `Command.Run`). C++ is the
+ # exception: a `foo().bar()` receiver with an unrecordable return type
+ # (`auto`/trailing/decltype, e.g. fmt's get_container(out).append) fell to
+ # the bare-method trie before chained typing existed, so preserve that
+ # fallback for C++ to avoid dropping edges the typing can't yet recover.
+ return self._resolve_chained_call(
+ call_name,
+ module_qn,
+ local_var_types,
+ class_context,
+ caller_qn,
+ language,
+ )
if result := self._try_resolve_via_imports(
- call_name, module_qn, local_var_types
+ call_name, module_qn, local_var_types, language
):
+ if use_cache:
+ self._simple_resolution_cache[cache_key] = result
return result
if result := self._try_resolve_same_module(call_name, module_qn):
+ if use_cache:
+ self._simple_resolution_cache[cache_key] = result
return result
- return self._try_resolve_via_trie(call_name, module_qn)
+ if class_context and (
+ result := self._resolve_self_sibling_method(call_name, class_context)
+ ):
+ return result
+
+ # A bare name explicitly imported from outside the project binds to that
+ # external symbol. Since precise import / same-module resolution above
+ # already failed, the symbol is unindexed; do NOT let the simple-name
+ # trie fallback rebind it to an unrelated first-party symbol of the same
+ # name. (The instantiation eval caught `from evals import GraphData;
+ # GraphData()` being resolved to codebase_rag's own GraphData class.)
+ if cs.SEPARATOR_DOT not in call_name and self._is_external_import(
+ call_name, module_qn
+ ):
+ if use_cache:
+ self._simple_resolution_cache[cache_key] = None
+ return None
+
+ # A dotted call on an import whose target still holds a raw
+ # slash-separated module path (github.com/x/y) is a call into an
+ # EXTERNAL module: local Go paths were rewritten to project qns at
+ # import time, so a surviving slash path is definitionally outside
+ # the repo. The symbol is unindexed; do not let the last-segment trie
+ # fallback rebind it to an unrelated first-party function.
+ if self._is_external_path_import(call_name, module_qn):
+ if use_cache:
+ self._simple_resolution_cache[cache_key] = None
+ return None
+
+ # A member call `obj.method` whose receiver has a KNOWN inferred type that is
+ # not a first-party class is a call on an external object (e.g. a
+ # `std::string`). Precise local-type resolution above already failed, so the
+ # method lives on the external type; do NOT let the simple-name trie fallback
+ # rebind it to an unrelated first-party method of the same name. Untyped
+ # receivers keep the fallback (their type is unknown, not known-external).
+ if self._receiver_type_is_external(call_name, module_qn, local_var_types):
+ if use_cache:
+ self._simple_resolution_cache[cache_key] = None
+ return None
+
+ # A JS/TS/Dart member call on an UNTYPED receiver (`view.render(...)`
+ # where `view` is a param constructed in the caller) targets a MEMBER:
+ # the bare-name trie would rebind it to an arbitrary same-named
+ # candidate (express's application.render; a Dart `h.greet()` picking
+ # the closest class's greet), a false edge that also hides the real
+ # method from dead-code. Bind the UNIQUE member-like candidate (parent
+ # qn itself registered) or drop. Typed Dart receivers never reach this:
+ # the local-type path above already bound them.
+ if (
+ language in cs.JS_TS_LANGUAGES or language == cs.SupportedLanguage.DART
+ ) and cs.SEPARATOR_DOT in call_name:
+ if language == cs.SupportedLanguage.DART and (
+ result := self._resolve_dart_external_base_arg_member(
+ call_name, class_context, local_var_types
+ )
+ ):
+ return result
+ result = self._resolve_js_member_call_unique(call_name, module_qn)
+ if use_cache:
+ self._simple_resolution_cache[cache_key] = result
+ return result
+
+ result = self._try_resolve_via_trie(call_name, module_qn)
+ if use_cache:
+ self._simple_resolution_cache[cache_key] = result
+ return result
+
+ def _resolve_dart_external_base_arg_member(
+ self,
+ call_name: str,
+ class_context: str | None,
+ local_var_types: dict[str, str] | None,
+ ) -> tuple[str, str] | None:
+ # `recv.member(...)` on a receiver undeclared in first-party scope,
+ # inside a class extending an EXTERNAL generic base: the only
+ # first-party types the base can hand back are its type arguments
+ # (`State.widget` is a GridBtn), so bind when exactly one
+ # argument's class defines the member (issue #875). A first-party
+ # base, a declared receiver, or a member named like the receiver on
+ # the class or ANY registered ancestor (a first-party mixin's getter
+ # is inherited, not handed back by the external base) all fall
+ # through to the unique-member gate instead.
+ if not class_context:
+ return None
+ type_args = self.type_inference.dart_extends_type_args.get(class_context)
+ if not type_args:
+ return None
+ receiver, sep, member = call_name.partition(cs.SEPARATOR_DOT)
+ if not sep or cs.SEPARATOR_DOT in member:
+ return None
+ if local_var_types and receiver in local_var_types:
+ return None
+ if self._dart_registered_member_named(class_context, receiver):
+ return None
+ bases = self.class_inheritance.get(class_context)
+ # resolve_deferred_inherits rewrote bases in place, so a registered
+ # bases[0] means a first-party extends base owns its members.
+ if not bases or self.function_registry.get(bases[0]) is not None:
+ return None
+ matches = [
+ qn
+ for arg_qn in dict.fromkeys(type_args)
+ if (qn := f"{arg_qn}{cs.SEPARATOR_DOT}{member}") in self.function_registry
+ ]
+ if len(matches) == 1:
+ return self.function_registry[matches[0]], matches[0]
+ return None
+
+ def _dart_registered_member_named(self, class_qn: str, name: str) -> bool:
+ # True when the class or any registered first-party ancestor
+ # (extends base or `with` mixin, both in class_inheritance) defines
+ # a member with this name. Dart `implements` contributes interface
+ # only, never implementation, so it cannot supply a receiver.
+ seen: set[str] = set()
+ stack = [class_qn]
+ while stack:
+ current = stack.pop()
+ if current in seen:
+ continue
+ seen.add(current)
+ if f"{current}{cs.SEPARATOR_DOT}{name}" in self.function_registry:
+ return True
+ stack.extend(self.class_inheritance.get(current, ()))
+ return False
+
+ def _resolve_js_member_call_unique(
+ self, call_name: str, module_qn: str
+ ) -> tuple[str, str] | None:
+ method_name = call_name.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ candidates: list[str] = []
+ for qn in self.function_registry.find_ending_with(method_name):
+ parent_qn, sep, leaf = qn.rpartition(cs.SEPARATOR_DOT)
+ if not sep or leaf != method_name:
+ continue
+ # Member-like: the parent is itself a registered node (a class, a
+ # prototype constructor Function, an object scope); a free
+ # function's parent is a module, which is never in the registry.
+ if parent_qn in self.function_registry:
+ candidates.append(qn)
+ # Only candidates VISIBLE to the calling module count: parent module
+ # imported here or defined here (express's tryRender imports ./view, so
+ # View.render qualifies; an unrelated example's GithubView.render does
+ # not). Required even for a SINGLETON: an untyped `client.render()` in
+ # a file with no relation to the sole View.render must drop, not grow a
+ # false cross-module edge that hides real dead code. A JS require maps
+ # the MODULE (`require('./view')` -> express.lib.view), so a visible
+ # candidate's parent may equal an import or sit anywhere under one.
+ import_map = self.import_processor.import_mapping.get(module_qn) or {}
+ imported = set(import_map.values())
+ visible = [
+ qn
+ for qn in candidates
+ if qn.startswith(f"{module_qn}{cs.SEPARATOR_DOT}")
+ or any(
+ qn.rpartition(cs.SEPARATOR_DOT)[0] == imp
+ or qn.rpartition(cs.SEPARATOR_DOT)[0].startswith(
+ f"{imp}{cs.SEPARATOR_DOT}"
+ )
+ for imp in imported
+ )
+ ]
+ if len(visible) == 1:
+ return self.function_registry[visible[0]], visible[0]
+ return None
+
+ def _is_external_path_import(self, call_name: str, module_qn: str) -> bool:
+ # True when the dotted call's object segment is imported from a target
+ # that is still a slash-separated module path; for Go, every local
+ # path was rewritten to a project qn at import time, so a surviving
+ # slash means external. JS/TS non-standard-scheme imports
+ # (ext:deno_node/y) alias first-party code and keep their trie
+ # fallback, mirroring _is_external_import.
+ if cs.SEPARATOR_DOT not in call_name:
+ return False
+ object_name = call_name.split(cs.SEPARATOR_DOT, 1)[0]
+ import_map = self.import_processor.import_mapping.get(module_qn)
+ if not import_map:
+ return False
+ target = import_map.get(object_name)
+ if not target or cs.SEPARATOR_SLASH not in target:
+ return False
+ bare_imports = self.import_processor.js_ts_bare_imports.get(module_qn)
+ return not (bare_imports and object_name in bare_imports)
+
+ def _is_external_import(self, call_name: str, module_qn: str) -> bool:
+ # True when call_name is imported in module_qn from a module outside the
+ # project. First-party imports are written either project-prefixed
+ # (`from proj.w import X`) or bare (`from utils.helpers import X`, where
+ # the registered node is `proj.utils.helpers.X`); both are first-party
+ # and left to the trie fallback. Only a target neither rooted at the
+ # project nor registered under the project prefix is external, so this
+ # suppresses cross-project fuzzy rebinds without dropping real
+ # first-party calls.
+ import_map = self.import_processor.import_mapping.get(module_qn)
+ if not import_map:
+ return False
+ target = import_map.get(call_name)
+ if not target:
+ return False
+ # A PHP `use function A\B\c` target is a namespace path, which never
+ # matches cgr's file-path qualified name (a global helper declares
+ # `namespace Illuminate\Support` from Collections/functions.php). Treating
+ # it as external would suppress the simple-name trie fallback that a bare
+ # PHP call already relies on, dropping the call; leave it to the trie.
+ # LIMITATION: cgr qualifies PHP functions by file path and does not track
+ # the `namespace` declaration, so a genuinely external
+ # `use function Vendor\pkg\helper` cannot be told apart from a
+ # path-mismatched first-party one; both defer to the trie, as a bare
+ # `helper()` call already does. Precise disambiguation would need
+ # systemic PHP namespace tracking.
+ php_imports = self.import_processor.php_function_imports.get(module_qn)
+ if php_imports and call_name in php_imports:
+ return False
+ # A JS/TS import with a non-standard scheme (deno `ext:deno_node/x`) does
+ # not resolve to a file-path module qn, so its target is unregistered and
+ # looks external even though it aliases first-party code. Defer to the
+ # simple-name trie (like a relative import that misses) instead of
+ # suppressing. Ordinary package specifiers (bare, scoped, node:/npm:) are
+ # NOT recorded here, so genuine external calls stay suppressed.
+ bare_imports = self.import_processor.js_ts_bare_imports.get(module_qn)
+ if bare_imports and call_name in bare_imports:
+ return False
+ # Only dotted absolute-path imports (Python/Java `pkg.mod.Name`) are
+ # judged here. Rust/C++ record relative or `::`-separated targets
+ # (`super::b::helper`) that never carry the project prefix and rely on
+ # the trie fallback to resolve, so they must not be mistaken external.
+ if cs.SEPARATOR_DOT not in target or cs.SEPARATOR_DOUBLE_COLON in target:
+ return False
+ project_root = module_qn.split(cs.SEPARATOR_DOT, 1)[0]
+ if target.split(cs.SEPARATOR_DOT, 1)[0] == project_root:
+ return False
+ return f"{project_root}{cs.SEPARATOR_DOT}{target}" not in self.function_registry
+
+ def _receiver_type_is_external(
+ self,
+ call_name: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ ) -> bool:
+ # True only for a two-part dotted member call `obj.method` whose `obj` has an
+ # inferred local type that is known to be external. The receiver type is
+ # external when it resolves to nothing, or to a qn neither registered nor
+ # rooted at the project (a `std::string` -> `std.string`). Then the method
+ # lives on the external type, so the simple-name trie fallback must not
+ # rebind it to a same-named first-party method. An untyped receiver (obj
+ # absent from the map) or a project-rooted type is left alone: its method may
+ # still be resolved by the fallback (e.g. a cross-file imported-class call the
+ # precise path missed), so only a provably external type is suppressed.
+ if not local_var_types or cs.SEPARATOR_DOT not in call_name:
+ return False
+ parts = call_name.split(cs.SEPARATOR_DOT)
+ if len(parts) != 2:
+ return False
+ var_type = local_var_types.get(parts[0])
+ if var_type is None:
+ return False
+ import_map = self.import_processor.import_mapping.get(module_qn, {})
+ class_qn = self._resolve_class_qn_from_type(var_type, import_map, module_qn)
+ if not class_qn:
+ return True
+ # First-party class qns may be written without the project prefix (a bare
+ # `from models.user import User` resolves to `models.user.User` while the
+ # registry stores `proj.models.user.User`), so check both the qn as-is and
+ # the project-prefixed form before judging a type external, mirroring
+ # _is_external_import. A project-rooted qn is always treated as first-party.
+ project_root = module_qn.split(cs.SEPARATOR_DOT, 1)[0]
+ if class_qn.split(cs.SEPARATOR_DOT, 1)[0] == project_root:
+ return False
+ return (
+ class_qn not in self.function_registry
+ and f"{project_root}{cs.SEPARATOR_DOT}{class_qn}"
+ not in self.function_registry
+ )
def _try_resolve_iife(
self, call_name: str, module_qn: str
@@ -96,17 +1015,25 @@ def _try_resolve_via_imports(
call_name: str,
module_qn: str,
local_var_types: dict[str, str] | None,
+ language: cs.SupportedLanguage | None = None,
) -> tuple[str, str] | None:
- if module_qn not in self.import_processor.import_mapping:
- return None
-
- import_map = self.import_processor.import_mapping[module_qn]
+ import_map = self.import_processor.import_mapping.get(module_qn)
+ if import_map is None:
+ # A module with no `use`/import statements can still resolve a member
+ # call `obj.method()` through an inferred local/self type (a
+ # self-contained Rust lib.rs is the common case). Only the
+ # import-dependent lookups below (direct, qualified-by-import,
+ # wildcard) are no-ops here, so proceed with an empty map when there
+ # is type info to drive resolution; otherwise nothing would match.
+ if not local_var_types:
+ return None
+ import_map = {}
if result := self._try_resolve_direct_import(call_name, import_map):
return result
if result := self._try_resolve_qualified_call(
- call_name, import_map, module_qn, local_var_types
+ call_name, import_map, module_qn, local_var_types, language
):
return result
@@ -119,9 +1046,7 @@ def _try_resolve_direct_import(
return None
imported_qn = import_map[call_name]
if imported_qn in self.function_registry:
- logger.debug(
- ls.CALL_DIRECT_IMPORT.format(call_name=call_name, qn=imported_qn)
- )
+ logger.debug(ls.CALL_DIRECT_IMPORT, call_name=call_name, qn=imported_qn)
return self.function_registry[imported_qn], imported_qn
return None
@@ -131,16 +1056,28 @@ def _try_resolve_qualified_call(
import_map: dict[str, str],
module_qn: str,
local_var_types: dict[str, str] | None,
+ language: cs.SupportedLanguage | None = None,
) -> tuple[str, str] | None:
- if not self._has_separator(call_name):
+ if cs.SEPARATOR_DOUBLE_COLON in call_name:
+ separator = cs.SEPARATOR_DOUBLE_COLON
+ elif cs.CHAR_COLON in call_name:
+ separator = cs.CHAR_COLON
+ elif cs.SEPARATOR_DOT in call_name:
+ separator = cs.SEPARATOR_DOT
+ else:
return None
- separator = self._get_separator(call_name)
parts = call_name.split(separator)
if len(parts) == 2:
if result := self._resolve_two_part_call(
- parts, call_name, separator, import_map, module_qn, local_var_types
+ parts,
+ call_name,
+ separator,
+ import_map,
+ module_qn,
+ local_var_types,
+ language,
):
return result
@@ -157,22 +1094,30 @@ def _has_separator(self, call_name: str) -> bool:
return (
cs.SEPARATOR_DOT in call_name
or cs.SEPARATOR_DOUBLE_COLON in call_name
- or cs.SEPARATOR_COLON in call_name
+ or cs.CHAR_COLON in call_name
)
def _get_separator(self, call_name: str) -> str:
if cs.SEPARATOR_DOUBLE_COLON in call_name:
return cs.SEPARATOR_DOUBLE_COLON
- if cs.SEPARATOR_COLON in call_name:
- return cs.SEPARATOR_COLON
+ if cs.CHAR_COLON in call_name:
+ return cs.CHAR_COLON
return cs.SEPARATOR_DOT
def _try_resolve_wildcard_imports(
self, call_name: str, import_map: dict[str, str]
) -> tuple[str, str] | None:
- for local_name, imported_qn in import_map.items():
- if not local_name.startswith("*"):
- continue
+ map_id = id(import_map)
+ if map_id not in self._wildcard_cache:
+ self._wildcard_cache[map_id] = (
+ [(k, v) for k, v in import_map.items() if k[0] == "*"]
+ if import_map
+ else []
+ )
+ wildcards = self._wildcard_cache[map_id]
+ if not wildcards:
+ return None
+ for _, imported_qn in wildcards:
if result := self._try_wildcard_qns(call_name, imported_qn):
return result
return None
@@ -187,9 +1132,7 @@ def _try_wildcard_qns(
for wildcard_qn in potential_qns:
if wildcard_qn in self.function_registry:
- logger.debug(
- ls.CALL_WILDCARD.format(call_name=call_name, qn=wildcard_qn)
- )
+ logger.debug(ls.CALL_WILDCARD, call_name=call_name, qn=wildcard_qn)
return self.function_registry[wildcard_qn], wildcard_qn
return None
@@ -199,7 +1142,7 @@ def _try_resolve_same_module(
same_module_func_qn = f"{module_qn}.{call_name}"
if same_module_func_qn in self.function_registry:
logger.debug(
- ls.CALL_SAME_MODULE.format(call_name=call_name, qn=same_module_func_qn)
+ ls.CALL_SAME_MODULE, call_name=call_name, qn=same_module_func_qn
)
return self.function_registry[same_module_func_qn], same_module_func_qn
return None
@@ -207,19 +1150,39 @@ def _try_resolve_same_module(
def _try_resolve_via_trie(
self, call_name: str, module_qn: str
) -> tuple[str, str] | None:
- search_name = re.split(r"[.:]|::", call_name)[-1]
+ search_name = _SEARCH_NAME_CACHE.get(call_name)
+ if search_name is None:
+ search_name = _SEPARATOR_PATTERN.split(call_name)[-1]
+ _SEARCH_NAME_CACHE[call_name] = search_name
possible_matches = self.function_registry.find_ending_with(search_name)
if not possible_matches:
- logger.debug(ls.CALL_UNRESOLVED.format(call_name=call_name))
+ logger.debug(ls.CALL_UNRESOLVED, call_name=call_name)
return None
- possible_matches.sort(
- key=lambda qn: self._calculate_import_distance(qn, module_qn)
- )
- best_candidate_qn = possible_matches[0]
- logger.debug(
- ls.CALL_TRIE_FALLBACK.format(call_name=call_name, qn=best_candidate_qn)
- )
+ if len(possible_matches) == 1:
+ best_candidate_qn = possible_matches[0]
+ else:
+ caller_parts = module_qn.split(cs.SEPARATOR_DOT)
+ caller_len = len(caller_parts)
+ caller_parent_prefix = (
+ cs.SEPARATOR_DOT.join(caller_parts[:-1]) + cs.SEPARATOR_DOT
+ if caller_len > 1
+ else ""
+ )
+ best_candidate_qn = min(
+ possible_matches,
+ key=lambda qn: (
+ # An @abstractmethod stub never runs when a concrete override
+ # exists, so prefer concrete candidates over abstract ones
+ # even when the abstract stub is closer by import distance.
+ self.function_registry.is_abstract(qn),
+ self._import_distance_fast(
+ qn, caller_parts, caller_len, caller_parent_prefix
+ ),
+ qn,
+ ),
+ )
+ logger.debug(ls.CALL_TRIE_FALLBACK, call_name=call_name, qn=best_candidate_qn)
return self.function_registry[best_candidate_qn], best_candidate_qn
def _resolve_two_part_call(
@@ -230,6 +1193,7 @@ def _resolve_two_part_call(
import_map: dict[str, str],
module_qn: str,
local_var_types: dict[str, str] | None,
+ language: cs.SupportedLanguage | None = None,
) -> tuple[str, str] | None:
object_name, method_name = parts
@@ -249,8 +1213,48 @@ def _resolve_two_part_call(
):
return result
+ # A same-module associated-function call `Type::assoc()` (`Ping::new()`):
+ # the object is a type defined in this module, not an import. Resolve it to
+ # its class node and look up the method there. Gated to `::` so a `.`-dotted
+ # receiver of unknown type still falls through to the trie fallback.
+ if separator == cs.SEPARATOR_DOUBLE_COLON and (
+ result := self._try_resolve_static_type_method(
+ object_name, method_name, call_name, module_qn
+ )
+ ):
+ return result
+
+ # A JS/TS dotted call binds to a same-module free function ONLY through
+ # a module-ish receiver (`exports.render()`, `this.render()` in the
+ # CommonJS/prototype pattern). An ordinary identifier receiver
+ # (`view.render()`) is an instance call: binding it to the free
+ # function is a false edge that also kills the real prototype method
+ # (express's View.render); let it fall to the unique-member gate.
+ if (
+ language in cs.JS_TS_LANGUAGES
+ and separator == cs.SEPARATOR_DOT
+ and object_name not in cs.JS_MODULE_RECEIVERS
+ ):
+ return None
return self._try_resolve_module_method(method_name, call_name, module_qn)
+ def _try_resolve_static_type_method(
+ self, object_name: str, method_name: str, call_name: str, module_qn: str
+ ) -> tuple[str, str] | None:
+ if not (class_qn := self._resolve_class_name(object_name, module_qn)):
+ return None
+ method_qn = f"{class_qn}{cs.SEPARATOR_DOT}{method_name}"
+ if method_qn in self.function_registry:
+ logger.debug(
+ ls.CALL_TYPE_INFERRED,
+ call_name=call_name,
+ method_qn=method_qn,
+ obj=object_name,
+ var_type=object_name,
+ )
+ return self.function_registry[method_qn], method_qn
+ return self._resolve_inherited_method(class_qn, method_name)
+
def _try_resolve_via_local_type(
self,
object_name: str,
@@ -293,23 +1297,21 @@ def _try_method_on_class(
method_qn = f"{class_qn}{separator}{method_name}"
if method_qn in self.function_registry:
logger.debug(
- ls.CALL_TYPE_INFERRED.format(
- call_name=call_name,
- method_qn=method_qn,
- obj=object_name,
- var_type=var_type,
- )
+ ls.CALL_TYPE_INFERRED,
+ call_name=call_name,
+ method_qn=method_qn,
+ obj=object_name,
+ var_type=var_type,
)
return self.function_registry[method_qn], method_qn
if inherited := self._resolve_inherited_method(class_qn, method_name):
logger.debug(
- ls.CALL_TYPE_INFERRED_INHERITED.format(
- call_name=call_name,
- method_qn=inherited[1],
- obj=object_name,
- var_type=var_type,
- )
+ ls.CALL_TYPE_INFERRED_INHERITED,
+ call_name=call_name,
+ method_qn=inherited[1],
+ obj=object_name,
+ var_type=var_type,
)
return inherited
return None
@@ -330,16 +1332,44 @@ def _try_resolve_via_import(
)
registry_separator = (
- separator if separator == cs.SEPARATOR_COLON else cs.SEPARATOR_DOT
+ separator if separator == cs.CHAR_COLON else cs.SEPARATOR_DOT
)
method_qn = f"{class_qn}{registry_separator}{method_name}"
if method_qn in self.function_registry:
logger.debug(
- ls.CALL_IMPORT_STATIC.format(call_name=call_name, method_qn=method_qn)
+ ls.CALL_IMPORT_STATIC, call_name=call_name, method_qn=method_qn
)
return self.function_registry[method_qn], method_qn
- return None
+ return self._try_resolve_package_member(class_qn, method_name)
+
+ def _try_resolve_package_member(
+ self, package_qn: str, member_name: str
+ ) -> tuple[str, str] | None:
+ # A Go package spans multiple files and cgr qualifies its members by
+ # FILE (pkg.file.Func), so an import-mapped package qn plus member name
+ # misses the registry by one segment. Search the package's file modules
+ # for the member; Go names are unique per package, so at most one
+ # function matches (min() keeps a same-named type-method collision
+ # deterministic). The module ROOT package maps to the bare project name
+ # (no dot), so accept it alongside project-dot-prefixed package qns.
+ project_name = self.import_processor.project_name
+ if package_qn != project_name and not package_qn.startswith(
+ f"{project_name}{cs.SEPARATOR_DOT}"
+ ):
+ return None
+ member_depth = package_qn.count(cs.SEPARATOR_DOT) + 2
+ candidates = [
+ qn
+ for qn, _ in self.function_registry.find_with_prefix(package_qn)
+ if qn.count(cs.SEPARATOR_DOT) == member_depth
+ and qn.rsplit(cs.SEPARATOR_DOT, 1)[-1] == member_name
+ ]
+ if not candidates:
+ return None
+ member_qn = min(candidates)
+ logger.debug(ls.CALL_PACKAGE_MEMBER, member=member_name, qn=member_qn)
+ return self.function_registry[member_qn], member_qn
def _resolve_imported_class_qn(
self,
@@ -361,12 +1391,16 @@ def _resolve_rust_class_qn(self, class_qn: str) -> str:
rust_parts = class_qn.split(cs.SEPARATOR_DOUBLE_COLON)
class_name = rust_parts[-1]
+ # A Rust receiver type may be a struct (Class), an enum (`Frame`), or a
+ # type alias, all of which carry impl methods; match any of them, else an
+ # enum-typed receiver fails to resolve and its type reads as external,
+ # wrongly suppressing the trie fallback.
matching_qns = self.function_registry.find_ending_with(class_name)
return next(
(
qn
for qn in matching_qns
- if self.function_registry.get(qn) == NodeType.CLASS
+ if self.function_registry.get(qn) in _RS_TYPE_NODE_TYPES
),
class_qn,
)
@@ -377,7 +1411,7 @@ def _try_resolve_module_method(
method_qn = f"{module_qn}.{method_name}"
if method_qn in self.function_registry:
logger.debug(
- ls.CALL_OBJECT_METHOD.format(call_name=call_name, method_qn=method_qn)
+ ls.CALL_OBJECT_METHOD, call_name=call_name, method_qn=method_qn
)
return self.function_registry[method_qn], method_qn
return None
@@ -401,12 +1435,11 @@ def _resolve_self_attribute_call(
method_qn = f"{class_qn}.{method_name}"
if method_qn in self.function_registry:
logger.debug(
- ls.CALL_INSTANCE_ATTR.format(
- call_name=call_name,
- method_qn=method_qn,
- attr_ref=attribute_ref,
- var_type=var_type,
- )
+ ls.CALL_INSTANCE_ATTR,
+ call_name=call_name,
+ method_qn=method_qn,
+ attr_ref=attribute_ref,
+ var_type=var_type,
)
return self.function_registry[method_qn], method_qn
@@ -414,12 +1447,11 @@ def _resolve_self_attribute_call(
class_qn, method_name
):
logger.debug(
- ls.CALL_INSTANCE_ATTR_INHERITED.format(
- call_name=call_name,
- method_qn=inherited_method[1],
- attr_ref=attribute_ref,
- var_type=var_type,
- )
+ ls.CALL_INSTANCE_ATTR_INHERITED,
+ call_name=call_name,
+ method_qn=inherited_method[1],
+ attr_ref=attribute_ref,
+ var_type=var_type,
)
return inherited_method
@@ -441,9 +1473,9 @@ def _resolve_multi_part_call(
method_qn = f"{class_qn}.{method_name}"
if method_qn in self.function_registry:
logger.debug(
- ls.CALL_IMPORT_QUALIFIED.format(
- call_name=call_name, method_qn=method_qn
- )
+ ls.CALL_IMPORT_QUALIFIED,
+ call_name=call_name,
+ method_qn=method_qn,
)
return self.function_registry[method_qn], method_qn
@@ -455,12 +1487,11 @@ def _resolve_multi_part_call(
method_qn = f"{class_qn}.{method_name}"
if method_qn in self.function_registry:
logger.debug(
- ls.CALL_INSTANCE_QUALIFIED.format(
- call_name=call_name,
- method_qn=method_qn,
- class_name=class_name,
- var_type=var_type,
- )
+ ls.CALL_INSTANCE_QUALIFIED,
+ call_name=call_name,
+ method_qn=method_qn,
+ class_name=class_name,
+ var_type=var_type,
)
return self.function_registry[method_qn], method_qn
@@ -468,16 +1499,122 @@ def _resolve_multi_part_call(
class_qn, method_name
):
logger.debug(
- ls.CALL_INSTANCE_INHERITED.format(
- call_name=call_name,
- method_qn=inherited_method[1],
- class_name=class_name,
- var_type=var_type,
- )
+ ls.CALL_INSTANCE_INHERITED,
+ call_name=call_name,
+ method_qn=inherited_method[1],
+ class_name=class_name,
+ var_type=var_type,
)
return inherited_method
- return None
+ return self._resolve_field_hop_method(
+ parts, call_name, import_map, module_qn, local_var_types
+ )
+
+ def _resolve_field_hop_method(
+ self,
+ parts: list[str],
+ call_name: str,
+ import_map: dict[str, str],
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ ) -> tuple[str, str] | None:
+ # A paren-free field-hop receiver called inline (gin's `c.writermem.reset`):
+ # `c` is a typed local (Context), each middle segment is a struct FIELD whose
+ # recorded type advances the receiver (writermem -> responseWriter), and the
+ # final segment is a method on the last field's type. Resolves ONLY when every
+ # middle segment is a known field and the method exists (never a name-only
+ # fallback), so it can revive a dropped edge but never mis-bind. Distinct
+ # from the stored-local field-hop (`root := e.field.get(); root.m()`) which is
+ # already typed via _enrich_go_call_locals; this is the no-local direct form.
+ if len(parts) < 3 or not local_var_types:
+ return None
+ current_type = local_var_types.get(parts[0])
+ if not current_type:
+ return None
+ class_qn = self._resolve_class_qn_from_type(current_type, import_map, module_qn)
+ for field in parts[1:-1]:
+ if not class_qn:
+ return None
+ field_type = self.type_inference.class_field_types.get(class_qn, {}).get(
+ field
+ )
+ if not field_type:
+ return None
+ class_qn = self._chain_class_qn(field_type, module_qn)
+ if not class_qn:
+ return None
+ method_name = parts[-1]
+ method_qn = f"{class_qn}.{method_name}"
+ if method_qn in self.function_registry:
+ logger.debug(
+ ls.CALL_INSTANCE_QUALIFIED,
+ call_name=call_name,
+ method_qn=method_qn,
+ class_name=parts[0],
+ var_type=current_type,
+ )
+ return self.function_registry[method_qn], method_qn
+ return self._resolve_inherited_method(class_qn, method_name)
+
+ def operator_dunder_targets(
+ self,
+ operand_text: str,
+ dunder: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ ) -> set[tuple[str, str]]:
+ # Operator syntax dispatches to a dunder on the operand's type. Resolve only
+ # when the operand type is known; never via the name-only trie fallback, so a
+ # builtin container does not borrow a first-party dunder. A Protocol-typed
+ # operand dispatches to the dunder on each structural implementer (which may
+ # define the dunder even when the Protocol stub does not, e.g. __len__).
+ if not local_var_types or not (var_type := local_var_types.get(operand_text)):
+ return set()
+ import_map = self.import_processor.import_mapping.get(module_qn, {})
+ class_qn = self._resolve_class_qn_from_type(var_type, import_map, module_qn)
+ if not class_qn:
+ return set()
+ if class_qn in self._protocol_classes():
+ # Naming convention (XxxProtocol -> Xxx) is robust when it applies;
+ # structural conformance covers protocols whose implementer is named
+ # differently. Union both so neither gap drops a concrete target.
+ classes = set(self._protocol_structural_implementers(class_qn))
+ if named_impl := self._protocol_impl_map().get(class_qn):
+ classes.add(named_impl)
+ else:
+ classes = {class_qn}
+ targets: set[tuple[str, str]] = set()
+ for candidate in classes:
+ if resolved := self._try_resolve_method(candidate, dunder):
+ targets.add(resolved)
+ return targets
+
+ def _protocol_structural_implementers(self, protocol_qn: str) -> set[str]:
+ # Classes that define every method declared on the Protocol (own or
+ # inherited). Used to dispatch operator dunders to the concrete type when the
+ # Protocol/implementer names don't follow the XxxProtocol convention.
+ if protocol_qn in self._struct_impl_cache:
+ return self._struct_impl_cache[protocol_qn]
+ sep = cs.SEPARATOR_DOT
+ protocol_methods = {
+ qn.rsplit(sep, 1)[-1]
+ for qn, node_type in self.function_registry.find_with_prefix(protocol_qn)
+ if node_type == NodeType.METHOD and qn.rsplit(sep, 1)[0] == protocol_qn
+ }
+ result: set[str] = set()
+ if protocol_methods:
+ protocols = self._protocol_classes()
+ for candidate in self.class_inheritance:
+ if candidate in protocols:
+ continue
+ if all(
+ self._try_resolve_method(candidate, method)
+ for method in protocol_methods
+ ):
+ result.add(candidate)
+ self._struct_impl_cache[protocol_qn] = result
+ return result
def resolve_builtin_call(self, call_name: str) -> tuple[str, str] | None:
if call_name in cs.JS_BUILTIN_PATTERNS:
@@ -499,15 +1636,62 @@ def resolve_builtin_call(self, call_name: str) -> tuple[str, str] | None:
return None
+ def cpp_operator_for_type(
+ self, call_name: str, operand_type_qn: str
+ ) -> tuple[str, str] | None:
+ # Operand-type-directed operator binding: the overload is either a
+ # member of the operand's own class or a free overload in that
+ # class's module (the beside-the-class convention). A typed operand
+ # with NEITHER is a builtin operation (enum/int comparison) with no
+ # first-party callee; nlohmann's `token == token_type::x` must
+ # not rebind to an unrelated class's operator== and fan out to all
+ # its overload variants.
+ # _try_resolve_method covers both the direct member and one
+ # INHERITED from a base (Derived : Base with Base::operator==).
+ if member := self._try_resolve_method(operand_type_qn, call_name):
+ return member
+ # ADL: a free overload may live in ANY enclosing namespace of the
+ # operand's type, not only its immediate parent scope.
+ parts = operand_type_qn.split(cs.SEPARATOR_DOT)
+ for depth in range(len(parts) - 1, 0, -1):
+ scope = cs.SEPARATOR_DOT.join(parts[:depth])
+ free_qn = f"{scope}{cs.SEPARATOR_DOT}{call_name}"
+ if free_qn in self.function_registry:
+ return (self.function_registry[free_qn], free_qn)
+ return None
+
+ def cpp_operand_class_qn(
+ self,
+ operand_name: str | None,
+ local_var_types: dict[str, str] | None,
+ module_qn: str,
+ ) -> str | None:
+ # A bare-identifier operand with a locally inferred type resolves
+ # to a REGISTERED first-party type qn, or nothing: only a known
+ # type may direct or suppress the operator binding; anything
+ # uninferable keeps the caller on the legacy best-candidate path.
+ if not operand_name or not local_var_types:
+ return None
+ var_type = local_var_types.get(operand_name)
+ if not var_type:
+ return None
+ import_map = self.import_processor.import_mapping.get(module_qn, {})
+ resolved = self._resolve_class_qn_from_type(var_type, import_map, module_qn)
+ if resolved and resolved in self.function_registry:
+ return resolved
+ return None
+
def resolve_cpp_operator_call(
self, call_name: str, module_qn: str
) -> tuple[str, str] | None:
if not call_name.startswith(cs.OPERATOR_PREFIX):
return None
- if call_name in cs.CPP_OPERATORS:
- return (cs.NodeLabel.FUNCTION, cs.CPP_OPERATORS[call_name])
-
+ # A user-defined overload always beats the builtin: the old table
+ # of synthetic `builtin.cpp.operator_*` qns shadowed real overloads
+ # and produced edges to nodes that never exist (dropped by the
+ # database). A primitive builtin operator is not a first-party
+ # callee, so with no registered overload there is no edge at all.
if possible_matches := self.function_registry.find_ending_with(call_name):
same_module_ops = [
qn
@@ -521,6 +1705,153 @@ def resolve_cpp_operator_call(
return None
+ def _infer_chained_object_type(
+ self,
+ object_expr: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None = None,
+ caller_qn: str | None = None,
+ language: cs.SupportedLanguage | None = None,
+ ) -> str | None:
+ # Type of a chained receiver expression like `c.Root()` using the shared
+ # method_return_types map: the base is a typed local (`c` -> Command), and
+ # each `.method()` hop advances the type by that method's return type
+ # (Root() -> Command). Language-agnostic; returns the bare type name of the
+ # final hop, or None if any hop is untyped/unknown (then the chain stays
+ # unresolved). No early-out on an empty method_return_types map: a
+ # constructor-temporary base (`Reader(...).m()`) types itself from the
+ # class registry alone.
+ parts = _split_receiver_chain(object_expr)
+ base = parts[0]
+ if not base:
+ return None
+ if cs.CHAR_PAREN_OPEN in base:
+ # A Rust chain rooted in an associated-function call
+ # (`Ping::new(msg).into_frame()`): type the base from the assoc fn's
+ # recorded return type. A bare-identifier factory call
+ # (`parser(ia, cb).parse()`, C++): type it from the factory's recorded
+ # return type. Other paren bases stay unresolved.
+ current_type = self._infer_rust_assoc_base_type(
+ base, module_qn
+ ) or self._infer_call_base_type(
+ base, module_qn, local_var_types, class_context, caller_qn, language
+ )
+ elif local_var_types:
+ current_type = local_var_types.get(base)
+ else:
+ return None
+ for part in parts[1:]:
+ if not current_type or cs.CHAR_PAREN_OPEN not in part:
+ return None
+ method = part.split(cs.CHAR_PAREN_OPEN, 1)[0]
+ class_qn = self._chain_class_qn(current_type, module_qn)
+ current_type = self.type_inference.method_return_types.get(
+ f"{class_qn}{cs.SEPARATOR_DOT}{method}"
+ )
+ return current_type
+
+ def _chain_class_qn(self, type_name: str, module_qn: str) -> str:
+ # Resolve a bare type name from a chained-call hop to its class qn, honoring
+ # imports (a Rust `use` target is a raw `::`-path, not a registry qn), so a
+ # method-return-type lookup keyed by the class qn hits.
+ import_map = self.import_processor.import_mapping.get(module_qn, {})
+ return (
+ self._resolve_class_qn_from_type(type_name, import_map, module_qn)
+ or type_name
+ )
+
+ def _infer_call_base_type(
+ self,
+ base: str,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ class_context: str | None,
+ caller_qn: str | None,
+ language: cs.SupportedLanguage | None = None,
+ ) -> str | None:
+ # `parser(ia, cb).parse()`: the receiver is a bare-identifier factory call.
+ # Resolve the callee to its function/method qn and return its recorded
+ # return type. When none resolves, a C++ callee may instead be a
+ # CONSTRUCTOR TEMPORARY (`Reader(...)`, nlohmann's
+ # from_cbor): the callee names the receiver's class itself. The callee is
+ # cut at `<` or `(`, whichever comes first, since template args can carry
+ # their own parens.
+ cut = len(base)
+ for bracket in (cs.CHAR_ANGLE_OPEN, cs.CHAR_PAREN_OPEN):
+ idx = base.find(bracket)
+ if idx != -1 and idx < cut:
+ cut = idx
+ callee = base[:cut]
+ if not callee:
+ return None
+ if cs.SEPARATOR_DOUBLE_COLON in callee:
+ # A `::`-qualified non-C++ callee is the Rust assoc path's job,
+ # handled by the caller; C++ namespace paths normalize to dots.
+ if language != cs.SupportedLanguage.CPP:
+ return None
+ callee = callee.replace(cs.SEPARATOR_DOUBLE_COLON, cs.SEPARATOR_DOT)
+ resolved = self.resolve_function_call(
+ callee, module_qn, local_var_types, class_context, caller_qn, language
+ )
+ if resolved is not None:
+ return_type = self.type_inference.method_return_types.get(resolved[1])
+ if return_type:
+ return self._resolve_type_to_class_qn(return_type, module_qn)
+ if language in (cs.SupportedLanguage.CPP, cs.SupportedLanguage.DART):
+ # Constructor temporary: `X(...)` resolved to a ctor (never a
+ # recorded return) or to nothing at all; if X names a registered
+ # class, that class IS the receiver type (C++ `Reader(...)`,
+ # Dart `_Usage(...).generate()`).
+ return self._resolve_type_to_class_qn(callee, module_qn)
+ return None
+
+ def _resolve_type_to_class_qn(self, type_path: str, module_qn: str) -> str | None:
+ # Resolve a recorded return-type path to a registered CLASS qn. A factory
+ # return type names a class, but the plain class-name resolver can return a
+ # same-named factory METHOD (nlohmann's basic_json has both a `parser` class
+ # and a `parser()` factory), so filter to class-labeled nodes. Try the
+ # import-aware resolver first (same-file bare types, imports), then a
+ # class-only suffix match on the qualified path, then on the bare name.
+ candidate = self._chain_class_qn(type_path, module_qn)
+ if candidate and self.function_registry.get(candidate) == cs.NodeLabel.CLASS:
+ return candidate
+ matches = [
+ qn
+ for qn in self.function_registry.find_ending_with(type_path)
+ if self.function_registry.get(qn) == cs.NodeLabel.CLASS
+ ]
+ if not matches and cs.SEPARATOR_DOT in type_path:
+ simple = type_path.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ matches = [
+ qn
+ for qn in self.function_registry.find_ending_with(simple)
+ if self.function_registry.get(qn) == cs.NodeLabel.CLASS
+ ]
+ if not matches:
+ return None
+ matches.sort(key=lambda qn: (len(qn), qn))
+ return matches[0]
+
+ def _infer_rust_assoc_base_type(self, base: str, module_qn: str) -> str | None:
+ # `Ping::new(msg)` -> the return type recorded for `Ping::new` (Ping).
+ # The callee is the text before the first paren; only a `::`-rooted
+ # associated call (`Type::assoc`) is handled.
+ callee = base.split(cs.CHAR_PAREN_OPEN, 1)[0]
+ if cs.SEPARATOR_DOUBLE_COLON not in callee:
+ return None
+ segments = callee.split(cs.SEPARATOR_DOUBLE_COLON)
+ if len(segments) < 2:
+ return None
+ # Keep the full path prefix (`crate::parse::Parse`) so a qualified
+ # associated call resolves; only the trailing segment is the method.
+ type_name = cs.SEPARATOR_DOUBLE_COLON.join(segments[:-1])
+ method = segments[-1]
+ class_qn = self._chain_class_qn(type_name, module_qn)
+ return self.type_inference.method_return_types.get(
+ f"{class_qn}{cs.SEPARATOR_DOT}{method}"
+ )
+
def _is_method_chain(self, call_name: str) -> bool:
if cs.CHAR_PAREN_OPEN not in call_name or cs.CHAR_PAREN_CLOSE not in call_name:
return False
@@ -535,8 +1866,11 @@ def _resolve_chained_call(
call_name: str,
module_qn: str,
local_var_types: dict[str, str] | None = None,
+ class_context: str | None = None,
+ caller_qn: str | None = None,
+ language: cs.SupportedLanguage | None = None,
) -> tuple[str, str] | None:
- match = re.search(r"\.([^.()]+)$", call_name)
+ match = _CHAINED_METHOD_PATTERN.search(call_name)
if not match:
return None
@@ -544,27 +1878,36 @@ def _resolve_chained_call(
object_expr = call_name[: match.start()]
- if (
- object_type
- := self.type_inference.python_type_inference._infer_expression_return_type(
+ object_type = (
+ self.type_inference.python_type_inference._infer_expression_return_type(
object_expr, module_qn, local_var_types
)
- ):
+ or self._infer_chained_object_type(
+ object_expr,
+ module_qn,
+ local_var_types,
+ class_context,
+ caller_qn,
+ language,
+ )
+ )
+ if object_type:
full_object_type = object_type
if cs.SEPARATOR_DOT not in object_type:
- if resolved_class := self._resolve_class_name(object_type, module_qn):
+ # Honor imports (Rust `use` targets are raw `::`-paths) so an
+ # imported chained type (`Get::new(k).into_frame()`) resolves.
+ if resolved_class := self._chain_class_qn(object_type, module_qn):
full_object_type = resolved_class
method_qn = f"{full_object_type}.{final_method}"
if method_qn in self.function_registry:
logger.debug(
- ls.CALL_CHAINED.format(
- call_name=call_name,
- method_qn=method_qn,
- obj_expr=object_expr,
- obj_type=object_type,
- )
+ ls.CALL_CHAINED,
+ call_name=call_name,
+ method_qn=method_qn,
+ obj_expr=object_expr,
+ obj_type=object_type,
)
return self.function_registry[method_qn], method_qn
@@ -572,15 +1915,35 @@ def _resolve_chained_call(
full_object_type, final_method
):
logger.debug(
- ls.CALL_CHAINED_INHERITED.format(
- call_name=call_name,
- method_qn=inherited_method[1],
- obj_expr=object_expr,
- obj_type=object_type,
- )
+ ls.CALL_CHAINED_INHERITED,
+ call_name=call_name,
+ method_qn=inherited_method[1],
+ obj_expr=object_expr,
+ obj_type=object_type,
)
return inherited_method
+ # C/C++ only, and ONLY when the receiver type was never inferred: its return
+ # type is unrecordable (`auto`/trailing/decltype, e.g. fmt's
+ # get_container(out).append). Before chained typing existed the bare method
+ # name resolved via the trie, so fall back to it here rather than dropping an
+ # edge that used to land. When the type WAS inferred but lacks the method, we
+ # must NOT rebind to an unrelated same-named method; drop instead. C shares
+ # the field_expression call shape but has no method dispatch, so it always
+ # lands here = its exact prior behaviour. Go/Rust deliberately drop.
+ if not object_type and language in (
+ cs.SupportedLanguage.CPP,
+ cs.SupportedLanguage.C,
+ ):
+ return self._resolve_function_call(
+ final_method,
+ module_qn,
+ local_var_types,
+ class_context,
+ caller_qn,
+ language,
+ )
+
return None
def _resolve_super_call(
@@ -596,45 +1959,118 @@ def _resolve_super_call(
current_class_qn = class_context
if not current_class_qn:
- logger.debug(ls.CALL_SUPER_NO_CONTEXT.format(call_name=call_name))
+ logger.debug(ls.CALL_SUPER_NO_CONTEXT, call_name=call_name)
return None
if current_class_qn not in self.class_inheritance:
- logger.debug(ls.CALL_SUPER_NO_INHERITANCE.format(class_qn=current_class_qn))
+ logger.debug(ls.CALL_SUPER_NO_INHERITANCE, class_qn=current_class_qn)
return None
parent_classes = self.class_inheritance[current_class_qn]
if not parent_classes:
- logger.debug(ls.CALL_SUPER_NO_PARENTS.format(class_qn=current_class_qn))
+ logger.debug(ls.CALL_SUPER_NO_PARENTS, class_qn=current_class_qn)
return None
if result := self._resolve_inherited_method(current_class_qn, method_name):
callee_type, parent_method_qn = result
logger.debug(
- ls.CALL_SUPER_RESOLVED.format(
- call_name=call_name, method_qn=parent_method_qn
- )
+ ls.CALL_SUPER_RESOLVED,
+ call_name=call_name,
+ method_qn=parent_method_qn,
)
return callee_type, parent_method_qn
logger.debug(
- ls.CALL_SUPER_UNRESOLVED.format(
- call_name=call_name, class_qn=current_class_qn
- )
+ ls.CALL_SUPER_UNRESOLVED,
+ call_name=call_name,
+ class_qn=current_class_qn,
)
return None
+ def _resolve_self_sibling_method(
+ self, call_name: str, class_context: str
+ ) -> tuple[str, str] | None:
+ # self.method() in a mixin may call a method defined on a SIBLING mixin
+ # (neither is the other's base); both are combined into a concrete class.
+ # Resolve through the concrete subclasses' MRO and accept the target only
+ # when it is unambiguous, so an unrelated same-named method cannot win.
+ parts = call_name.split(cs.SEPARATOR_DOT)
+ if len(parts) != 2 or parts[0] != cs.KEYWORD_SELF:
+ return None
+ method_name = parts[1]
+ candidates: set[str] = set()
+ for subclass_qn in self._concrete_subclasses(class_context):
+ candidates |= self._mro_method_qns(subclass_qn, method_name)
+ if not candidates:
+ return None
+ # An @abstractmethod stub never runs when a concrete sibling implements the
+ # method, so prefer concrete candidates; resolve only when unambiguous.
+ chosen = {
+ qn for qn in candidates if not self.function_registry.is_abstract(qn)
+ } or candidates
+ if len(chosen) != 1:
+ return None
+ method_qn = next(iter(chosen))
+ logger.debug(
+ ls.CALL_INSTANCE_ATTR_INHERITED,
+ call_name=call_name,
+ method_qn=method_qn,
+ attr_ref=cs.KEYWORD_SELF,
+ var_type=class_context,
+ )
+ return self.function_registry[method_qn], method_qn
+
+ def _mro_method_qns(self, class_qn: str, method_name: str) -> set[str]:
+ results: set[str] = set()
+ visited: set[str] = set()
+ queue: deque[str] = deque([class_qn])
+ while queue:
+ current = self._follow_reexports(queue.popleft())
+ if current in visited:
+ continue
+ visited.add(current)
+ method_qn = f"{current}.{method_name}"
+ if method_qn in self.function_registry:
+ results.add(method_qn)
+ queue.extend(self.class_inheritance.get(current, ()))
+ return results
+
+ def _subclass_map(self) -> dict[str, set[str]]:
+ if self._subclass_map_cache is None:
+ mapping: dict[str, set[str]] = defaultdict(set)
+ for subclass_qn, bases in self.class_inheritance.items():
+ for base in bases:
+ mapping[self._follow_reexports(base)].add(subclass_qn)
+ self._subclass_map_cache = mapping
+ return self._subclass_map_cache
+
+ def _concrete_subclasses(self, class_qn: str) -> set[str]:
+ subclass_map = self._subclass_map()
+ found: set[str] = set()
+ stack = list(subclass_map.get(class_qn, ()))
+ while stack:
+ current = stack.pop()
+ if current in found:
+ continue
+ found.add(current)
+ stack.extend(subclass_map.get(current, ()))
+ return found
+
def _resolve_inherited_method(
self, class_qn: str, method_name: str
) -> tuple[str, str] | None:
if class_qn not in self.class_inheritance:
return None
- queue = list(self.class_inheritance.get(class_qn, []))
- visited = set(queue)
+ bfs_queue = deque(self.class_inheritance.get(class_qn, []))
+ visited = set(bfs_queue)
- while queue:
- parent_class_qn = queue.pop(0)
+ while bfs_queue:
+ # Base classes are recorded by the name the subclass imported, which
+ # may be a package re-export (class_ingest.ClassIngestMixin) rather than
+ # the real definition (class_ingest.mixin.ClassIngestMixin); follow the
+ # re-export so the inherited method qn matches the registry.
+ parent_class_qn = self._follow_reexports(bfs_queue.popleft())
parent_method_qn = f"{parent_class_qn}.{method_name}"
if parent_method_qn in self.function_registry:
@@ -647,7 +2083,7 @@ def _resolve_inherited_method(
for grandparent_qn in self.class_inheritance[parent_class_qn]:
if grandparent_qn not in visited:
visited.add(grandparent_qn)
- queue.append(grandparent_qn)
+ bfs_queue.append(grandparent_qn)
return None
@@ -673,21 +2109,67 @@ def _calculate_import_distance(
return base_distance
+ def _import_distance_fast(
+ self,
+ candidate_qn: str,
+ caller_parts: list[str],
+ caller_len: int,
+ caller_parent_prefix: str,
+ ) -> int:
+ if candidate_qn in _QN_SPLIT_CACHE:
+ candidate_parts, candidate_len = _QN_SPLIT_CACHE[candidate_qn]
+ else:
+ candidate_parts = candidate_qn.split(cs.SEPARATOR_DOT)
+ candidate_len = len(candidate_parts)
+ _QN_SPLIT_CACHE[candidate_qn] = (candidate_parts, candidate_len)
+ common_prefix = 0
+ for i in range(min(caller_len, candidate_len)):
+ if caller_parts[i] == candidate_parts[i]:
+ common_prefix += 1
+ else:
+ break
+ base_distance = max(caller_len, candidate_len) - common_prefix
+ if caller_parent_prefix and candidate_qn.startswith(caller_parent_prefix):
+ base_distance -= 1
+ return base_distance
+
+ def _dealias_type(self, type_name: str) -> str:
+ # Follow C++ typedef/using aliases (`typedef Mutex MutexAlias;`) to the
+ # underlying class name so an alias'd receiver resolves like the class it
+ # names. Bounded against an alias cycle; a no-op when the name is not an
+ # alias (and always, for languages with no aliases collected).
+ seen: set[str] = set()
+ while type_name in self.type_aliases and type_name not in seen:
+ seen.add(type_name)
+ type_name = self.type_aliases[type_name]
+ return type_name
+
def _resolve_class_name(self, class_name: str, module_qn: str) -> str | None:
+ # Call resolution runs in Pass 3, after every definition pass, so a
+ # class qn missing from the registry can never be a real node;
+ # require registration so an import-map module entry (a C++ header
+ # stem shadowing its class name) cannot mask the real class.
return resolve_class_name(
- class_name, module_qn, self.import_processor, self.function_registry
+ self._dealias_type(class_name),
+ module_qn,
+ self.import_processor,
+ self.function_registry,
+ require_registered=True,
)
def resolve_java_method_call(
self,
call_node: Node,
module_qn: str,
- local_var_types: dict[str, str],
+ local_var_types: dict[str, str] | None,
+ caller_qn: str | None = None,
) -> tuple[str, str] | None:
java_engine = self.type_inference.java_type_inference
- result = java_engine.resolve_java_method_call(
- call_node, local_var_types, module_qn
+ result = self._redirect_protocol_method(
+ java_engine.resolve_java_method_call(
+ call_node, local_var_types, module_qn, caller_qn
+ )
)
if result:
@@ -697,7 +2179,18 @@ def resolve_java_method_call(
else cs.TEXT_UNKNOWN
)
logger.debug(
- ls.CALL_JAVA_RESOLVED.format(call_text=call_text, method_qn=result[1])
+ ls.CALL_JAVA_RESOLVED, call_text=call_text, method_qn=result[1]
)
return result
+
+ def resolve_csharp_method_call(
+ self,
+ call_node: Node,
+ module_qn: str,
+ local_var_types: dict[str, str] | None,
+ caller_qn: str | None = None,
+ ) -> tuple[str, str] | None:
+ return self.type_inference.csharp_type_inference.resolve_csharp_method_call(
+ call_node, local_var_types, module_qn, caller_qn
+ )
diff --git a/codebase_rag/parsers/class_ingest/cpp_modules.py b/codebase_rag/parsers/class_ingest/cpp_modules.py
index a5db9bc47..746e9d13a 100644
--- a/codebase_rag/parsers/class_ingest/cpp_modules.py
+++ b/codebase_rag/parsers/class_ingest/cpp_modules.py
@@ -8,6 +8,7 @@
from ... import constants as cs
from ... import logs
+from ...utils.path_utils import cached_relative_path, cached_resolve_posix
from ..utils import safe_decode_text, safe_decode_with_fallback
from .utils import decode_node_stripped
@@ -22,26 +23,40 @@ def ingest_cpp_module_declarations(
repo_path: Path,
project_name: str,
ingestor: IngestorProtocol,
+ module_interfaces: set[str],
+ deferred_impls: list[tuple[str, str]],
) -> None:
module_declarations = _find_module_declarations(root_node)
for _, decl_text in module_declarations:
if decl_text.startswith(cs.CPP_EXPORT_MODULE_PREFIX):
_process_export_module(
- decl_text, module_qn, file_path, repo_path, project_name, ingestor
+ decl_text,
+ module_qn,
+ file_path,
+ repo_path,
+ project_name,
+ ingestor,
+ module_interfaces,
)
elif decl_text.startswith(cs.CPP_MODULE_PREFIX) and not decl_text.startswith(
cs.CPP_MODULE_PRIVATE_PREFIX
):
_process_module_implementation(
- decl_text, module_qn, file_path, repo_path, project_name, ingestor
+ decl_text,
+ module_qn,
+ file_path,
+ repo_path,
+ project_name,
+ ingestor,
+ deferred_impls,
)
def _find_module_declarations(root_node: Node) -> list[tuple[Node, str]]:
module_declarations: list[tuple[Node, str]] = []
- def find_declarations(node: Node) -> None:
+ for node in root_node.children:
if node.type == cs.TS_MODULE_DECLARATION:
module_declarations.append((node, decode_node_stripped(node)))
elif node.type == cs.CppNodeType.DECLARATION:
@@ -56,10 +71,6 @@ def find_declarations(node: Node) -> None:
if has_module:
module_declarations.append((node, decode_node_stripped(node)))
- for child in node.children:
- find_declarations(child)
-
- find_declarations(root_node)
return module_declarations
@@ -70,6 +81,7 @@ def _process_export_module(
repo_path: Path,
project_name: str,
ingestor: IngestorProtocol,
+ module_interfaces: set[str],
) -> None:
parts = decl_text.split()
if len(parts) < 3:
@@ -77,13 +89,15 @@ def _process_export_module(
module_name = parts[2].rstrip(cs.CHAR_SEMICOLON)
interface_qn = f"{project_name}.{module_name}"
+ module_interfaces.add(interface_qn)
ingestor.ensure_node_batch(
cs.NodeLabel.MODULE_INTERFACE,
{
cs.KEY_QUALIFIED_NAME: interface_qn,
cs.KEY_NAME: module_name,
- cs.KEY_PATH: str(file_path.relative_to(repo_path)),
+ cs.KEY_PATH: cached_relative_path(file_path, repo_path).as_posix(),
+ cs.KEY_ABSOLUTE_PATH: cached_resolve_posix(file_path),
cs.KEY_MODULE_TYPE: cs.CPP_MODULE_TYPE_INTERFACE,
},
)
@@ -104,6 +118,7 @@ def _process_module_implementation(
repo_path: Path,
project_name: str,
ingestor: IngestorProtocol,
+ deferred_impls: list[tuple[str, str]],
) -> None:
parts = decl_text.split()
if len(parts) < 2:
@@ -117,7 +132,8 @@ def _process_module_implementation(
{
cs.KEY_QUALIFIED_NAME: impl_qn,
cs.KEY_NAME: f"{module_name}{cs.CPP_IMPL_SUFFIX}",
- cs.KEY_PATH: str(file_path.relative_to(repo_path)),
+ cs.KEY_PATH: cached_relative_path(file_path, repo_path).as_posix(),
+ cs.KEY_ABSOLUTE_PATH: cached_resolve_posix(file_path),
cs.KEY_IMPLEMENTS_MODULE: module_name,
cs.KEY_MODULE_TYPE: cs.CPP_MODULE_TYPE_IMPLEMENTATION,
},
@@ -129,39 +145,38 @@ def _process_module_implementation(
(cs.NodeLabel.MODULE_IMPLEMENTATION, cs.KEY_QUALIFIED_NAME, impl_qn),
)
+ # The exporting file may not be parsed yet (or may not exist at all);
+ # hold the IMPLEMENTS edge back until every ModuleInterface is known,
+ # so an implementation unit of an absent interface emits no phantom.
interface_qn = f"{project_name}.{module_name}"
- ingestor.ensure_relationship_batch(
- (cs.NodeLabel.MODULE_IMPLEMENTATION, cs.KEY_QUALIFIED_NAME, impl_qn),
- cs.RelationshipType.IMPLEMENTS,
- (cs.NodeLabel.MODULE_INTERFACE, cs.KEY_QUALIFIED_NAME, interface_qn),
- )
+ deferred_impls.append((impl_qn, interface_qn))
logger.info(logs.CLASS_CPP_MODULE_IMPL.format(qn=impl_qn))
def find_cpp_exported_classes(root_node: Node) -> list[Node]:
exported_class_nodes: list[Node] = []
+ stack = list(root_node.children)
- def traverse(node: Node) -> None:
+ while stack:
+ node = stack.pop()
if node.type == cs.CppNodeType.FUNCTION_DEFINITION:
node_text = decode_node_stripped(node)
if node_text.startswith(cs.CPP_EXPORT_PREFIXES):
+ found = False
for child in node.children:
if child.type == cs.TS_ERROR and child.text:
error_text = safe_decode_text(child)
if error_text in cs.CPP_EXPORTED_CLASS_KEYWORDS:
exported_class_nodes.append(node)
+ found = True
break
- else:
- if (
- cs.CPP_EXPORT_CLASS_PREFIX in node_text
- or cs.CPP_EXPORT_STRUCT_PREFIX in node_text
- ):
- exported_class_nodes.append(node)
-
- for child in node.children:
- traverse(child)
+ if not found and (
+ cs.CPP_EXPORT_CLASS_PREFIX in node_text
+ or cs.CPP_EXPORT_STRUCT_PREFIX in node_text
+ ):
+ exported_class_nodes.append(node)
+ stack.extend(node.children)
- traverse(root_node)
return exported_class_nodes
diff --git a/codebase_rag/parsers/class_ingest/identity.py b/codebase_rag/parsers/class_ingest/identity.py
index 85f670444..a19dd1df8 100644
--- a/codebase_rag/parsers/class_ingest/identity.py
+++ b/codebase_rag/parsers/class_ingest/identity.py
@@ -7,7 +7,7 @@
from ... import constants as cs
from ...language_spec import LANGUAGE_FQN_SPECS
-from ...utils.fqn_resolver import resolve_fqn_from_ast
+from .. import export_detection
from ..cpp import utils as cpp_utils
from ..rs import utils as rs_utils
from ..utils import safe_decode_text
@@ -22,22 +22,32 @@ def resolve_class_identity(
language: cs.SupportedLanguage,
lang_config: LanguageSpec,
file_path: Path | None,
- repo_path: Path,
- project_name: str,
) -> tuple[str, str, bool] | None:
if (fqn_config := LANGUAGE_FQN_SPECS.get(language)) and file_path:
- if class_qn := resolve_fqn_from_ast(
- class_node,
- file_path,
- repo_path,
- project_name,
- fqn_config,
- ):
- class_name = class_qn.split(cs.SEPARATOR_DOT)[-1]
- is_exported = language == cs.SupportedLanguage.CPP and (
- class_node.type == cs.CppNodeType.FUNCTION_DEFINITION
- or cpp_utils.is_exported(class_node)
- )
+ class_name = fqn_config.get_name(class_node)
+ if class_name:
+ parts = [class_name]
+ current = class_node.parent
+ while current:
+ if current.type in fqn_config.scope_node_types:
+ if scope_name := fqn_config.get_name(current):
+ parts.append(scope_name)
+ current = current.parent
+ parts.reverse()
+
+ # Use the module's already-resolved (and collision-disambiguated)
+ # qualified name as the prefix rather than recomputing from the path,
+ # so same-stem cross-language siblings get distinct class/method qns.
+ class_qn = module_qn + cs.SEPARATOR_DOT + cs.SEPARATOR_DOT.join(parts)
+ if language == cs.SupportedLanguage.CPP:
+ is_exported = (
+ class_node.type == cs.CppNodeType.FUNCTION_DEFINITION
+ or cpp_utils.is_exported(class_node)
+ )
+ else:
+ is_exported = export_detection.is_exported(
+ class_node, class_name, language
+ )
return class_qn, class_name, is_exported
return resolve_class_identity_fallback(class_node, module_qn, language, lang_config)
@@ -68,7 +78,8 @@ def resolve_class_identity_fallback(
nested_qn = build_nested_qualified_name_for_class(
class_node, module_qn, class_name, lang_config
)
- return nested_qn or f"{module_qn}.{class_name}", class_name, False
+ is_exported = export_detection.is_exported(class_node, class_name, language)
+ return nested_qn or f"{module_qn}.{class_name}", class_name, is_exported
def extract_cpp_class_name(class_node: Node) -> str | None:
diff --git a/codebase_rag/parsers/class_ingest/method_override.py b/codebase_rag/parsers/class_ingest/method_override.py
index 686ff26e6..77fe30872 100644
--- a/codebase_rag/parsers/class_ingest/method_override.py
+++ b/codebase_rag/parsers/class_ingest/method_override.py
@@ -1,6 +1,7 @@
from __future__ import annotations
from collections import deque
+from itertools import chain
from typing import TYPE_CHECKING
from loguru import logger
@@ -18,9 +19,13 @@ def process_all_method_overrides(
function_registry: FunctionRegistryTrieProtocol,
class_inheritance: dict[str, list[str]],
ingestor: IngestorProtocol,
+ interface_implementers: dict[str, set[str]] | None = None,
+ csharp_methods: set[str] | None = None,
+ csharp_override_methods: set[str] | None = None,
) -> None:
logger.info(logs.CLASS_PASS_4)
+ implemented_interfaces = _invert_implementers(interface_implementers or {})
for method_qn in function_registry.keys():
if (
function_registry[method_qn] == NodeType.METHOD
@@ -36,7 +41,180 @@ def process_all_method_overrides(
function_registry,
class_inheritance,
ingestor,
+ implemented_interfaces,
+ csharp_methods,
+ csharp_override_methods,
)
+ _process_mro_shadow_overrides(function_registry, class_inheritance, ingestor)
+
+
+def _process_mro_shadow_overrides(
+ function_registry: FunctionRegistryTrieProtocol,
+ class_inheritance: dict[str, list[str]],
+ ingestor: IngestorProtocol,
+) -> None:
+ # A mixin's method can shadow a same-name method from a SIBLING base
+ # branch only in a combining subclass's MRO: django's
+ # SearchVector(SearchVectorCombinable, Func) dispatches Combinable's
+ # `self._combine()` to SearchVectorCombinable._combine, yet the mixin
+ # never inherits Combinable, so the per-method ancestor walk above
+ # cannot see the relation. For every class, linearize its ancestry in
+ # reverse post-order (a C3-compatible MRO stand-in) and link each method
+ # name's FIRST provider (the runtime dispatch target) to every later
+ # provider; dead-code override expansion then revives the shadowing
+ # method when the shadowed one has live callers. Interfaces are not
+ # walked: default-method shadowing is rare and Java resolves it
+ # differently. ponytail: name-exact matching only, so a Java generic
+ # type-var rename across branches is not linked.
+ method_names_cache: dict[str, list[str]] = {}
+ ancestor_cache: dict[str, set[str]] = {}
+ emitted: set[tuple[str, str]] = set()
+ for class_qn in sorted(class_inheritance):
+ providers: dict[str, list[str]] = {}
+ for ancestor_qn in _linearized_ancestors(class_qn, class_inheritance):
+ if ancestor_qn not in method_names_cache:
+ method_names_cache[ancestor_qn] = _direct_method_names(
+ ancestor_qn, function_registry
+ )
+ for name in method_names_cache[ancestor_qn]:
+ providers.setdefault(name, []).append(ancestor_qn)
+ for name, classes in providers.items():
+ if len(classes) < 2:
+ continue
+ # Same-branch pairs (the provider inherits the shadowed class)
+ # are the per-method walk's territory and already linked; this
+ # pass adds only cross-branch sibling shadows.
+ if classes[0] not in ancestor_cache:
+ ancestor_cache[classes[0]] = set(
+ _linearized_ancestors(classes[0], class_inheritance)[1:]
+ )
+ first_qn = f"{classes[0]}{cs.SEPARATOR_DOT}{name}"
+ for shadowed_class in classes[1:]:
+ if shadowed_class in ancestor_cache[classes[0]]:
+ continue
+ pair = (first_qn, f"{shadowed_class}{cs.SEPARATOR_DOT}{name}")
+ if pair in emitted:
+ continue
+ emitted.add(pair)
+ ingestor.ensure_relationship_batch(
+ (cs.NodeLabel.METHOD, cs.KEY_QUALIFIED_NAME, pair[0]),
+ cs.RelationshipType.OVERRIDES,
+ (cs.NodeLabel.METHOD, cs.KEY_QUALIFIED_NAME, pair[1]),
+ )
+ logger.debug(
+ logs.CLASS_METHOD_OVERRIDE,
+ method_qn=pair[0],
+ parent_method_qn=pair[1],
+ )
+
+
+def _linearized_ancestors(
+ class_qn: str, class_inheritance: dict[str, list[str]]
+) -> list[str]:
+ # Reverse post-order over the ancestor DAG: a subclass always precedes
+ # its bases and a diamond's common ancestor sinks below BOTH branches
+ # (D(B, C) with B(A), C(A) linearizes [D, B, C, A], matching the C3
+ # MRO), so a shadowed common base can never outrank the sibling branch
+ # that shadows it. A plain depth-first preorder gets diamonds backwards
+ # and would emit reversed OVERRIDES edges. The `expanded` guard also
+ # keeps a malformed inheritance cycle from looping.
+ order: list[str] = []
+ expanded: set[str] = set()
+ stack: list[tuple[str, bool]] = [(class_qn, False)]
+ while stack:
+ current, processed = stack.pop()
+ if processed:
+ order.append(current)
+ continue
+ if current in expanded:
+ continue
+ expanded.add(current)
+ stack.append((current, True))
+ stack.extend((base, False) for base in class_inheritance.get(current, []))
+ return list(reversed(order))
+
+
+def _direct_method_names(
+ class_qn: str, function_registry: FunctionRegistryTrieProtocol
+) -> list[str]:
+ prefix = f"{class_qn}{cs.SEPARATOR_DOT}"
+ names: list[str] = []
+ for qn, node_type in function_registry.find_with_prefix(class_qn):
+ if node_type != NodeType.METHOD or not qn.startswith(prefix):
+ continue
+ leaf = qn[len(prefix) :]
+ if cs.SEPARATOR_DOT in leaf.split(cs.CHAR_PAREN_OPEN, 1)[0]:
+ continue
+ names.append(leaf)
+ return names
+
+
+def _invert_implementers(
+ interface_implementers: dict[str, set[str]],
+) -> dict[str, list[str]]:
+ # class_inheritance holds only superclasses (an `implements` clause or a
+ # Rust `impl Trait for Type` never enters it), so the override walk needs
+ # the implementer -> interfaces direction too, or no interface/trait
+ # implementation ever gets an OVERRIDES edge. Both loops sorted: the map
+ # and its sets are hash-ordered and edge emission must be deterministic
+ # (the inner sort makes the dict order deterministic too).
+ inverted: dict[str, list[str]] = {}
+ for interface_qn, implementer_qns in sorted(interface_implementers.items()):
+ for implementer_qn in sorted(implementer_qns):
+ inverted.setdefault(implementer_qn, []).append(interface_qn)
+ return inverted
+
+
+def _signature_arity(method_name: str) -> int | None:
+ # Number of top-level parameters in a signatured method name
+ # (`readField(A,JsonReader,BoundField)` -> 3, `create()` -> 0); None when the
+ # name carries no signature (Python/JS methods). Commas inside generics
+ # (`Map`) are nested, so only depth-0 commas separate parameters.
+ open_idx = method_name.find(cs.CHAR_PAREN_OPEN)
+ if open_idx < 0:
+ return None
+ inner = method_name[open_idx + 1 : method_name.rfind(cs.CHAR_PAREN_CLOSE)]
+ if not inner.strip():
+ return 0
+ depth = 0
+ count = 1
+ for ch in inner:
+ if ch in "<([":
+ depth += 1
+ elif ch in ">)]":
+ depth -= 1
+ elif ch == "," and depth == 0:
+ count += 1
+ return count
+
+
+def _find_override_by_arity(
+ parent_class: str,
+ method_name: str,
+ function_registry: FunctionRegistryTrieProtocol,
+) -> str | None:
+ # Override matching by exact signature fails when a subclass renames a generic
+ # type parameter (base `readField(A,...)` vs override `readField(T,...)`), which
+ # is a distinct qn. Java overriding is by name + erased parameter types, so fall
+ # back to a UNIQUE parent method with the same simple name and arity; ambiguous
+ # overloads (>1 candidate) are left unmatched rather than guessed.
+ arity = _signature_arity(method_name)
+ if arity is None:
+ return None
+ base_name = method_name.split(cs.CHAR_PAREN_OPEN, 1)[0]
+ prefix = f"{parent_class}{cs.SEPARATOR_DOT}"
+ matches: list[str] = []
+ for qn, node_type in function_registry.find_with_prefix(parent_class):
+ if node_type != NodeType.METHOD or not qn.startswith(prefix):
+ continue
+ leaf = qn[len(prefix) :]
+ if cs.SEPARATOR_DOT in leaf.split(cs.CHAR_PAREN_OPEN, 1)[0]:
+ continue # a method of a nested class, not directly on parent_class
+ if leaf.split(cs.CHAR_PAREN_OPEN, 1)[0] == base_name and (
+ _signature_arity(leaf) == arity
+ ):
+ matches.append(qn)
+ return matches[0] if len(matches) == 1 else None
def check_method_overrides(
@@ -46,10 +224,25 @@ def check_method_overrides(
function_registry: FunctionRegistryTrieProtocol,
class_inheritance: dict[str, list[str]],
ingestor: IngestorProtocol,
+ implemented_interfaces: dict[str, list[str]] | None = None,
+ csharp_methods: set[str] | None = None,
+ csharp_override_methods: set[str] | None = None,
) -> None:
- if class_qn not in class_inheritance:
+ implemented = implemented_interfaces or {}
+ if class_qn not in class_inheritance and class_qn not in implemented:
return
+ # A C# method overrides a base CLASS member only with the explicit
+ # `override` modifier; a `new`/implicit-hide member must not. Interface
+ # members need no modifier, so gate only class-parent matches.
+ csharp_gated = (
+ csharp_methods is not None
+ and method_qn in csharp_methods
+ and (
+ csharp_override_methods is None or method_qn not in csharp_override_methods
+ )
+ )
+
queue = deque([class_qn])
visited = {class_qn}
@@ -58,22 +251,50 @@ def check_method_overrides(
if current_class != class_qn:
parent_method_qn = f"{current_class}.{method_name}"
+ if parent_method_qn not in function_registry:
+ # Fall back to name+arity so a generic type-var rename in the
+ # override signature still matches the base method.
+ parent_method_qn = (
+ _find_override_by_arity(
+ current_class, method_name, function_registry
+ )
+ or parent_method_qn
+ )
- if parent_method_qn in function_registry:
- ingestor.ensure_relationship_batch(
- (cs.NodeLabel.METHOD, cs.KEY_QUALIFIED_NAME, method_qn),
- cs.RelationshipType.OVERRIDES,
- (cs.NodeLabel.METHOD, cs.KEY_QUALIFIED_NAME, parent_method_qn),
+ # The parent member must BE a method: a ctor of a nested class
+ # that inherits its encloser (node::inner_node : node) shares
+ # its name with the nested CLASS registered at parent.name, and
+ # an OVERRIDES onto that class qn is a label-mismatched phantom.
+ if function_registry.get(parent_method_qn) == NodeType.METHOD:
+ # Skip a gated C# member's CLASS-parent match, but keep
+ # walking: it may still implement an interface member deeper.
+ parent_is_interface = (
+ function_registry.get(current_class) == NodeType.INTERFACE
)
- logger.debug(
- logs.CLASS_METHOD_OVERRIDE.format(
- method_qn=method_qn, parent_method_qn=parent_method_qn
+ if not (csharp_gated and not parent_is_interface):
+ ingestor.ensure_relationship_batch(
+ (cs.NodeLabel.METHOD, cs.KEY_QUALIFIED_NAME, method_qn),
+ cs.RelationshipType.OVERRIDES,
+ (
+ cs.NodeLabel.METHOD,
+ cs.KEY_QUALIFIED_NAME,
+ parent_method_qn,
+ ),
)
- )
- return
+ logger.debug(
+ logs.CLASS_METHOD_OVERRIDE,
+ method_qn=method_qn,
+ parent_method_qn=parent_method_qn,
+ )
+ return
- if current_class in class_inheritance:
- for parent_class_qn in class_inheritance[current_class]:
- if parent_class_qn not in visited:
- visited.add(parent_class_qn)
- queue.append(parent_class_qn)
+ # Superclasses first: when both a base class and an interface declare
+ # the method, the edge lands on the base, matching Java resolution.
+ # chain() instead of list concat: this runs per BFS node per method.
+ for parent_class_qn in chain(
+ class_inheritance.get(current_class, ()),
+ implemented.get(current_class, ()),
+ ):
+ if parent_class_qn not in visited:
+ visited.add(parent_class_qn)
+ queue.append(parent_class_qn)
diff --git a/codebase_rag/parsers/class_ingest/mixin.py b/codebase_rag/parsers/class_ingest/mixin.py
index 2ba3f8f8c..1833e6859 100644
--- a/codebase_rag/parsers/class_ingest/mixin.py
+++ b/codebase_rag/parsers/class_ingest/mixin.py
@@ -1,29 +1,59 @@
from __future__ import annotations
from abc import abstractmethod
+from bisect import bisect_left, bisect_right
+from collections.abc import Mapping
from pathlib import Path
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, NamedTuple
from loguru import logger
from tree_sitter import Node, QueryCursor
from ... import constants as cs
from ... import logs
-from ...types_defs import ASTNode, PropertyDict
+from ...config import settings
+from ...language_spec import LanguageSpec
+from ...types_defs import (
+ ASTNode,
+ CppDefinitionSpan,
+ DeferredCppInherit,
+ DeferredInherit,
+ FunctionLocation,
+ FunctionSpanKey,
+ NodeType,
+ PropertyDict,
+)
+from ...utils.path_utils import cached_relative_path, cached_resolve_posix
+from ..cpp import CppTypeInferenceEngine
+from ..cpp import utils as cpp_utils
+from ..csharp import utils as csharp_utils
+from ..dart import utils as dart_utils
+from ..dart.type_inference import DartTypeInferenceEngine
+from ..go import GoTypeInferenceEngine
from ..java import utils as java_utils
-from ..py import resolve_class_name
+from ..py import external_stdlib_base_method_names, resolve_class_name
+from ..rs import RustTypeInferenceEngine
from ..rs import utils as rs_utils
-from ..utils import ingest_method, safe_decode_text
+from ..utils import (
+ extract_modifiers_and_decorators,
+ function_span_key,
+ ingest_method,
+ record_cpp_definition_span,
+ safe_decode_text,
+ sorted_captures,
+)
from . import cpp_modules
from . import identity as id_
from . import method_override as mo
from . import node_type as nt
+from . import parent_extraction as pe
from . import relationships as rel
+from .utils import csharp_has_override_modifier, find_child_by_type
if TYPE_CHECKING:
- from ...language_spec import LanguageSpec
from ...services import IngestorProtocol
from ...types_defs import (
+ DeferredParentLink,
FunctionRegistryTrieProtocol,
LanguageQueries,
SimpleNameLookup,
@@ -31,7 +61,86 @@
from ..import_processor import ImportProcessor
+def _java_anonymous_base_type(method_node: Node, class_node: Node) -> str | None:
+ # If `method_node` sits inside an anonymous class body between it and
+ # `class_node` (`new Base(){ ... m() ... }`), return the anon class's base type
+ # name (the object_creation's `type` field, generic args stripped). None when the
+ # method belongs directly to the enclosing class, not an anonymous subclass.
+ current = method_node.parent
+ while current is not None and current is not class_node:
+ if current.type == cs.TS_CLASS_BODY:
+ parent = current.parent
+ if parent is not None and parent.type == cs.TS_OBJECT_CREATION_EXPRESSION:
+ type_node = parent.child_by_field_name(cs.FIELD_TYPE)
+ if type_node is not None and type_node.text is not None:
+ return type_node.text.decode(cs.ENCODING_UTF8).split(
+ cs.CHAR_ANGLE_OPEN, 1
+ )[0]
+ return None
+ current = current.parent
+ return None
+
+
+def _is_nested_inside_function(
+ node: Node, class_body: Node, lang_config: LanguageSpec
+) -> bool:
+ current = node.parent
+ while current and current is not class_body:
+ if (
+ current.type in lang_config.function_node_types
+ and current.child_by_field_name(cs.FIELD_BODY) is not None
+ ):
+ return True
+ current = current.parent
+ return False
+
+
+def _method_belongs_directly(
+ method_node: Node, class_node: Node, lang_config: LanguageSpec
+) -> bool:
+ current = method_node.parent
+ while current is not None:
+ if current == class_node:
+ return True
+ if current.type in lang_config.class_node_types or (
+ current.type in lang_config.function_node_types
+ and current.child_by_field_name(cs.FIELD_BODY) is not None
+ ):
+ return False
+ current = current.parent
+ return False
+
+
+def _skip_method(
+ method_node: Node, class_node: Node, class_body: Node, lang_config: LanguageSpec
+) -> bool:
+ if settings.CAPTURE_FUNCTION_LOCAL_DEFINITIONS:
+ return not _method_belongs_directly(method_node, class_node, lang_config)
+ return _is_nested_inside_function(method_node, class_body, lang_config)
+
+
+class _DeferredForwardDecl(NamedTuple):
+ # A C/C++ forward declaration held back until every file's real definitions
+ # are registered, so we can tell an only-forward-declared type (keep it) from
+ # one that also has a bodied definition elsewhere (drop the phantom).
+ class_node: Node
+ class_name: str
+ # The namespace-qualified name (module-file prefix stripped, so `A::Foo` is
+ # `A.Foo` regardless of which header declares it). Comparing on this, not the
+ # bare simple name, keeps a forward-declared `B::Foo` when only `A::Foo` is
+ # defined, while still matching a cross-file forward/definition of one type.
+ ns_qn: str
+ module_qn: str
+ language: cs.SupportedLanguage
+ lang_queries: LanguageQueries
+ lang_config: LanguageSpec
+ file_path: Path | None
+ sorted_func_nodes: list[Node] | None
+ func_node_starts: list[int] | None
+
+
class ClassIngestMixin:
+ __slots__ = ()
ingestor: IngestorProtocol
repo_path: Path
project_name: str
@@ -40,6 +149,55 @@ class ClassIngestMixin:
module_qn_to_file_path: dict[str, Path]
import_processor: ImportProcessor
class_inheritance: dict[str, list[str]]
+ dart_annotated_overrides: dict[str, list[tuple[str, str]]]
+ dart_extends_type_args: dict[str, list[str]]
+ class_field_types: dict[str, dict[str, str]]
+ java_anon_overrides: list[tuple[str, str, str, str]]
+ csharp_methods: set[str]
+ csharp_override_methods: set[str]
+ csharp_partial_groups: dict[str, list[str]]
+ csharp_generic_methods: set[str]
+ csharp_class_generic_arity: dict[str, int]
+ csharp_method_return_types: dict[str, tuple[str, int]]
+ _csharp_partial_index: dict[str, list[str]]
+ csharp_extension_methods: dict[str, list[tuple[str, str, str, int]]]
+ csharp_base_kinds: dict[tuple[str, int], dict[str, str]]
+ csharp_type_locations: dict[tuple[str, int], str]
+ class_field_guard_inner: dict[str, dict[str, str]]
+ method_return_types: dict[str, str]
+ interface_implementers: dict[str, set[str]]
+ function_locations: dict[FunctionSpanKey, FunctionLocation]
+ cpp_definition_spans: dict[str, list[CppDefinitionSpan]]
+ _deferred_forward_decls: list[_DeferredForwardDecl]
+ _deferred_parent_links: list[DeferredParentLink]
+ _deferred_cpp_inherits: list[DeferredCppInherit]
+ _deferred_inherits: list[DeferredInherit]
+ cpp_module_interfaces: set[str]
+ _deferred_cpp_module_impls: list[tuple[str, str]]
+ declared_module_qns: set[str]
+ pending_endpoints: list[tuple[cs.NodeLabel, str, list[str], str | None]]
+
+ def _namespace_qn(self, class_qn: str, module_qn: str) -> str:
+ # Strip the module-file prefix so two nodes for the same C++ type in
+ # different headers share one key (`leveldb.db.x.h.leveldb.VersionSet` and
+ # `...y.h.leveldb.VersionSet` both -> `leveldb.VersionSet`), while types in
+ # different namespaces stay distinct.
+ prefix = f"{module_qn}{cs.SEPARATOR_DOT}"
+ return class_qn[len(prefix) :] if class_qn.startswith(prefix) else class_qn
+
+ def _namespace_qn_has_definition(self, ns_qn: str) -> bool:
+ # A real definition of this namespace-qualified type is registered iff some
+ # class qn ends with it (`....leveldb.VersionSet`). find_ending_with is
+ # indexed by simple name, and because it is queried AFTER the registry is
+ # rehydrated from the graph, it also covers definitions in files an
+ # incremental run did not re-parse (issue: a forward decl must still drop
+ # when its definition lives in an unchanged file).
+ simple = ns_qn.rsplit(cs.SEPARATOR_DOT, 1)[-1]
+ suffix = f"{cs.SEPARATOR_DOT}{ns_qn}"
+ return any(
+ qn.endswith(suffix)
+ for qn in self.function_registry.find_ending_with(simple)
+ )
@abstractmethod
def _get_docstring(self, node: ASTNode) -> str | None: ...
@@ -47,6 +205,34 @@ def _get_docstring(self, node: ASTNode) -> str | None: ...
@abstractmethod
def _extract_decorators(self, node: ASTNode) -> list[str]: ...
+ @abstractmethod
+ def _emit_or_defer_defines(
+ self,
+ parent_label: str,
+ parent_qn: str,
+ child_label: str,
+ child_qn: str,
+ module_qn: str,
+ fallback_label: str | None = None,
+ fallback_qn: str | None = None,
+ parent_span: FunctionSpanKey | None = None,
+ ) -> None: ...
+
+ @abstractmethod
+ def _determine_function_parent(
+ self,
+ func_node: Node,
+ func_qn: str,
+ module_qn: str,
+ lang_config: LanguageSpec,
+ language: cs.SupportedLanguage | None = None,
+ ) -> tuple[str, str, FunctionSpanKey | None]: ...
+
+ @abstractmethod
+ def _resolve_cpp_class_qn(
+ self, class_name: str, module_qn: str, exclude_qn: str | None = None
+ ) -> tuple[str, bool]: ...
+
def _resolve_to_qn(self, name: str, module_qn: str) -> str:
return self._resolve_class_name(name, module_qn) or f"{module_qn}.{name}"
@@ -63,8 +249,33 @@ def _ingest_cpp_module_declarations(
self.repo_path,
self.project_name,
self.ingestor,
+ self.cpp_module_interfaces,
+ self._deferred_cpp_module_impls,
)
+ def resolve_deferred_cpp_module_impls(self) -> int:
+ """Emit ModuleImplementation IMPLEMENTS edges for interfaces that exist.
+
+ An implementation unit (`module X;`) whose interface (`export module
+ X;`) lives in no parsed file has nothing real to point at; emitting
+ the edge anyway would mint a phantom the database drops.
+ """
+ deferred = self._deferred_cpp_module_impls
+ if not deferred:
+ return 0
+ self._deferred_cpp_module_impls = []
+ emitted = 0
+ for impl_qn, interface_qn in deferred:
+ if interface_qn not in self.cpp_module_interfaces:
+ continue
+ self.ingestor.ensure_relationship_batch(
+ (cs.NodeLabel.MODULE_IMPLEMENTATION, cs.KEY_QUALIFIED_NAME, impl_qn),
+ cs.RelationshipType.IMPLEMENTS,
+ (cs.NodeLabel.MODULE_INTERFACE, cs.KEY_QUALIFIED_NAME, interface_qn),
+ )
+ emitted += 1
+ return emitted
+
def _find_cpp_exported_classes(self, root_node: Node) -> list[Node]:
return cpp_modules.find_cpp_exported_classes(root_node)
@@ -73,36 +284,461 @@ def _ingest_classes_and_methods(
root_node: Node,
module_qn: str,
language: cs.SupportedLanguage,
- queries: dict[cs.SupportedLanguage, LanguageQueries],
+ queries: Mapping[cs.SupportedLanguage, LanguageQueries],
+ combined_captures: dict[str, list] | None = None,
) -> None:
lang_queries = queries[language]
- if not (query := lang_queries[cs.QUERY_CLASSES]):
- return
-
lang_config: LanguageSpec = lang_queries[cs.QUERY_CONFIG]
- cursor = QueryCursor(query)
- captures = cursor.captures(root_node)
- class_nodes = captures.get(cs.CAPTURE_CLASS, [])
- module_nodes = captures.get(cs.ONEOF_MODULE, [])
+
+ if combined_captures is not None:
+ class_nodes = list(combined_captures.get(cs.CAPTURE_CLASS, []))
+ module_nodes = combined_captures.get(cs.ONEOF_MODULE, [])
+ else:
+ if not (query := lang_queries[cs.QUERY_CLASSES]):
+ return
+ cursor = QueryCursor(query)
+ captures = sorted_captures(cursor, root_node)
+ class_nodes = captures.get(cs.CAPTURE_CLASS, [])
+ module_nodes = captures.get(cs.ONEOF_MODULE, [])
if language == cs.SupportedLanguage.CPP:
class_nodes.extend(self._find_cpp_exported_classes(root_node))
file_path = self.module_qn_to_file_path.get(module_qn)
+ sorted_func_nodes: list[Node] | None = None
+ func_node_starts: list[int] | None = None
+ if combined_captures is not None and cs.CAPTURE_FUNCTION in combined_captures:
+ sorted_func_nodes = combined_captures[cs.CAPTURE_FUNCTION]
+ func_node_starts = [n.start_byte for n in sorted_func_nodes]
+
for class_node in class_nodes:
- if isinstance(class_node, Node):
- self._process_class_node(
- class_node,
- module_qn,
- language,
- lang_queries,
- lang_config,
- file_path,
- )
+ self._process_class_node(
+ class_node,
+ module_qn,
+ language,
+ lang_queries,
+ lang_config,
+ file_path,
+ sorted_func_nodes=sorted_func_nodes,
+ func_node_starts=func_node_starts,
+ )
self._process_inline_modules(module_nodes, module_qn, lang_config)
+ def resolve_deferred_forward_declarations(self) -> int:
+ # Run after every file's definitions are registered. A deferred forward
+ # declaration whose class name already produced a real node is a phantom
+ # (the bodied definition exists) -> drop it. Otherwise it is the only
+ # representation of the type -> register it now. Deterministic: the
+ # deferred list is in file (sorted) order, and the first surviving forward
+ # declaration of an only-declared type claims the name for the rest.
+ deferred = getattr(self, "_deferred_forward_decls", None)
+ if not deferred:
+ return 0
+ self._deferred_forward_decls = []
+ registered = 0
+ for entry in deferred:
+ # Drop the forward declaration only when a real definition of the SAME
+ # namespace-qualified type exists (not merely the same simple name in
+ # another namespace). Otherwise it is the type's only node -> keep it.
+ if self._namespace_qn_has_definition(entry.ns_qn):
+ continue
+ self._process_class_node(
+ entry.class_node,
+ entry.module_qn,
+ entry.language,
+ entry.lang_queries,
+ entry.lang_config,
+ entry.file_path,
+ sorted_func_nodes=entry.sorted_func_nodes,
+ func_node_starts=entry.func_node_starts,
+ allow_defer=False,
+ )
+ registered += 1
+ return registered
+
+ def resolve_deferred_cpp_inherits(self) -> int:
+ """Emit C++ INHERITS edges now that every class is registered.
+
+ A base written in another header resolves scope-first across all
+ parsed files (the same lookup out-of-class methods use); a base that
+ resolves nowhere emits no edge, because the module-anchored guess is a
+ phantom endpoint the database silently drops anyway. Resolved qns
+ replace the guesses in class_inheritance in place so Pass-3 method
+ resolution and override detection walk the real hierarchy.
+ """
+ deferred = self._deferred_cpp_inherits
+ if not deferred:
+ return 0
+ self._deferred_cpp_inherits = []
+ emitted = 0
+ for entry in deferred:
+ parent_qn = self._resolve_cpp_base_qn(entry)
+ if parent_qn is None:
+ continue
+ bases = self.class_inheritance.get(entry.child_qn)
+ if bases is not None and entry.base_index < len(bases):
+ bases[entry.base_index] = parent_qn
+ rel.create_inheritance_relationship(
+ entry.child_label,
+ entry.child_qn,
+ parent_qn,
+ self.function_registry,
+ self.ingestor,
+ entry.base_index,
+ )
+ emitted += 1
+ return emitted
+
+ def _resolve_cpp_base_qn(self, entry: DeferredCppInherit) -> str | None:
+ normalized = entry.base_name.replace(
+ cs.SEPARATOR_DOUBLE_COLON, cs.SEPARATOR_DOT
+ )
+ # Scope-first (see resolve_deferred_cpp_methods): the enclosing
+ # namespaces distinguish same-leaf classes. The child itself is
+ # excluded so `class Type : public other::Type` never self-inherits.
+ candidates = [normalized]
+ if entry.namespace_path:
+ candidates.insert(
+ 0, f"{entry.namespace_path}{cs.SEPARATOR_DOT}{normalized}"
+ )
+ for candidate in candidates:
+ parent_qn, resolved = self._resolve_cpp_class_qn(
+ candidate, "", exclude_qn=entry.child_qn
+ )
+ if resolved:
+ return parent_qn
+ # The guess strips a qualified base (`other::Type`) to its leaf, so
+ # it can collide with the CHILD's own qn when the base is
+ # unresolvable; a self-INHERITS is never real.
+ if (
+ entry.guess_qn != entry.child_qn
+ and self.function_registry.get(entry.guess_qn) is not None
+ ):
+ return entry.guess_qn
+ return None
+
+ def resolve_deferred_inherits(self) -> int:
+ """Emit non-C++ INHERITS/IMPLEMENTS edges now that every class is registered.
+
+ A parent in a file parsed later resolves against the full registry; a
+ parent that resolves nowhere (java.lang.Exception, Rust Send/Sync)
+ emits no edge, because the module-anchored guess is a phantom endpoint
+ the database silently drops anyway. Resolved base qns replace the
+ guesses in class_inheritance in place so Pass-3 method resolution and
+ override detection walk the real hierarchy.
+ """
+ deferred = self._deferred_inherits
+ self._deferred_inherits = []
+ emitted = 0
+ # `implements` targets never enter class_inheritance, so the override
+ # ancestry walk needs the resolved first-party IMPLEMENTS parents
+ # collected here or a registered interface's method is wrongly rooted.
+ dart_implements: dict[str, list[str]] = {}
+ for entry in deferred:
+ child_type = self.function_registry.get(entry.child_qn)
+ if child_type is None:
+ continue
+ resolved = self._resolve_deferred_parent_qn(entry)
+ is_dart = entry.language == cs.SupportedLanguage.DART
+ if resolved is None:
+ continue
+ parent_qn, is_external = resolved
+ external_label: str | None = None
+ if is_external:
+ # The import pass mints the same node for IMPORTS edges, so
+ # this MERGEs idempotently when the base was imported.
+ self.import_processor.ensure_external_module_node(parent_qn)
+ external_label = cs.NodeLabel.EXTERNAL_MODULE.value
+ if entry.rel_type == cs.RelationshipType.IMPLEMENTS:
+ # Dart has no `interface` keyword: `implements X` targets a
+ # concrete class, so a hardcoded Interface label would dangle.
+ # Resolve the target's real registered label (Interface for a
+ # true interface, Class/Enum for a Dart type); external stays
+ # EXTERNAL_MODULE.
+ interface_label = external_label or rel.get_node_type_for_inheritance(
+ parent_qn, self.function_registry
+ )
+ rel.create_implements_relationship(
+ str(child_type),
+ entry.child_qn,
+ parent_qn,
+ self.ingestor,
+ interface_label=interface_label,
+ )
+ self.interface_implementers.setdefault(parent_qn, set()).add(
+ entry.child_qn
+ )
+ if is_dart and not is_external:
+ dart_implements.setdefault(entry.child_qn, []).append(parent_qn)
+ else:
+ bases = self.class_inheritance.get(entry.child_qn)
+ if bases is not None and entry.base_index < len(bases):
+ bases[entry.base_index] = parent_qn
+ rel.create_inheritance_relationship(
+ str(child_type),
+ entry.child_qn,
+ parent_qn,
+ self.function_registry,
+ self.ingestor,
+ entry.base_index,
+ parent_label=external_label,
+ )
+ emitted += 1
+ self._flag_dart_external_overrides(dart_implements)
+ return emitted
+
+ def _flag_dart_external_overrides(
+ self,
+ implements_map: dict[str, list[str]],
+ ) -> None:
+ # Every Dart class ultimately extends Object, so an @override whose
+ # name NO registered ancestor defines can only target external code:
+ # a framework base (direct or through any chain of first-party
+ # classes), an implemented external interface, or Object itself
+ # (toString/hashCode/operator==, invoked by interpolation and
+ # collections). Those methods root the dead-code walk; a name a
+ # registered ancestor defines resolves via OVERRIDES edges and must
+ # stay an ordinary candidate. Runs here because ancestry is only
+ # known after every class is registered; the partial row MERGEs into
+ # the node ingested during Pass 2.
+ for child_qn, methods in self.dart_annotated_overrides.items():
+ for method_qn, method_name in methods:
+ if self._registered_ancestor_defines(
+ child_qn, method_name, implements_map
+ ):
+ continue
+ self.ingestor.ensure_node_batch(
+ cs.NodeLabel.METHOD,
+ {
+ cs.KEY_QUALIFIED_NAME: method_qn,
+ cs.KEY_OVERRIDES_EXTERNAL: True,
+ },
+ )
+
+ def _registered_ancestor_defines(
+ self,
+ class_qn: str,
+ name: str,
+ implements_map: dict[str, list[str]],
+ ) -> bool:
+ seen: set[str] = set()
+ stack = list(self.class_inheritance.get(class_qn, []))
+ stack.extend(implements_map.get(class_qn, []))
+ while stack:
+ ancestor = stack.pop()
+ if ancestor in seen:
+ continue
+ seen.add(ancestor)
+ if self.function_registry.get(ancestor) is None:
+ continue
+ if (
+ self.function_registry.get(f"{ancestor}{cs.SEPARATOR_DOT}{name}")
+ is not None
+ ):
+ return True
+ stack.extend(self.class_inheritance.get(ancestor, []))
+ stack.extend(implements_map.get(ancestor, []))
+ return False
+
+ def _resolve_deferred_parent_qn(
+ self, entry: DeferredInherit
+ ) -> tuple[str, bool] | None:
+ """Resolve a deferred parent to (qn, is_external), or None for no edge.
+
+ First-party wins; a qn outside the project prefix is positive external
+ knowledge (import-mapped or ::-qualified) and keeps its edge onto an
+ ExternalModule node. A project-prefixed qn that is not a real class
+ may still name one through a src-root layout (setup.py maps src/ to
+ the distribution name), recovered by a unique whole-segment suffix
+ match. A module-anchored guess that re-resolves nowhere externalizes
+ its WRITTEN name (canonicalized for JS globals and java.lang): a base
+ name resolving to no indexed class is by construction defined outside
+ the indexed tree, and dropping it would lose a syntactic inheritance
+ fact the source declares.
+ """
+ if entry.parent_qn == entry.child_qn:
+ # Parse-time resolution can land on the child ITSELF. A
+ # self-edge is never real. In C# the written base can be an
+ # ARITY sibling (`class Foo : Foo
@@ -35,855 +67,128 @@ An accurate Retrieval-Augmented Generation (RAG) system that analyzes multi-lang
## Latest News 🔥
-- **[NEW]** **MCP Server Integration**: Code-Graph-RAG now works as an MCP server with Claude Code! Query and edit your codebase using natural language directly from Claude Code. [Setup Guide](docs/claude-code-setup.md)
-- [2025/10/21] **Semantic Code Search**: Added intent-based code search using UniXcoder embeddings. Find functions by describing what they do (e.g., "error handling functions", "authentication code") rather than by exact names.
-
-## 🚀 Features
-
-- **Multi-Language Support**:
-
-
-| Language | Status | Extensions | Functions | Classes/Structs | Modules | Package Detection | Additional Features |
-|--------|------|----------|---------|---------------|-------|-----------------|-------------------|
-| C++ | Fully Supported | .cpp, .h, .hpp, .cc, .cxx, .hxx, .hh, .ixx, .cppm, .ccm | ✓ | ✓ | ✓ | ✓ | Constructors, destructors, operator overloading, templates, lambdas, C++20 modules, namespaces |
-| Java | Fully Supported | .java | ✓ | ✓ | ✓ | - | Generics, annotations, modern features (records/sealed classes), concurrency, reflection |
-| JavaScript | Fully Supported | .js, .jsx | ✓ | ✓ | ✓ | - | ES6 modules, CommonJS, prototype methods, object methods, arrow functions |
-| Lua | Fully Supported | .lua | ✓ | - | ✓ | - | Local/global functions, metatables, closures, coroutines |
-| Python | Fully Supported | .py | ✓ | ✓ | ✓ | ✓ | Type inference, decorators, nested functions |
-| Rust | Fully Supported | .rs | ✓ | ✓ | ✓ | ✓ | impl blocks, associated functions |
-| TypeScript | Fully Supported | .ts, .tsx | ✓ | ✓ | ✓ | - | Interfaces, type aliases, enums, namespaces, ES6/CommonJS modules |
-| C# | In Development | .cs | ✓ | ✓ | ✓ | - | Classes, interfaces, generics (planned) |
-| Go | In Development | .go | ✓ | ✓ | ✓ | - | Methods, type declarations |
-| PHP | In Development | .php | ✓ | ✓ | ✓ | - | Classes, functions, namespaces |
-| Scala | In Development | .scala, .sc | ✓ | ✓ | ✓ | - | Case classes, objects |
-
-- **🌳 Tree-sitter Parsing**: Uses Tree-sitter for robust, language-agnostic AST parsing
-- **📊 Knowledge Graph Storage**: Uses Memgraph to store codebase structure as an interconnected graph
-- **🗣️ Natural Language Querying**: Ask questions about your codebase in plain English
-- **🤖 AI-Powered Cypher Generation**: Supports both cloud models (Google Gemini), local models (Ollama), and OpenAI models for natural language to Cypher translation
-- **🤖 OpenAI Integration**: Leverage OpenAI models to enhance AI functionalities.
-- **📝 Code Snippet Retrieval**: Retrieves actual source code snippets for found functions/methods
-- **✍️ Advanced File Editing**: Surgical code replacement with AST-based function targeting, visual diff previews, and exact code block modifications
-- **⚡️ Shell Command Execution**: Can execute terminal commands for tasks like running tests or using CLI tools.
-- **🚀 Interactive Code Optimization**: AI-powered codebase optimization with language-specific best practices and interactive approval workflow
-- **📚 Reference-Guided Optimization**: Use your own coding standards and architectural documents to guide optimization suggestions
-- **🔗 Dependency Analysis**: Parses `pyproject.toml` to understand external dependencies
-- **🎯 Nested Function Support**: Handles complex nested functions and class hierarchies
-- **🔄 Language-Agnostic Design**: Unified graph schema across all supported languages
-
-## 🏗️ Architecture
-
-The system consists of two main components:
-
-1. **Multi-language Parser**: Tree-sitter based parsing system that analyzes codebases and ingests data into Memgraph
-2. **RAG System** (`codebase_rag/`): Interactive CLI for querying the stored knowledge graph
-
-
-## 📋 Prerequisites
-
-- Python 3.12+
-- Docker & Docker Compose (for Memgraph)
-- **cmake** (required for building pymgclient dependency)
-- **ripgrep** (`rg`) (required for shell command text searching)
-- **For cloud models**: Google Gemini API key
-- **For local models**: Ollama installed and running
-- `uv` package manager
-
-### Installing cmake and ripgrep
-
-On macOS:
-```bash
-brew install cmake ripgrep
-```
+
+- **Ruby Support**: Ruby joins the graph through a new pluggable ast-grep tier that adds a language from a single YAML pattern file, emitting `Module`, `Function`, and `Class` nodes plus import edges without a hand-written parser.
+- **Structural Search & Replace**: Find and rewrite code by AST pattern with ast-grep, exposed as agent tools so you can match and transform structure across the whole codebase instead of relying on text or regex.
+- **Data-Flow Tracing**: New `FLOWS_TO` taint edges follow values through assignments, function calls, and I/O sinks, with coverage across C#, Java, C, and Go.
+
-On Linux (Ubuntu/Debian):
-```bash
-sudo apt-get update
-sudo apt-get install cmake ripgrep
-```
+See [NEWS.md](NEWS.md) for the full history.
-On Linux (CentOS/RHEL):
-```bash
-sudo yum install cmake
-sudo dnf install ripgrep
-# Note: ripgrep may need to be installed from EPEL or via cargo
-```
+## What It Does
-## 🛠️ Installation
+Point Code-Graph-RAG at a repository and it reads every source file, extracts functions, classes, methods, modules, and the relationships between them, and stores the result as an interconnected graph. Once the graph exists you can:
-```bash
-git clone https://github.com/vitali87/code-graph-rag.git
+- Ask questions about the codebase in natural language and get answers grounded in the real structure.
+- Retrieve the actual source of any function, class, or method by name or by intent.
+- Edit code through the agent with AST-based surgical patching and a diff preview before anything changes.
+- Optimise code against language best practices or your own coding standards.
+- Find dead code by walking call and reference edges from entry points.
+- Search and rewrite structurally by AST pattern with ast-grep.
-cd code-graph-rag
-```
+## How It Works
-2. **Install dependencies**:
+The system has two components:
-For basic Python support:
-```bash
-uv sync
-```
+1. **Multi-language parser.** A Tree-sitter based parser reads the codebase and ingests functions, classes, methods, modules, and their relationships into Memgraph under a single language-agnostic schema.
+2. **RAG system** (`codebase_rag/`). An interactive CLI that turns natural language into Cypher queries, retrieves matching code, and drives AI-powered editing and optimisation.
-For full multi-language support:
-```bash
-uv sync --extra treesitter-full
```
-
-For development (including tests and pre-commit hooks):
-```bash
-make dev
+Source Code -> Tree-sitter Parser -> AST Analysis -> Memgraph Knowledge Graph
+ |
+User Query -> AI Model (Cypher Gen) -> Cypher Query -> Graph Results -> Response
```
-This installs all dependencies and sets up pre-commit hooks automatically.
+See the [Architecture Overview](docs/architecture/overview.md) and [Graph Schema](docs/architecture/graph-schema.md) for the full picture.
-This installs Tree-sitter grammars for all supported languages (see Multi-Language Support section).
-
-3. **Set up environment variables**:
-```bash
-cp .env.example .env
-# Edit .env with your configuration (see options below)
-```
+## Supported Languages
-### Configuration Options
+Python, TypeScript, TSX, JavaScript, Rust, Go, Java, C, C++, C#, PHP, Lua, and Dart are fully supported. Scala is in development, and Ruby has structural support (modules, functions, classes, and imports) through the pluggable ast-grep tier. See the [Language Support](docs/architecture/language-support.md) matrix for per-language capabilities.
-The new provider-explicit configuration supports mixing different providers for orchestrator and cypher models.
+## Installation
-#### Option 1: All Ollama (Local Models)
+`cgr` is published to PyPI. Install it system-wide with the `treesitter-full` (all languages) and `semantic` (vector search) extras:
```bash
-# .env file
-ORCHESTRATOR_PROVIDER=ollama
-ORCHESTRATOR_MODEL=llama3.2
-ORCHESTRATOR_ENDPOINT=http://localhost:11434/v1
-
-CYPHER_PROVIDER=ollama
-CYPHER_MODEL=codellama
-CYPHER_ENDPOINT=http://localhost:11434/v1
-```
-
-#### Option 2: All OpenAI Models
-```bash
-# .env file
-ORCHESTRATOR_PROVIDER=openai
-ORCHESTRATOR_MODEL=gpt-4o
-ORCHESTRATOR_API_KEY=sk-your-openai-key
-
-CYPHER_PROVIDER=openai
-CYPHER_MODEL=gpt-4o-mini
-CYPHER_API_KEY=sk-your-openai-key
-```
+# with uv (recommended)
+uv tool install "code-graph-rag[treesitter-full,semantic]"
-#### Option 3: All Google Models
-```bash
-# .env file
-ORCHESTRATOR_PROVIDER=google
-ORCHESTRATOR_MODEL=gemini-2.5-pro
-ORCHESTRATOR_API_KEY=your-google-api-key
-
-CYPHER_PROVIDER=google
-CYPHER_MODEL=gemini-2.5-flash
-CYPHER_API_KEY=your-google-api-key
+# or with pipx
+pipx install "code-graph-rag[treesitter-full,semantic]"
```
-#### Option 4: Mixed Providers
-```bash
-# .env file - Google orchestrator + Ollama cypher
-ORCHESTRATOR_PROVIDER=google
-ORCHESTRATOR_MODEL=gemini-2.5-pro
-ORCHESTRATOR_API_KEY=your-google-api-key
-
-CYPHER_PROVIDER=ollama
-CYPHER_MODEL=codellama
-CYPHER_ENDPOINT=http://localhost:11434/v1
-```
+You also need Docker (for Memgraph), `cmake`, and `ripgrep`. Full prerequisites, source installs, and environment setup are in the [Installation](docs/getting-started/installation.md) guide.
-Get your Google API key from [Google AI Studio](https://aistudio.google.com/app/apikey).
+## Quick Start
-**Install and run Ollama**:
```bash
-# Install Ollama (macOS/Linux)
-curl -fsSL https://ollama.ai/install.sh | sh
-
-# Pull required models
-ollama pull llama3.2
-# Or try other models like:
-# ollama pull llama3
-# ollama pull mistral
-# ollama pull codellama
+# Start the packaged Memgraph + Qdrant stack (no compose file needed)
+cgr daemon up
-# Ollama will automatically start serving on localhost:11434
+# Parse a repository into the graph, then query it
+cgr start --repo-path /path/to/repo --update-graph --clean
+cgr start --repo-path /path/to/repo
```
-> **Note**: Local models provide privacy and no API costs, but may have lower accuracy compared to cloud models like Gemini.
+The [Quick Start](docs/getting-started/quickstart.md) guide walks through parsing, querying, and exporting in five minutes.
-4. **Start Memgraph database**:
-```bash
-docker-compose up -d
-```
+## MCP Server
-## 🛠️ Makefile Commands
-
-Use the Makefile for common development tasks:
-
-
-| Command | Description |
-|-------|-----------|
-| `make help` | Show this help message |
-| `make all` | Install everything for full development environment (deps, grammars, hooks, tests) |
-| `make install` | Install project dependencies with full language support |
-| `make python` | Install project dependencies for Python only |
-| `make dev` | Setup development environment (install deps + pre-commit hooks) |
-| `make test` | Run unit tests only (fast, no Docker) |
-| `make test-parallel` | Run unit tests in parallel (fast, no Docker) |
-| `make test-integration` | Run integration tests (requires Docker) |
-| `make test-all` | Run all tests including integration and e2e (requires Docker) |
-| `make test-parallel-all` | Run all tests in parallel including integration and e2e (requires Docker) |
-| `make clean` | Clean up build artifacts and cache |
-| `make build-grammars` | Build grammar submodules |
-| `make watch` | Watch repository for changes and update graph in real-time |
-| `make readme` | Regenerate README.md from codebase |
-| `make lint` | Run ruff check |
-| `make format` | Run ruff format |
-| `make typecheck` | Run type checking with ty |
-| `make check` | Run all checks: lint, typecheck, test |
-| `make pre-commit` | Run all pre-commit checks locally (comprehensive test before commit) |
-
-
-## 🎯 Usage
-
-The Code-Graph-RAG system offers four main modes of operation:
-1. **Parse & Ingest**: Build knowledge graph from your codebase
-2. **Interactive Query**: Ask questions about your code in natural language
-3. **Export & Analyze**: Export graph data for programmatic analysis
-4. **AI Optimization**: Get AI-powered optimization suggestions for your code.
-5. **Editing**: Perform surgical code replacements and modifications with precise targeting.
-
-### Step 1: Parse a Repository
-
-Parse and ingest a multi-language repository into the knowledge graph:
-
-**For the first repository (clean start):**
-```bash
-cgr start --repo-path /path/to/repo1 --update-graph --clean
-```
+Code-Graph-RAG runs as an [MCP](https://modelcontextprotocol.io) server so Claude Code and other MCP clients can query and edit your codebase directly. See the [MCP Server](docs/guide/mcp-server.md) guide for setup.
-**For additional repositories (preserve existing data):**
-```bash
-cgr start --repo-path /path/to/repo2 --update-graph
-cgr start --repo-path /path/to/repo3 --update-graph
-```
+## Documentation
-**Control Memgraph batch flushing:**
-```bash
-# Flush every 5,000 records instead of the default from settings
-cgr start --repo-path /path/to/repo --update-graph \
- --batch-size 5000
-```
+**Getting Started**
+- [Installation](docs/getting-started/installation.md)
+- [Quick Start](docs/getting-started/quickstart.md)
+- [Configuration](docs/getting-started/configuration.md)
-The system automatically detects and processes files for all supported languages (see Multi-Language Support section).
+**User Guide**
+- [CLI Reference](docs/guide/cli-reference.md)
+- [Interactive Querying](docs/guide/interactive-querying.md)
+- [Code Optimisation](docs/guide/code-optimization.md)
+- [Dead Code Detection](docs/guide/dead-code.md)
+- [Graph Export](docs/guide/graph-export.md)
+- [Real-Time Updates](docs/guide/realtime-updates.md)
+- [MCP Server](docs/guide/mcp-server.md)
-### Step 2: Query the Codebase
+**Architecture**
+- [Overview](docs/architecture/overview.md)
+- [Graph Schema](docs/architecture/graph-schema.md)
+- [Language Support](docs/architecture/language-support.md)
+- [Data-Flow Edges](docs/architecture/data-flow-edges.md)
-Start the interactive RAG CLI:
+**Python SDK**
+- [Overview](docs/sdk/overview.md)
+- [Graph Loader](docs/sdk/graph-loader.md)
+- [Cypher Generator](docs/sdk/cypher-generator.md)
+- [Semantic Search](docs/sdk/semantic-search.md)
-```bash
-cgr start --repo-path /path/to/your/repo
-```
+**Advanced**
+- [Adding Languages](docs/advanced/adding-languages.md)
+- [Ignore Patterns](docs/advanced/ignore-patterns.md)
+- [Building Binaries](docs/advanced/building-binaries.md)
+- [Troubleshooting](docs/advanced/troubleshooting.md)
-### Step 2.5: Real-Time Graph Updates (Optional)
+## Enterprise Services
-For active development, you can keep your knowledge graph automatically synchronized with code changes using the realtime updater. This is particularly useful when you're actively modifying code and want the AI assistant to always work with the latest codebase structure.
-
-**What it does:**
-- Watches your repository for file changes (create, modify, delete)
-- Automatically updates the knowledge graph in real-time
-- Maintains consistency by recalculating all function call relationships
-- Filters out irrelevant files (`.git`, `node_modules`, etc.)
-
-**How to use:**
-
-Run the realtime updater in a separate terminal:
-
-```bash
-# Using Python directly
-python realtime_updater.py /path/to/your/repo
-
-# Or using the Makefile
-make watch REPO_PATH=/path/to/your/repo
-```
-
-**With custom Memgraph settings:**
-```bash
-# Python
-python realtime_updater.py /path/to/your/repo --host localhost --port 7687 --batch-size 1000
-
-# Makefile
-make watch REPO_PATH=/path/to/your/repo HOST=localhost PORT=7687 BATCH_SIZE=1000
-```
-
-**Multi-terminal workflow:**
-```bash
-# Terminal 1: Start the realtime updater
-python realtime_updater.py ~/my-project
-
-# Terminal 2: Run the AI assistant
-cgr start --repo-path ~/my-project
-```
-
-**Performance note:** The updater currently recalculates all CALLS relationships on every file change to ensure consistency. This prevents "island" problems where changes in one file aren't reflected in relationships from other files, but may impact performance on very large codebases with frequent changes. **Note:** Optimization of this behavior is a work in progress.
-
-**CLI Arguments:**
-- `repo_path` (required): Path to repository to watch
-- `--host`: Memgraph host (default: `localhost`)
-- `--port`: Memgraph port (default: `7687`)
-- `--batch-size`: Number of buffered nodes/relationships before flushing to Memgraph
-
-**Specify Custom Models:**
-```bash
-# Use specific local models
-cgr start --repo-path /path/to/your/repo \
- --orchestrator ollama:llama3.2 \
- --cypher ollama:codellama
-
-# Use specific Gemini models
-cgr start --repo-path /path/to/your/repo \
- --orchestrator google:gemini-2.0-flash-thinking-exp-01-21 \
- --cypher google:gemini-2.5-flash-lite-preview-06-17
-
-# Use mixed providers
-cgr start --repo-path /path/to/your/repo \
- --orchestrator google:gemini-2.0-flash-thinking-exp-01-21 \
- --cypher ollama:codellama
-```
-
-Example queries (works across all supported languages):
-- "Show me all classes that contain 'user' in their name"
-- "Find functions related to database operations"
-- "What methods does the User class have?"
-- "Show me functions that handle authentication"
-- "List all TypeScript components"
-- "Find Rust structs and their methods"
-- "Show me Go interfaces and implementations"
-- "Find all C++ operator overloads in the Matrix class"
-- "Show me C++ template functions with their specializations"
-- "List all C++ namespaces and their contained classes"
-- "Find C++ lambda expressions used in algorithms"
-- "Add logging to all database connection functions"
-- "Refactor the User class to use dependency injection"
-- "Convert these Python functions to async/await pattern"
-- "Add error handling to authentication methods"
-- "Optimize this function for better performance"
-
-### Step 3: Export Graph Data
-
-For programmatic access and integration with other tools, you can export the entire knowledge graph to JSON:
-
-**Export during graph update:**
-```bash
-cgr start --repo-path /path/to/repo --update-graph --clean -o my_graph.json
-```
-
-**Export existing graph without updating:**
-```bash
-cgr export -o my_graph.json
-```
-
-**Optional: adjust Memgraph batching during export:**
-```bash
-cgr export -o my_graph.json --batch-size 5000
-```
-
-**Working with exported data:**
-```python
-from codebase_rag.graph_loader import load_graph
-
-# Load the exported graph
-graph = load_graph("my_graph.json")
-
-# Get summary statistics
-summary = graph.summary()
-print(f"Total nodes: {summary['total_nodes']}")
-print(f"Total relationships: {summary['total_relationships']}")
-
-# Find specific node types
-functions = graph.find_nodes_by_label("Function")
-classes = graph.find_nodes_by_label("Class")
-
-# Analyze relationships
-for func in functions[:5]:
- relationships = graph.get_relationships_for_node(func.node_id)
- print(f"Function {func.properties['name']} has {len(relationships)} relationships")
-```
-
-**Example analysis script:**
-```bash
-python examples/graph_export_example.py my_graph.json
-```
-
-This provides a reliable, programmatic way to access your codebase structure without LLM restrictions, perfect for:
-- Integration with other tools
-- Custom analysis scripts
-- Building documentation generators
-- Creating code metrics dashboards
-
-### Step 4: Code Optimization
-
-For AI-powered codebase optimization with best practices guidance:
-
-**Basic optimization for a specific language:**
-```bash
-cgr optimize python --repo-path /path/to/your/repo
-```
-
-**Optimization with reference documentation:**
-```bash
-cgr optimize python \
- --repo-path /path/to/your/repo \
- --reference-document /path/to/best_practices.md
-```
-
-**Using specific models for optimization:**
-```bash
-cgr optimize javascript \
- --repo-path /path/to/frontend \
- --orchestrator google:gemini-2.0-flash-thinking-exp-01-21
-
-# Optional: override Memgraph batch flushing during optimization
-cgr optimize javascript --repo-path /path/to/frontend \
- --batch-size 5000
-```
+Code-Graph-RAG is open source and free to use. For organisations that need more, we offer **fully managed cloud-hosted solutions** and **on-premise deployments**:
-**Supported Languages for Optimization:**
-All supported languages: `python`, `javascript`, `typescript`, `rust`, `go`, `java`, `scala`, `cpp`
+- **Cloud-Hosted Deployment**: Managed cloud infrastructure for both the graph database and the AI agent connection. Zero infrastructure overhead, so we handle scaling, updates, and availability while your team focuses on building.
+- **On-Premise & Air-Gapped Deployment**: Deploy Code-Graph-RAG entirely within your own environment, including air-gapped networks. Full data sovereignty for regulated industries and security-sensitive organisations.
-**How It Works:**
-1. **Analysis Phase**: The agent analyzes your codebase structure using the knowledge graph
-2. **Pattern Recognition**: Identifies common anti-patterns, performance issues, and improvement opportunities
-3. **Best Practices Application**: Applies language-specific best practices and patterns
-4. **Interactive Approval**: Presents each optimization suggestion for your approval before implementation
-5. **Guided Implementation**: Implements approved changes with detailed explanations
-
-**Example Optimization Session:**
-```
-Starting python optimization session...
-┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
-┃ The agent will analyze your python codebase and propose specific ┃
-┃ optimizations. You'll be asked to approve each suggestion before ┃
-┃ implementation. Type 'exit' or 'quit' to end the session. ┃
-┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
-
-🔍 Analyzing codebase structure...
-📊 Found 23 Python modules with potential optimizations
-
-💡 Optimization Suggestion #1:
- File: src/data_processor.py
- Issue: Using list comprehension in a loop can be optimized
- Suggestion: Replace with generator expression for memory efficiency
-
- [y/n] Do you approve this optimization?
-```
-
-**Reference Document Support:**
-You can provide reference documentation (like coding standards, architectural guidelines, or best practices documents) to guide the optimization process:
-
-```bash
-# Use company coding standards
-cgr optimize python \
- --reference-document ./docs/coding_standards.md
-
-# Use architectural guidelines
-cgr optimize java \
- --reference-document ./ARCHITECTURE.md
-
-# Use performance best practices
-cgr optimize rust \
- --reference-document ./docs/performance_guide.md
-```
-
-The agent will incorporate the guidance from your reference documents when suggesting optimizations, ensuring they align with your project's standards and architectural decisions.
-
-**Common CLI Arguments:**
-- `--orchestrator`: Specify provider:model for main operations (e.g., `google:gemini-2.0-flash-thinking-exp-01-21`, `ollama:llama3.2`)
-- `--cypher`: Specify provider:model for graph queries (e.g., `google:gemini-2.5-flash-lite-preview-06-17`, `ollama:codellama`)
-- `--repo-path`: Path to repository (defaults to current directory)
-- `--batch-size`: Override Memgraph flush batch size (defaults to `MEMGRAPH_BATCH_SIZE` in settings)
-- `--reference-document`: Path to reference documentation (optimization only)
-
-## 🔌 MCP Server (Claude Code Integration)
-
-Code-Graph-RAG can run as an MCP (Model Context Protocol) server, enabling seamless integration with Claude Code and other MCP clients.
-
-### Quick Setup
-
-```bash
-claude mcp add --transport stdio code-graph-rag \
- --env TARGET_REPO_PATH=/absolute/path/to/your/project \
- --env CYPHER_PROVIDER=openai \
- --env CYPHER_MODEL=gpt-4 \
- --env CYPHER_API_KEY=your-api-key \
- -- uv run --directory /path/to/code-graph-rag code-graph-rag mcp-server
-```
-
-### Available Tools
-
-
-| Tool | Description |
-|----|-----------|
-| `list_projects` | List all indexed projects in the knowledge graph database. Returns a list of project names that have been indexed. |
-| `delete_project` | Delete a specific project from the knowledge graph database. This removes all nodes associated with the project while preserving other projects. Use list_projects first to see available projects. |
-| `wipe_database` | WARNING: Completely wipe the entire database, removing ALL indexed projects. This cannot be undone. Use delete_project for removing individual projects. |
-| `index_repository` | Parse and ingest the repository into the Memgraph knowledge graph. This builds a comprehensive graph of functions, classes, dependencies, and relationships. Note: This preserves other projects - only the current project is re-indexed. |
-| `query_code_graph` | Query the codebase knowledge graph using natural language. Ask questions like 'What functions call UserService.create_user?' or 'Show me all classes that implement the Repository interface'. |
-| `get_code_snippet` | Retrieve source code for a function, class, or method by its qualified name. Returns the source code, file path, line numbers, and docstring. |
-| `surgical_replace_code` | Surgically replace an exact code block in a file using diff-match-patch. Only modifies the exact target block, leaving the rest unchanged. |
-| `read_file` | Read the contents of a file from the project. Supports pagination for large files. |
-| `write_file` | Write content to a file, creating it if it doesn't exist. |
-| `list_directory` | List contents of a directory in the project. |
-
-
-### Example Usage
-
-```
-> Index this repository
-> What functions call UserService.create_user?
-> Update the login function to add rate limiting
-```
-
-For detailed setup, see [Claude Code Setup Guide](docs/claude-code-setup.md).
-
-## 📊 Graph Schema
-
-The knowledge graph uses the following node types and relationships:
-
-### Node Types
-
-
-| Label | Properties |
-|-----|----------|
-| Project | `{name: string}` |
-| Package | `{qualified_name: string, name: string, path: string}` |
-| Folder | `{path: string, name: string}` |
-| File | `{path: string, name: string, extension: string}` |
-| Module | `{qualified_name: string, name: string, path: string}` |
-| Class | `{qualified_name: string, name: string, decorators: list[string]}` |
-| Function | `{qualified_name: string, name: string, decorators: list[string]}` |
-| Method | `{qualified_name: string, name: string, decorators: list[string]}` |
-| Interface | `{qualified_name: string, name: string}` |
-| Enum | `{qualified_name: string, name: string}` |
-| Type | `{qualified_name: string, name: string}` |
-| Union | `{qualified_name: string, name: string}` |
-| ModuleInterface | `{qualified_name: string, name: string, path: string}` |
-| ModuleImplementation | `{qualified_name: string, name: string, path: string, implements_module: string}` |
-| ExternalPackage | `{name: string, version_spec: string}` |
-
-
-### Language-Specific Mappings
-
-
-- **C++**: `class_specifier`, `declaration`, `enum_specifier`, `field_declaration`, `function_definition`, `lambda_expression`, `struct_specifier`, `template_declaration`, `union_specifier`
-- **Java**: `annotation_type_declaration`, `class_declaration`, `constructor_declaration`, `enum_declaration`, `interface_declaration`, `method_declaration`, `record_declaration`
-- **JavaScript**: `arrow_function`, `class`, `class_declaration`, `function_declaration`, `function_expression`, `generator_function_declaration`, `method_definition`
-- **Lua**: `function_declaration`, `function_definition`
-- **Python**: `class_definition`, `function_definition`
-- **Rust**: `closure_expression`, `enum_item`, `function_item`, `function_signature_item`, `impl_item`, `struct_item`, `trait_item`, `type_item`, `union_item`
-- **TypeScript**: `abstract_class_declaration`, `arrow_function`, `class`, `class_declaration`, `enum_declaration`, `function_declaration`, `function_expression`, `function_signature`, `generator_function_declaration`, `interface_declaration`, `internal_module`, `method_definition`, `type_alias_declaration`
-- **C#**: `anonymous_method_expression`, `class_declaration`, `constructor_declaration`, `destructor_declaration`, `enum_declaration`, `function_pointer_type`, `interface_declaration`, `lambda_expression`, `local_function_statement`, `method_declaration`, `struct_declaration`
-- **Go**: `function_declaration`, `method_declaration`, `type_declaration`
-- **PHP**: `anonymous_function`, `arrow_function`, `class_declaration`, `enum_declaration`, `function_definition`, `function_static_declaration`, `interface_declaration`, `trait_declaration`
-- **Scala**: `class_definition`, `function_declaration`, `function_definition`, `object_definition`, `trait_definition`
-
-
-### Relationships
-
-
-| Source | Relationship | Target |
-|------|------------|------|
-| Project, Package, Folder | CONTAINS_PACKAGE | Package |
-| Project, Package, Folder | CONTAINS_FOLDER | Folder |
-| Project, Package, Folder | CONTAINS_FILE | File |
-| Project, Package, Folder | CONTAINS_MODULE | Module |
-| Module | DEFINES | Class, Function |
-| Class | DEFINES_METHOD | Method |
-| Module | IMPORTS | Module |
-| Module | EXPORTS | Class, Function |
-| Module | EXPORTS_MODULE | ModuleInterface |
-| Module | IMPLEMENTS_MODULE | ModuleImplementation |
-| Class | INHERITS | Class |
-| Class | IMPLEMENTS | Interface |
-| Method | OVERRIDES | Method |
-| ModuleImplementation | IMPLEMENTS | ModuleInterface |
-| Project | DEPENDS_ON_EXTERNAL | ExternalPackage |
-| Function, Method | CALLS | Function, Method |
-
-
-## 🔧 Configuration
-
-Configuration is managed through environment variables in `.env` file:
-
-### Provider-Specific Settings
-
-#### Orchestrator Model Configuration
-- `ORCHESTRATOR_PROVIDER`: Provider name (`google`, `openai`, `ollama`)
-- `ORCHESTRATOR_MODEL`: Model ID (e.g., `gemini-2.5-pro`, `gpt-4o`, `llama3.2`)
-- `ORCHESTRATOR_API_KEY`: API key for the provider (if required)
-- `ORCHESTRATOR_ENDPOINT`: Custom endpoint URL (if required)
-- `ORCHESTRATOR_PROJECT_ID`: Google Cloud project ID (for Vertex AI)
-- `ORCHESTRATOR_REGION`: Google Cloud region (default: `us-central1`)
-- `ORCHESTRATOR_PROVIDER_TYPE`: Google provider type (`gla` or `vertex`)
-- `ORCHESTRATOR_THINKING_BUDGET`: Thinking budget for reasoning models
-- `ORCHESTRATOR_SERVICE_ACCOUNT_FILE`: Path to service account file (for Vertex AI)
-
-#### Cypher Model Configuration
-- `CYPHER_PROVIDER`: Provider name (`google`, `openai`, `ollama`)
-- `CYPHER_MODEL`: Model ID (e.g., `gemini-2.5-flash`, `gpt-4o-mini`, `codellama`)
-- `CYPHER_API_KEY`: API key for the provider (if required)
-- `CYPHER_ENDPOINT`: Custom endpoint URL (if required)
-- `CYPHER_PROJECT_ID`: Google Cloud project ID (for Vertex AI)
-- `CYPHER_REGION`: Google Cloud region (default: `us-central1`)
-- `CYPHER_PROVIDER_TYPE`: Google provider type (`gla` or `vertex`)
-- `CYPHER_THINKING_BUDGET`: Thinking budget for reasoning models
-- `CYPHER_SERVICE_ACCOUNT_FILE`: Path to service account file (for Vertex AI)
-
-### System Settings
-- `MEMGRAPH_HOST`: Memgraph hostname (default: `localhost`)
-- `MEMGRAPH_PORT`: Memgraph port (default: `7687`)
-- `MEMGRAPH_HTTP_PORT`: Memgraph HTTP port (default: `7444`)
-- `LAB_PORT`: Memgraph Lab port (default: `3000`)
-- `MEMGRAPH_BATCH_SIZE`: Batch size for Memgraph operations (default: `1000`)
-- `TARGET_REPO_PATH`: Default repository path (default: `.`)
-- `LOCAL_MODEL_ENDPOINT`: Fallback endpoint for Ollama (default: `http://localhost:11434/v1`)
-
-### Custom Ignore Patterns
-
-You can specify additional directories to exclude by creating a `.cgrignore` file in your repository root:
-
-```
-# Comments start with #
-vendor
-.custom_cache
-my_build_output
-```
-
-- One directory name per line
-- Lines starting with `#` are comments
-- Blank lines are ignored
-- Patterns are exact directory name matches (not globs)
-- Patterns from `.cgrignore` are merged with `--exclude` flags and auto-detected directories
-
-### Key Dependencies
-
-
-- **loguru**: Python logging made (stupidly) simple
-- **mcp**: Model Context Protocol SDK
-- **pydantic-ai**: Agent Framework / shim to use Pydantic with LLMs
-- **pydantic-settings**: Settings management using Pydantic
-- **pymgclient**: Memgraph database adapter for Python language
-- **python-dotenv**: Read key-value pairs from a .env file and set them as environment variables
-- **toml**: Python Library for Tom's Obvious, Minimal Language
-- **tree-sitter-python**: Python grammar for tree-sitter
-- **tree-sitter**: Python bindings to the Tree-sitter parsing library
-- **watchdog**: Filesystem events monitoring
-- **typer**: Typer, build great CLIs. Easy to code. Based on Python type hints.
-- **rich**: Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal
-- **prompt-toolkit**: Library for building powerful interactive command lines in Python
-- **diff-match-patch**: Repackaging of Google's Diff Match and Patch libraries.
-- **click**: Composable command line interface toolkit
-- **protobuf**
-- **defusedxml**: XML bomb protection for Python stdlib modules
-- **huggingface-hub**: Client library to download and publish models, datasets and other repos on the huggingface.co hub
-
-
-## 🤖 Agentic Workflow & Tools
-
-The agent is designed with a deliberate workflow to ensure it acts with context and precision, especially when modifying the file system.
-
-### Core Tools
-
-The agent has access to a suite of tools to understand and interact with the codebase:
-
-
-| Tool | Description |
-|----|-----------|
-| `query_graph` | Query the codebase knowledge graph using natural language questions. Ask in plain English about classes, functions, methods, dependencies, or code structure. Examples: 'Find all functions that call each other', 'What classes are in the user module', 'Show me functions with the longest call chains'. |
-| `read_file` | Reads the content of text-based files. For documents like PDFs or images, use the 'analyze_document' tool instead. |
-| `create_file` | Creates a new file with content. IMPORTANT: Check file existence first! Overwrites completely WITHOUT showing diff. Use only for new files, not existing file modifications. |
-| `replace_code` | Surgically replaces specific code blocks in files. Requires exact target code and replacement. Only modifies the specified block, leaving rest of file unchanged. True surgical patching. |
-| `list_directory` | Lists the contents of a directory to explore the codebase. |
-| `analyze_document` | Analyzes documents (PDFs, images) to answer questions about their content. |
-| `execute_shell` | Executes shell commands from allowlist. Read-only commands run without approval; write operations require user confirmation. |
-| `semantic_search` | Performs a semantic search for functions based on a natural language query describing their purpose, returning a list of potential matches with similarity scores. |
-| `get_function_source` | Retrieves the source code for a specific function or method using its internal node ID, typically obtained from a semantic search result. |
-| `get_code_snippet` | Retrieves the source code for a specific function, class, or method using its full qualified name. |
-
-
-### Intelligent and Safe File Editing
-
-The agent uses AST-based function targeting with Tree-sitter for precise code modifications. Features include:
-- **Visual diff preview** before changes
-- **Surgical patching** that only modifies target code blocks
-- **Multi-language support** across all supported languages
-- **Security sandbox** preventing edits outside project directory
-- **Smart function matching** with qualified names and line numbers
-
-
-
-## 🌍 Multi-Language Support
-
-### Adding New Languages
-
-Code-Graph-RAG makes it easy to add support for any language that has a Tree-sitter grammar. The system automatically handles grammar compilation and integration.
-
-> **⚠️ Recommendation**: While you can add languages yourself, we recommend waiting for official full support to ensure optimal parsing quality, comprehensive feature coverage, and robust integration. The languages marked as "In Development" above will receive dedicated optimization and testing.
-
-> **💡 Request Support**: If you want a specific language to be officially supported, please [submit an issue](https://github.com/vitali87/code-graph-rag/issues) with your language request.
-
-#### Quick Start: Add a Language
-
-Use the built-in language management tool to add any Tree-sitter supported language:
-
-```bash
-# Add a language using the standard tree-sitter repository
-cgr language add-grammar 
+ 
+ 
+ 
+