Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions codegraph-skill/codegraph/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Use `doctor` only when install, native-runtime, or artifact health is the task.
Then choose the smallest useful follow-up:

- explore: `codegraph explore "how does auth reach db?" --pretty`
- live file: `codegraph file <path> --offset 1 --limit 200 --pretty`
- packet: `codegraph packet get <file|symbol|sql-object|handle> --pretty`
- search: `codegraph search "auth user" --json`
- explain: `codegraph explain <file|symbol|sql-object|handle>`
Expand Down Expand Up @@ -66,15 +67,33 @@ Current high-value surfaces:
- `review --summary`: compact reviewer handoff
- `duplicates --profile cleanup`: refactor ROI ordering
- `duplicates --json`: full grouped duplicate data
- `file`: live bounded file bytes with JSON default, exact numbered lines, and explicit pagination

Treat duplicate leads and call-compatibility hints as review leads, not proof.

## Live File Views

Use `codegraph file <path>` for current disk content; use `--pretty` for a readable view or the default JSON for exact fields. Returned pages use a 1-based `--offset`, a `--limit` default/cap of 2000/10000 lines, and a `--max-bytes` default/cap of 80000/500000 unnumbered text bytes. A separate 16 MiB hard input-size limit rejects larger raw reads and structural text-config summaries before unbounded I/O, bounding complete-stream binary/UTF-8 validation and total-line counting.

```bash
codegraph file src/auth.ts --offset 201 --limit 200 --max-bytes 80000
codegraph file src/auth.ts --include-graph-context --pretty
```

`content` is exact `number<TAB>line` text with no line-number padding, while `text` omits number prefixes. A trailing newline becomes a final numbered empty line. Follow `page.nextOffset`; beyond EOF, JSON `content` and `text` are empty and pretty output says `Lines: none at offset <offset> of <totalLines>`.

Graph context is never automatic. Add `--include-graph-context` only when direct `usedBy` paths, imports, and symbols are worth an index/freshness check; ordinary file bytes stay live and independent of index freshness.

Within the 16 MiB input limit, recognized environment, authentication, and credential text configs return structural summaries after the full raw stream is validated. Default key-material summaries use metadata, may report size, and do not read raw secret bytes. Use `--allow-sensitive` only for deliberate raw access; it does not bypass the input-size, binary, NUL, or UTF-8 guards, so `.p12` and `.pfx` bundles summarize by default and reject raw access.

An exact project-relative file-path query such as `codegraph explore src/auth.ts --json` adds the same response as `fileView`; `--no-source` suppresses it. `--include-graph-context` and `--allow-sensitive` pass through only when explicitly present.

## MCP

If MCP tools are available, prefer them over repeated CLI invocations.
Use MCP `explore`, `orient`, `search`, `get_file`, `packet_get`, `goto`, `refs`, `deps`, `rdeps`, `path`, `impact`, `review`, and `query_sqlite` first.
Use `get_file` for bounded live file reads; its content uses exact `number<TAB>line` formatting. Set `includeGraphContext: true` to opt into direct graph context capped at 100 importers, imports, and symbols each.
After edits, check MCP response `freshness`: `refreshed` means Codegraph rebuilt before answering, and `stale` includes a reason plus bounded changed-file metadata before indexed context is trusted.
Use `get_file` for bounded live reads with `offset`, `limit`, and `maxBytes`; continue at `page.nextOffset`, and set `allowSensitive: true` only for intentional raw sensitive reads. Set `includeGraphContext: true` to opt into freshness-backed direct context capped at 100 importers, imports, and symbols each.
For plain `get_file` reads, `freshness` does not gate the live bytes. After indexed calls or graph-context reads, check it before trusting indexed data: `refreshed` means Codegraph rebuilt, and `stale` includes a reason plus bounded changed-file metadata.
Run `refresh_index` before `artifact_build` when MCP reports a stale index; artifact writes refuse stale snapshots.
Fall back to CLI when MCP is unavailable.

Expand Down
23 changes: 23 additions & 0 deletions docs/agent-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,29 @@ codegraph explore src/auth.ts --json --limit 5 --max-packets 3
Explore orchestrates existing search, packet, path, reverse-dependency, and candidate-test surfaces. It returns `schemaVersion: 1`, the query, analysis metadata, summary bullets, anchors, bounded packets, dependency paths, blast radius, candidate tests, follow-ups, flat limits, and omission counts.
Use `--no-source` when the caller only needs anchors, paths, and follow-up commands.

## Live file reads

Use `file` when the target path is known and the agent needs current source rather than an indexed explanation packet. Its JSON default is best for tool chaining; `--pretty` keeps the exact `number<TAB>line` source format while adding a readable header and next-page command.

```bash
codegraph file src/auth.ts --offset 1 --limit 200 --pretty
codegraph file src/auth.ts --offset 201 --limit 200 --max-bytes 80000
codegraph file src/auth.ts --include-graph-context --json
```

The live bytes, exact whole-file `totalLines`, and `page.nextOffset` do not depend on a fresh index. Graph context is never automatic: opt in with `--include-graph-context`, then treat `freshness` as the state of that indexed context rather than the file content.

The 16 MiB hard input-size limit is separate from the `--max-bytes` output-page cap. It rejects larger raw reads and structural text-config summaries before unbounded I/O, bounding complete-stream binary/UTF-8 validation and total-line counting. At an offset beyond EOF, JSON `content` and `text` are empty; pretty output says `Lines: none at offset <offset> of <totalLines>`.

If the `explore` query is only an exact indexed file path, JSON adds `fileView` with the same file-read contract and pretty output renders a `File view` section. An agent can continue from `fileView.page.nextOffset`; `--no-source` disables it.

```bash
codegraph explore src/auth.ts --json
codegraph explore src/auth.ts --include-graph-context --pretty
```

Secret-prone text configs return structural keys instead of raw values unless `--allow-sensitive` is passed intentionally; key material defaults to metadata, may report file size, and does not read raw secret bytes. Intentional raw access remains subject to the 16 MiB input limit and still rejects known binary extensions, NUL bytes, and malformed or incomplete UTF-8, so `.p12` and `.pfx` bundles remain metadata-only in practice. See [CLI reference](./cli.md#live-file-views) for exact fields, limits, trailing-newline behavior, and sensitive kinds.

## Orientation packets

Start with `orient` when an agent needs compact repo context without flooding the first prompt:
Expand Down
40 changes: 40 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,13 @@ codegraph explore src/auth.ts --json
codegraph search "build review report" --json
codegraph explain src/review.ts --json
codegraph packet get src/cli.ts --pretty
# Read a live file; JSON is the default
codegraph file src/cli.ts
codegraph file src/cli.ts --offset 201 --limit 100 --max-bytes 40000 --pretty

# Add direct importers, imports, and symbols only when needed
codegraph file src/cli.ts --include-graph-context --json

codegraph search "public users" --mode sql --json
codegraph search "handle login" --from src/auth.ts --mode graph --depth 1 --json
codegraph search --help
Expand Down Expand Up @@ -271,6 +278,39 @@ Short JSON shape:
- Use `packet get` with file paths, symbol names, SQL object names, file/symbol/chunk/SQL/graph handles, or review handles to retrieve bounded evidence plus follow-up commands.
- Agent commands reuse the incremental index path and default to disk cache. Use shared index flags such as `--cache`, `--cache-strict`, `--cache-verify`, `--threads`, `--native`, `--workers`, `--include-glob`, `--ignore-glob`, and `--no-gitignore` when the packet should match a specific scan mode.

#### Live file views

`file <path>` reads the current file bytes from disk, independent of index freshness. JSON is the default; pass `--pretty` for a header, optional graph summary, exact numbered lines, and a copyable next-page command.

- `--offset <line>` is the 1-based first line and defaults to `1`. `--limit <lines>` is the maximum line count, defaults to `2000`, and is capped at `10000`.
- For raw file pages, `--max-bytes <bytes>` bounds unnumbered text including line separators before numbering; it defaults to `80000` and is capped at `500000`. A raw page can therefore end before `--limit`, and `truncated` is true when the boundary cuts a selected line.
- A separate 16 MiB hard input-size limit rejects larger raw reads and structural text-config summaries before unbounded I/O. It bounds complete-stream binary/UTF-8 validation and total-line counting, not the returned page size.
- `totalLines` counts the complete live file, not only the returned prefix. Follow `page.nextOffset` when present; an offset beyond the end returns empty `content` and `text` while retaining the exact total, and pretty output says `Lines: none at offset <offset> of <totalLines>`.
- `content` uses `lineFormat: "number-tab-line"`: an unpadded decimal line number, one tab, then the source line. `text` contains the same selected source lines without number prefixes.
- A trailing newline creates a final numbered empty line. For example, `alpha\nbeta\n` has `totalLines: 3`, and its last `content` line is `3\t`.
- `--include-graph-context` is explicit opt-in and adds up to 100 direct `usedBy` paths, imports, and symbols. Plain file reads do not build or consult the index; `freshness` describes indexed context separately and never changes the live bytes returned.
- Within the 16 MiB input limit, ordinary file reads and structural summaries for recognized environment, authentication, and credential text configs validate the full raw stream, rejecting known binary extensions, NUL bytes, and malformed or incomplete UTF-8 before returning bounded content or extracting bounded keys. Default key-material summaries instead use file metadata, may report size, and do not read raw secret bytes; `--allow-sensitive` requests raw values but does not bypass the input-size, binary, NUL, or UTF-8 guards, so `.p12` and `.pfx` bundles summarize by default and reject raw access. For text-config summaries, `truncated` reports an incomplete bounded structural scan.

The JSON response fields are `schemaVersion`, `file`, effective `offset` and `limit`, exact `totalLines`, numbered `content`, `lineFormat`, unnumbered `text`, `truncated`, and `freshness`. Optional fields are `page: { nextOffset }`, `graphContext: { usedBy, imports, symbols }`, and `sensitive: { kind, redacted, allowSensitiveRequired }`.

```json
{
"schemaVersion": 1,
"file": "src/cli.ts",
"offset": 201,
"limit": 2,
"totalLines": 487,
"content": "201\texport function run(): void {\n202\t return;",
"lineFormat": "number-tab-line",
"text": "export function run(): void {\n return;",
"truncated": false,
"freshness": { "state": "fresh" },
"page": { "nextOffset": 203 }
}
```

An `explore` query that is only an indexed project-relative file path, such as `codegraph explore src/auth.ts --json`, adds this same live response as top-level `fileView`. A uniquely matching basename also resolves; `--no-source` suppresses the file view, while `--include-graph-context` and `--allow-sensitive` pass through explicitly.

`search` is deterministic and vectorless. Hybrid search is code-first by default: source symbols and implementation files outrank docs unless `--mode text` is explicit or docs are the strongest remaining evidence. Search JSON now includes top-level `analysis` metadata plus per-result `provenance` so mixed or reduced runs stay visible. `explain` resolves file paths, symbol names, SQL object names, and search handles into bounded packets with symbols, graph context, references, snippets, duplicate context, SQL facts, review tasks, candidate tests, analysis metadata, limits, omissions, and follow-ups. Use `--max-duplicates` to tune duplicate context in `explain` and `packet get`; duplicate context also uses an internal pair budget and reports skipped duplicate work through omission counts.

`explore` is a facade over existing primitives, not a second search engine. It returns `schemaVersion: 1`, the original query, `analysis`, summary bullets, anchors, packets, dependency paths, blast radius with per-entry omitted lower bounds, candidate tests, follow-ups, flat limits, and omission counts; path and blast-radius omissions may be lower bounds after bounded scans reach their caps.
Expand Down
67 changes: 63 additions & 4 deletions docs/library-api.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Library API

Programmatic APIs for indexing, graph building, agent search/explain/artifacts, MCP handlers, chunking, SQL artifact facts, read-only SQLite inspection, and impact analysis.
Programmatic APIs for indexing, graph building, live file views, agent search/explain/artifacts, MCP handlers, chunking, SQL artifact facts, read-only SQLite inspection, and impact analysis.

For sessions, streaming workflows, tool wrappers, and review-oriented recipes, see [docs/agent-workflows.md](./agent-workflows.md).

Expand Down Expand Up @@ -45,8 +45,8 @@ const index = await buildProjectIndex(root, {
The npm package exposes these supported entry points:

- `@lzehrung/codegraph` for the compatibility root surface.
- `@lzehrung/codegraph/agent` for agent sessions, orient/search/explain,
artifacts, and MCP handler helpers.
- `@lzehrung/codegraph/agent` for agent sessions, live file views,
orient/search/explain, artifacts, and MCP handler helpers.
- `@lzehrung/codegraph/graphs` for graph builders, graph queries, renderers,
symbol graphs, grep, hotspots, cycles, and unresolved-import helpers.
- `@lzehrung/codegraph/indexer` for project indexing, navigation, references,
Expand All @@ -66,7 +66,7 @@ as three groups:
navigation (`buildProjectIndex`, `buildProjectIndexIncremental`,
`goToDefinition`, `findReferences`, symbol handles, graph builders and
renderers), impact and review reports, sessions, agent search/explain/artifact
helpers, MCP handlers, SQLite helpers, SQL artifact APIs, chunking, config,
and file-view helpers, MCP handlers, SQLite helpers, SQL artifact APIs, chunking, config,
language metadata, and native runtime capability checks.
- Public-legacy APIs remain exported for existing callers but are lower-level
building blocks. This includes parser-facing helpers such as `parseFile`,
Expand All @@ -83,6 +83,57 @@ Future API narrowing should happen by first documenting replacements on these
subpath facades, then adding deprecation notes before removing root
compatibility exports.

## Live file views

`getCodegraphFileView()` reads a confined project file directly from disk. `getCodegraphFileViewWithSession()` accepts an existing `AgentSession` for optional graph reuse, and `formatAgentFileViewResponse()` renders the same response as stable pretty text.

```ts
import {
createAgentSession,
formatAgentFileViewResponse,
getCodegraphFileView,
getCodegraphFileViewWithSession,
} from "@lzehrung/codegraph";

const page = await getCodegraphFileView({
root: process.cwd(),
file: "src/auth.ts",
offset: 1,
limit: 200,
maxBytes: 80_000,
});

if (page.page?.nextOffset) {
const next = await getCodegraphFileView({
root: process.cwd(),
file: page.file,
offset: page.page.nextOffset,
limit: page.limit,
});
console.log(next.content);
}

const session = createAgentSession({ root: process.cwd() });
const contextual = await getCodegraphFileViewWithSession(session, {
root: process.cwd(),
file: "src/auth.ts",
includeGraphContext: true,
});
console.log(formatAgentFileViewResponse(contextual));
```

`AgentFileViewRequest` has `root`, `file`, optional 1-based `offset`, `limit`, `maxBytes`, `includeGraphContext`, `allowSensitive`, and `buildOptions`. Line and output-page byte defaults are `2000` and `80000`, capped at `10000` and `500000`; the page byte budget applies to unnumbered raw text, including line separators. A separate 16 MiB hard input-size limit rejects larger raw reads and structural text-config summaries before unbounded I/O, bounding complete-stream binary/UTF-8 validation and total-line counting.

`AgentFileViewResponse` always has `schemaVersion`, normalized project-relative `file`, effective `offset` and `limit`, exact whole-file `totalLines`, numbered `content`, `lineFormat: "number-tab-line"`, unnumbered `text`, `truncated`, and `freshness`. Optional `page.nextOffset`, `graphContext`, and `sensitive` fields describe remaining lines, requested indexed context, and sensitive-file handling respectively.

The numbered format is exactly unpadded decimal line number, tab, source line. A trailing newline contributes a final empty line, and pagination should resume from `page.nextOffset`; for raw pages, `maxBytes` may return fewer than `limit` lines and sets `truncated` when it cuts a selected line. An offset beyond EOF returns empty `content` and `text`, while `formatAgentFileViewResponse()` says `Lines: none at offset <offset> of <totalLines>`.

Graph context defaults off, so a plain call neither creates nor consults an index and its live bytes are independent of session freshness. Set `includeGraphContext: true` to request at most 100 direct `usedBy` files, imports, and `{ name, kind, line }` symbols; only this indexed context is governed by `freshness`.

Within the 16 MiB input limit, ordinary reads and structural summaries for recognized environment, authentication, and credential text configs validate the full raw stream before returning bounded content or extracting bounded keys; known binary extensions, NUL bytes, and malformed or incomplete UTF-8 are rejected. Default key-material summaries use file metadata, may report size, and do not read raw secret bytes; `allowSensitive: true` requests raw values but does not bypass the input-size, binary, NUL, or UTF-8 guards, so `.p12` and `.pfx` bundles summarize by default and reject raw access. For text-config summaries, `truncated` reports an incomplete bounded structural scan.

The root package and `@lzehrung/codegraph/agent` export these functions, constants, `AgentFileViewRequest`, `AgentFileViewResponse`, `AgentFileGraphContext`, `AgentFileViewSensitiveInfo`, and `AgentFileViewSensitiveKind`.

## Agent packets

`orientCodegraph()` returns compact first-turn context for an agent, and `getCodegraphPacket()` retrieves bounded evidence by file path, symbol name, SQL object name, or stable target:
Expand Down Expand Up @@ -136,9 +187,17 @@ const explored = await exploreCodegraph({
});

console.log(explored.summary, explored.paths, explored.followUps);

const exactFile = await exploreCodegraph({
root: process.cwd(),
query: "src/auth.ts",
includeGraphContext: false,
});
console.log(exactFile.fileView?.content, exactFile.fileView?.page?.nextOffset);
```

Use `exploreCodegraph()` when the caller has a broad question and needs one bounded response over the existing search, packet, path, reverse-dependency, and candidate-test surfaces. The response has `schemaVersion: 1`, the original query, `analysis`, summary bullets, anchors, packets, paths, blast radius with per-entry omitted lower bounds, candidate tests, follow-ups, flat limits, and omission counts. Path and blast-radius omissions may be lower bounds after bounded scans reach their caps.
When the entire query resolves to an indexed project-relative file path, or to one uniquely matching basename, the response also includes the live `fileView` described above. `includeSource: false` suppresses it; `includeGraphContext` and `allowSensitive` remain explicit request options and are never enabled automatically.

Use `mode: "sql"` for SQL objects, or pass `from` plus `depth` with `mode: "graph"` to boost matches near a file path, file/chunk/graph handle, symbol handle, SQL handle, or symbol name.

Expand Down
Loading