diff --git a/docs/plans/CODE-GENERATION-RESIDENT-SIZE-DESIGN.md b/docs/plans/CODE-GENERATION-RESIDENT-SIZE-DESIGN.md new file mode 100644 index 0000000000..8337279c8c --- /dev/null +++ b/docs/plans/CODE-GENERATION-RESIDENT-SIZE-DESIGN.md @@ -0,0 +1,204 @@ +# Code generation resident size design + +Status: proposed. No sealed-format revision is required. The sealed +generation format stays at `SEALED_GENERATION_FORMAT_REVISION_V1` 21. + +## The cost is in the decoded form, not the sealed bytes + +A cold build of this repository (6.1k files, 108 MB of source, 219k symbols, +458,883 chunks) holds a decoded `CodeIndexPublishedGenerationV1` whose +`retained_bytes` read 3.24 GB of a 4.11 GB process footprint. A 25.7k-file +repository read 4.32 GB of 5.49 GB. Each value is a lower bound because +`resident_bytes.rs` counts string and vector lengths, not allocator headers. + +| Part | Bytes | +| --- | ---: | +| Chunk records: text, exact terms, subtokens, and per-chunk identity and revision strings | 1.15 GB | +| Per-file records: clone bodies, schema evidence, and identities | 1.08 GB | +| Projection request and receipts: every chunk listed as added, with digests | 0.49 GB | +| Symbol records | 0.18 GB | +| Edges (684k) | 0.16 GB | +| Unresolved calls | 0.11 GB | +| Imports | 0.04 GB | + +The three largest rows look like candidates for sealed codec changes. +They are not. The sealed codec already writes each one compactly, and the +decoder expands each one back to its full domain shape: + +- **Per-chunk revisions and identities.** A file segment + (`PersistedFileChunksRefV2` in `sealed_codec.rs`) writes the generation + and file occurrence once per file and replaces both with markers + (`$tracedecay:g`, `$tracedecay:f`) before compression. It hoists the + language descriptor, chunker, sanitizer, and sensitivity revisions into + `chunk_defaults`. It omits a chunk id that the chunk-id recipe derives, + and it omits a content digest that the stored text determines. It stores + the source text the chunks cover once per file, and rows keep only spans. + In memory, every `CodeSearchChunkV1` owns its own `String` for six of + these values: `anchor.generation_id`, `anchor.file_occurrence_id`, + `language_descriptor_revision`, `chunker_revision`, `sanitizer_revision`, + and `sensitivity.policy_revision`. The domain newtypes from + `validated_string_newtype!` wrap `String`, so a clone is a new heap + allocation. +- **Projection request and receipts.** `projection_rows.rs` writes the + request's added chunks as `(file key, first position, count)` runs. It + writes only the receipt decisions that differ from what the request + implies. In memory, the request's `added_or_changed` holds one + `ChangedCodeChunkV1` per chunk. Each row repeats the chunk id and content + digest that the chunk record already owns. Receipt rows are no longer held. + `ProjectionPublicationHandoffV1` replays them from the request one row at a + time, so the remaining projection cost is the request rows. +- **Per-file records.** `clone_rows.rs` interns each file's syntax kinds and + token texts and writes token streams as integer codes. It omits payload + digests and the occurrence fields that the file authority fixes. In + memory, each `CodeIndexCloneBodyV1` holds expanded token streams, its + digests, and a `CloneBodyOccurrenceV1` that repeats project, repository, + generation, and path. Each file page also keeps `document.chunk_ids`, + which repeats the id of every chunk the page already holds. + +Shrinking the sealed bytes would not move the 3.24 GB. The work is in the +decoded representation. + +## Cold construction no longer holds the decoded generation + +The table above describes builds that kept the whole decoded generation +until publication. Cold construction now writes file segments to the store +preparation as each extraction window completes. It charges what it retains +to `ProcessResidentMemoryV1` under the `code_index.construction.v1` +component, through `ResidentMemoryOperationV1::reserve_process_shared`. A +refused build parks until its witnessed reconstruction demand fits, so a +concurrent second cold pass waits instead of stacking past the limit. +Serving uses the published generation's decode cache and recorded admission +demand. + +The same isolated cold pass on the release daemon now measures as follows. +Each value is one run. `footprint` reports the physical footprint peak, and +`tracedecay status --json` reports `memory.resident_bytes` and +`memory.reserved_bytes`. + +| Repository | Physical footprint peak | Peak occupancy | Peak construction charge | +| --- | ---: | ---: | ---: | +| This repository (6.1k files) | 1.58 GB | 1.58 GB | 0.92 GB | +| `t3code` (29k files) | 2.45 GB | 2.53 GB | 0.96 GB | + +During construction, occupancy stays below the reservations. The peak in +both runs comes after publication, while the interactive graph catalog +warms. `PendingInteractiveCatalogWarmV1::run` takes no reservation, so +occupancy reached 2.53 GB on `t3code` while 268 MB was reserved. The catalog +then stays resident as the 833 MB `graph_catalog` owner. That warm is a +separate admission gap and is outside this design. + +A decoded generation retains its full `retained_bytes`. When a decode peak +measurement exists, admission reserves the larger of +`decode_peak_growth_bytes` and `retained_bytes` under +`code-index-generation-decode-v1`. Without a peak measurement, admission uses +the retained bytes. A first active-generation decode with no recorded demand +currently takes no reservation. The `decoded_generation` owner reports +retained residency while the decode is held. The proposals below target both +retained residency and transient decode demand. They do not change these +admission rules or close the first-unmeasured-decode gap. + +## Proposed decoded encoding + +Apply the changes in this order. Measure after each one, and stop when the +remaining size no longer justifies the next change. + +1. **Share per-generation identity and revision strings.** Change the + identity and revision newtypes that a generation repeats per record to + wrap `Arc`. These are `CodeGenerationId`, `FileOccurrenceId`, + `LanguageDescriptorRevision`, `ChunkerRevision`, `SanitizerRevision`, and + `PolicyRevisionId`. The decoder and the builder intern each distinct value + once per generation and clone the `Arc` into every record. Each newtype + serializes through `as_str()`, so wire bytes and canonical digests do not + change. Explicit string serialization requires no additional serde feature + changes. The per-chunk cost of the six values drops from six + heap allocations to six reference-count increments. Hoisting the fields + out of `CodeSearchChunkV1` would save the 16-byte pointers too, but it + changes a domain contract that every chunk consumer reads. Do that only if + measurement after interning shows the pointers matter. +2. **Keep the projection request in its sealed shape.** Hold the request's + added chunks in memory as the same `(file key, first position, count)` + runs that `projection_rows.rs` writes. Keep explicit rows only for changed + and deleted chunks. Resolve each run against the file's chunk roster when + the projector, the digest, or receipt replay iterates the rows. Compute + the request digest by serializing the resolved rows in the same order. + `ProjectionPublicationHandoffV1::receipt_rows` already serializes receipt + rows this way without collecting them, so the digest stays + byte-identical. A cold build adds every chunk, so its + request shrinks to one run per file. +3. **Derive the document chunk roster.** Drop `document.chunk_ids` from the + decoded file page and derive the roster from the page's chunk records in + their existing file-row order. The document roster must match those rows + positionally. The chunker orders rows by source span and grain, with chunk + ID only as a tie-breaker. The separate projection roster uses chunk-id + order. The sealed segment already omits the document roster. +4. **Keep clone token streams interned.** Hold each file's interned string + table and integer token codes in memory, as `clone_rows.rs` writes them. + Rebuild the occurrence from the file authority when a caller asks for it. + The readers of `conservative_tokens` and `rename_tokens` in `clones.rs` + take a `CloneTokenStreamV1`, so they must read codes through the file's + table. It is the largest change of the four. Measure the share + of the 1.08 GB that clone bodies hold before you start it. + +The issue measurement does not split the 1.15 GB and 1.08 GB rows by +field. Before any change lands, measure the split with the method in +[Measure the win](#measure-the-win). Then order the work by the bytes each +change removes. + +## Keep shipped generations readable + +Beta builds have shipped, so live profiles hold sealed generations at +revision 21. All four changes alter only the decoded representation. Each +change must keep the sealed bytes identical for the same input. Then a +shipped generation decodes into the new representation, and a new build +writes the same segment and manifest digests as an old build. Segment reuse +across the upgrade keeps working, so the format revision stays at 21 and +there is no migration. + +Each change must include a test that seals one fixture and asserts the +segment and manifest digests equal those of the unchanged codec. The test +fails if the in-memory change leaks into the wire form. + +If a later change must alter the sealed bytes, bump +`SEALED_GENERATION_FORMAT_REVISION_V1` and rebuild from source. Do not write +a migration. The restore path already refuses an older revision with +`SupersededSealedGenerationRevision`, and the scheduler then rebuilds the +generation from the worktree. A sealed generation is a cache that the source +tree derives. A migration would need a reader for every shipped revision, +and `sealed_codec.rs` deliberately keeps no reader for older revisions. The +cost of a rebuild is one cold pass per worktree. Construction admission +bounds that pass, so concurrent rebuilds after an upgrade wait instead of +exceeding the limit. + +## Measure the win + +Use the task-supplied checkout and the release daemon in an isolated temporary +profile. Do not create a clone or another project-data copy. The earlier +figures are historical measurements, not instructions to preserve that setup. + +1. Build the release CLI with + `bazel build --config=release //crates/tracedecay-cli:tracedecay`. +2. Start the daemon with `TRACEDECAY_DATA_DIR` and `HOME` inside the temporary + profile, then run `tracedecay init `. +3. While the cold pass runs, sample `tracedecay status --json `. + Record `memory.resident_bytes`, `memory.reserved_bytes`, and + `memory.reservations`. +4. Run a query that decodes the generation, then record the + `decoded_generation` bytes in `memory.owners`. That value is the + generation's `retained_bytes`. Record the measured decode peak separately + and compare admission against the larger demand. +5. Record the on-disk size of the project's `code-index-v1/` store. The size + must not change. A change in size means a decoded change reached the + sealed bytes. + +Stop and join the isolated daemon before removing its temporary profile. +Do not read or reset the operator's active profile. + +For the per-part split, sum the `resident_bytes.rs` helpers per part: +`chunk_bytes` with each identity and revision string summed separately, +`file_bytes` with clone bodies, schema evidence, and `document.chunk_ids` +summed separately, and the projection request rows. Emit the sums as fields +of one `tracing` event at seal time. A trace event is the operator log +surface, so the event can stay after the work lands. + +Report the served generation's bytes and the peak `resident_bytes` before +and after each change, with the median of three cold passes per side.