Add resumable cursors to NDJSON terminals

TESTING_GUIDE.md

@@ -672,6 +672,8 @@ Candidate-ordered parallel-index recovery tests must prove that the fatal result
   uses one two-file recipe fixture to prove a compact search can emit a negative-score `next_cursor`, replay that exact cursor as a separated `--cursor` value, and return the next distinct page without an option-parsing error.
 - `QueryCommandRunnerTests.RunSearch_CursorDoesNotConsumeRecognizedShortOption_Issue4664`
   keeps the negative-cursor exception narrow by proving a recognized short option after `--cursor` remains an option and still produces the missing-cursor-value diagnostic.
+- `QueryCommandRunnerIssue5230Tests`
+  walks cursor-capable `search`, `symbols`, and `files` NDJSON across multiple pages without gaps or duplicates; verifies final and zero-result terminals omit continuations; rejects filter-mismatched and stale-generation cursors; keeps byte-capped output within budget while advancing both the envelope and nested terminal continuation by the actual emitted count (using `no_result_row_emitted` instead of an unsafe zero-progress cursor if no row fits); and fails closed for row selectors, a generation change during materialization, and page sizes whose next replay would cross the 10,000-row pagination boundary.
 - `QueryCommandRunnerTests.RunSearch_UnknownRecipeQuerySuggestsOnlyFromActiveRecipe_Issue4862`, `RunSearch_RecipeQuerySuggestionHandlesZeroOneAndManyQueries_Issue4862`, and `RunSearch_ExternalRecipeAliasesResolveAndSuggestCanonicalActiveQueries_Issue4862`
   keep human/JSON usage errors, canonical names, current/deprecated aliases, deterministic typo correction, active-recipe isolation, normalized filter preservation, and shell-safe replay quoting in one recipe-selection contract. Suggestions and replay selectors must remain canonical names from the active recipe even when an alias is the nearest match.
   Include/exclude typo recovery must preserve selector semantics, unresolved repeated selectors, and raw FTS mode. Missing close matches and recipe-name corrections with child selectors must not offer a replay that broadens the selection, while aliases that collide with canonical names or multiple query owners must be removed with bounded diagnostics.
@@ -1829,6 +1831,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
   は 2 file の recipe fixture を 1 つ使い、compact search が負の score で始まる `next_cursor` を生成し、その値を separated `--cursor` として変更せず再利用しても option parse error にならず、別の次ページを返すことを検証します。
 - `QueryCommandRunnerTests.RunSearch_CursorDoesNotConsumeRecognizedShortOption_Issue4664`
   は負の cursor に対する例外を狭く保ち、`--cursor` の直後にある既知の short option は cursor 値として消費されず、従来どおり cursor 値欠如の診断を返すことを検証します。
+- `QueryCommandRunnerIssue5230Tests`
+  は cursor 対応の `search`、`symbols`、`files` NDJSON を複数 page にわたって gap / duplicate なしで走査し、最終 terminal と 0 件 terminal が continuation を省略すること、filter mismatch と stale generation の cursor を拒否すること、byte-cap 出力が budget 内に収まり envelope と内包 terminal の continuation がともに実際の出力件数だけ進むこと（row が 1 件も収まらない場合は unsafe な zero-progress cursor の代わりに `no_result_row_emitted` を返すこと）、row selector、materialize 中の generation 変更、次回の同一 page size で 10,000 row の pagination 境界を越える場合に fail closed することを検証します。
 - `QueryCommandRunnerTests.RunSearch_UnknownRecipeQuerySuggestsOnlyFromActiveRecipe_Issue4862`、`RunSearch_RecipeQuerySuggestionHandlesZeroOneAndManyQueries_Issue4862`、`RunSearch_ExternalRecipeAliasesResolveAndSuggestCanonicalActiveQueries_Issue4862`
   は human / JSON usage error、canonical 名、現行 / deprecated alias、決定的な typo correction、active recipe への限定、正規化済み filter の保持、shell-safe な replay quoting を1つの recipe-selection 契約として維持します。alias が最も近い候補でも、suggestion と replay selector は active recipe の canonical 名でなければなりません。
   include / exclude の typo recovery は selector の意味、未解決の繰り返し selector、raw FTS mode を保持します。近い候補がない場合、および child selector を伴う recipe 名訂正では selection を広げる replay を提示せず、canonical 名または複数の query owner と衝突する alias は上限付き diagnostic とともに除去しなければなりません。

USER_GUIDE.md

@@ -401,6 +401,20 @@ array. `languages --json` accepts `--limit` / `--top`, `--cursor`, and
 the ordinary unbounded JSON shape. Pass each `next_cursor` back to the same
 command and filters. A cursor is bound to that selection and index generation,
 so changed inputs or a refreshed index require restarting the pagination.
+The default raw NDJSON streams for `search`, `symbols`, and `files` use the same
+opaque `response:v2` continuation contract. When a terminal record has
+`has_more: true` and at least one result row was emitted, its `next_cursor`
+resumes after the last emitted row; replay it with the unchanged command,
+query, filters, ordering, and page limit. The replay is returned in the shared
+bounded envelope, whose `metadata.stream_terminal` mirrors the continuation.
+Final and zero-result pages do not advertise a cursor. If a partial stream
+cannot make safe progress—for example, a byte cap leaves room only for the
+terminal record, the query uses row selectors or a recipe/named search, the
+10,000-row pagination window is exhausted or the unchanged page limit would
+cross it on replay, or the index generation changes while rows are being
+read—the terminal omits the cursor and reports
+`next_cursor_unavailable_reason`. Cursor bytes are included in the complete
+`--max-json-bytes` measurement.
 When a bounded `find --all` scan exits partially, its terminal record includes
 `next_cursor`; replaying it resumes after the last scanned line.
 The bounded-response commands `search`, `definition`, `find`, `status`,
@@ -4159,7 +4173,20 @@ row を省略した場合は、`omitted_match_count`、`truncated`、`has_more`
 受け付け、これらを指定した場合だけ bounded envelope を選択するため、通常の上限なし
 JSON 形状は変わりません。`next_cursor` は同じ command と filter に渡してください。
 cursor はその選択条件と index generation に束縛されるため、入力変更後または index
-更新後は pagination を最初からやり直す必要があります。上限に達した
+更新後は pagination を最初からやり直す必要があります。
+`search`、`symbols`、`files` の既定 raw NDJSON stream も同じ opaque な
+`response:v2` continuation 契約を使います。terminal record が `has_more: true` で、
+result row を 1 件以上出力した場合、`next_cursor` は最後に出力した row の次から再開します。
+command、query、filter、ordering、page limit を変えずに再利用してください。再開応答は
+共有 bounded envelope となり、`metadata.stream_terminal` にも同じ continuation が
+反映されます。最終 page と 0 件 page は cursor を公開しません。byte cap により terminal
+record しか出力できない場合、row selector または recipe / named search を使う場合、
+10,000 row の pagination window を使い切ったか、同じ page limit での再開時にその上限を
+越える場合、row の読み取り中に index generation が変わった場合のように、安全に再開できない
+partial stream では cursor を省略し、terminal の
+`next_cursor_unavailable_reason` で理由を報告します。cursor の byte 数も
+`--max-json-bytes` による stream 全体の計測に含まれます。
+上限に達した
 `find --all` scan が partial exit した場合、terminal record の `next_cursor` を
 再利用すると最後に scan した line の次から継続します。
 bounded-response command の `search`、`definition`、`find`、`status`、

changelog.d/unreleased/5230.fixed.md

@@ -0,0 +1,25 @@
+---
+category: fixed
+issues:
+  - 5230
+affected:
+  - src/CodeIndex/Cli/QueryCommandRunner.cs
+  - src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs
+  - src/CodeIndex/Cli/QueryCommandRunner.Database.cs
+  - src/CodeIndex/Cli/JsonOutputContracts.cs
+  - src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs
+  - src/CodeIndex/Cli/QueryCommandRunner.Ndjson.cs
+  - src/CodeIndex/Cli/QueryCommandRunner.SearchResults.cs
+  - tests/CodeIndex.Tests/QueryCommandRunnerIssue5230Tests.cs
+  - USER_GUIDE.md
+  - DEVELOPER_GUIDE.md
+  - TESTING_GUIDE.md
+---
+
+## English
+
+- **Truncated discovery NDJSON now exposes resumable cursors (#5230)** — Raw `search`, `symbols`, and `files` terminal records include a stable, generation-bound `next_cursor` after the last emitted row by reusing the response-v2 contract from #4730. Continuation rejects changed filters and stale indexes, advances both byte-trimmed envelope cursor copies by the actual emitted row count, omits cursors on final and zero-result pages, and reports a machine-readable reason when selectors, the next unchanged page would exceed the pagination window, or a concurrent index-generation change makes safe progress impossible.
+
+## 日本語
+
+- **切り詰められた discovery NDJSON が再開用 cursor を公開するようになりました (#5230)** — raw `search`、`symbols`、`files` の terminal record は、#4730 の response-v2 契約を再利用し、最後に出力した row の後から再開できる安定した generation-bound `next_cursor` を含みます。continuation は filter 変更と stale index を拒否し、byte trimming 後の envelope 内にある両方の cursor を実際に出力した row 数だけ進めます。最終 page と 0 件 page では cursor を省略し、selector、次回の同一 page が pagination window を越える場合、同時発生した index-generation 変更により安全に進められない場合は機械可読な理由を報告します。

src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs

@@ -668,7 +668,7 @@ JsonObject BuildCandidate(
             metadata["cursor_offset"] = controls.Offset;
             metadata["page_limit"] = controls.PageLimit;
             metadata["has_more"] = hasMore;
-            metadata["next_cursor"] = selectedScanCursor
+            var nextCursor = selectedScanCursor
                 ?? (hasMore && count > 0
                     ? FormatResponseCursor(
                         nextOffset,
@@ -680,6 +680,15 @@ JsonObject BuildCandidate(
                         controls.ResumeMatchOrdinal,
                         controls.ResumeByteOffset)
                     : null);
+            metadata["next_cursor"] = nextCursor;
+            if (scanCursor is not null
+                && metadata["stream_terminal"] is JsonObject adjustedTerminal)
+            {
+                if (nextCursor is null)
+                    adjustedTerminal.Remove("next_cursor");
+                else
+                    adjustedTerminal["next_cursor"] = nextCursor;
+            }
             metadata["truncated"] = scanCursor is not null || totalCount > count || byteLimitOmittedPathCount > 0;
             metadata["pagination_window_limit"] = MaxPageWindow;
             metadata["pagination_window_exhausted"] = paginationWindowExhausted;
@@ -2620,6 +2629,67 @@ private static ResponseSnapshot BuildResponseSnapshot(DbReader reader)
             reader.GetIndexedHeadForResponse());
     }
 
+    internal sealed record NdjsonResponseCursorContext(
+        Func<int, string>? CursorFactory,
+        string? UnavailableReason);
+
+    internal static string CaptureResponseGenerationFingerprint(DbReader reader)
+        => BuildResponseSnapshot(reader).GenerationFingerprint;
+
+    private static string? TryCaptureResponseGenerationFingerprint(string dbPath)
+    {
+        try
+        {
+            using var db = new DbContext(DbOpenIntent.QueryOnly, dbPath);
+            if (!db.TryValidateIsCodeIndexDb(out _))
+                return null;
+            return CaptureResponseGenerationFingerprint(new DbReader(db));
+        }
+        catch
+        {
+            return null;
+        }
+    }
+
+    internal static NdjsonResponseCursorContext BuildNdjsonResponseCursorFactory(
+        string command,
+        string[] args,
+        string dbPath,
+        string? expectedGenerationFingerprint)
+    {
+        if (expectedGenerationFingerprint is null)
+            return new(null, "index_generation_unavailable");
+        var currentGenerationFingerprint = TryCaptureResponseGenerationFingerprint(dbPath);
+        if (currentGenerationFingerprint is null)
+            return new(null, "index_generation_unavailable");
+        if (!string.Equals(
+                expectedGenerationFingerprint,
+                currentGenerationFingerprint,
+                StringComparison.Ordinal))
+        {
+            return new(null, "index_generation_changed_during_query");
+        }
+
+        var responseOffset = GetBoundedResponseOffset(command);
+        var queryFingerprint = BuildResponseFingerprint(command, args);
+        return new(
+            returnedCount => FormatResponseCursor(
+                checked(responseOffset + returnedCount),
+                queryFingerprint,
+                expectedGenerationFingerprint),
+            null);
+    }
+
+    internal static bool IsNdjsonResponseCursorWithinWindow(
+        string command,
+        int returnedCount,
+        int replayPageLimit)
+    {
+        var nextOffset = checked(GetBoundedResponseOffset(command) + returnedCount);
+        return nextOffset < MaxPageWindow
+               && checked(nextOffset + replayPageLimit) <= MaxPageWindow;
+    }
+
     private static ResponseSnapshot BuildFallbackResponseSnapshot(string appVersion)
         => new(BuildResponseValueFingerprint("catalog\0" + appVersion), null, null);
 

src/CodeIndex/Cli/JsonOutputContracts.cs

@@ -829,6 +829,10 @@ internal sealed record JsonStreamDoneResult(
     [property: JsonPropertyName("truncated")] bool Truncated,
     [property: JsonPropertyName("has_more")] bool HasMore,
     [property: JsonPropertyName("total_count_authoritative")] bool TotalCountAuthoritative,
+    [property: JsonPropertyName("next_cursor")]
+    [property: JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] string? NextCursor = null,
+    [property: JsonPropertyName("next_cursor_unavailable_reason")]
+    [property: JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] string? NextCursorUnavailableReason = null,
     [property: JsonPropertyName("total_count_lower_bound")]
     [property: JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] int? TotalCountLowerBound = null,
     [property: JsonPropertyName("selection_reason")]

src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs

@@ -250,7 +250,7 @@ internal QueryCommandOptions Parse(string[] args)
                     resolvedDbPath = DbContext.ToReadOnlyUri(resolvedDbPath);
             }
 
-            return BuildOptions(dbResolution, resolvedDbPath);
+            return BuildOptions(dbResolution, resolvedDbPath, args);
         }
 
         private void NormalizeOutputMode()
@@ -407,10 +407,14 @@ private void ValidateEnvironmentDefaults()
                 AddParseError(defaultMaxLineWidthError);
         }
 
-        private QueryCommandOptions BuildOptions(DbPathResolution dbResolution, string resolvedDbPath)
+        private QueryCommandOptions BuildOptions(
+            DbPathResolution dbResolution,
+            string resolvedDbPath,
+            string[] invocationArgs)
         {
             return new QueryCommandOptions
             {
+                InvocationArgs = [.. invocationArgs],
                 DbPath = resolvedDbPath,
                 DbPathExplicit = dbPathExplicit,
                 ReadOnly = readOnly,

src/CodeIndex/Cli/QueryCommandRunner.Database.cs

@@ -90,6 +90,11 @@ private static int WithDb(
             }
 
             reader.IncludeGenerated = options.IncludeGenerated;
+            options.InvocationGenerationFingerprint = options.Json
+                                                      && options.JsonOutputFormat == JsonOutputFormatNdjson
+                                                      && !options.ResultsOnly
+                ? JsonEnvelopeWrapper.CaptureResponseGenerationFingerprint(reader)
+                : null;
             var previousProjectRoot = s_activeQueryProjectRoot;
             var projectRootResolution = ResolveProjectFilterRoot(dbPath, options.DbPathExplicit);
             s_activeQueryProjectRoot = projectRootResolution.Root;