Skip to content

Add a first-class audit --all mode with bounded cross-recipe summaries #5238

Description

@Widthdom

Summary

There is no first-class way to execute every registered audit recipe.

A repository-wide audit currently requires:

  1. running cdidx recipes --names;
  2. translating all returned names into separate commands;
  3. building JSONL input for cdidx batch;
  4. manually merging per-recipe results, truncation, and failures.

The current registry contains 20 built-in recipes, making repeatable full-repository audits unnecessarily difficult.

Proposed behavior

Add:

cdidx audit --all

It should discover recipes from the authoritative registry used by recipes --names, execute them in deterministic order, and return a bounded aggregate result.

Existing issue relationship

Related history:

Neither provides a first-class all-registered-recipes workflow.

PR scope and implementation guidance

Keep this PR to orchestration and aggregate output contracts. Do not change individual recipe matching rules.

  • Never hard-code a second recipe-name list.
  • Include selected recipe names/count in structured output so omissions are auditable.
  • Define deterministic behavior for aggregate/composite recipes.
  • Keep per-recipe counts separate so overlap is not presented as a misleading unique total.
  • If cross-recipe deduplication is supported, use a stable finding identity and retain all recipe/query attribution.
  • Preserve path, language, test, origin, snippet, and other applicable search filters.
  • Define --total-limit as a global emission budget and retain documented per-query --limit semantics.
  • Continue other recipes after one fails, retain successful results, report bounded per-recipe errors, and use a documented partial/failure exit policy.
  • Reuse internal recipe execution APIs; do not spawn shells or recursively invoke the CLI.
  • Apply row, byte, cancellation, and time budgets during accumulation rather than after unbounded materialization.
  • Keep recipes --names plus batch as a supported lower-level workflow.

Structured output requirements

Summary/count/compact output should include at least:

  • selected recipe count and deterministic names;
  • completed, failed, partial, and omitted recipe counts;
  • bounded per-recipe status and match counts;
  • authoritative/approximate/truncated count facts;
  • global row/byte-limit metadata;
  • recovery guidance for failed, omitted, or truncated recipes.

Row-oriented output must retain recipe and child-query attribution.

Acceptance criteria

  • audit --all executes the authoritative registered set without a hand-authored batch.
  • Adding/removing a registry entry automatically changes the selected set.
  • Output order is deterministic.
  • Filters and global limits apply consistently.
  • Overlapping matches are not silently described as unique findings.
  • One recipe failure cannot erase successful results from other recipes.
  • Machine output reports partial execution and the documented exit code.
  • Existing single-recipe audit behavior remains unchanged.

Tests, documentation, and changelog

  • Add registry-consistency, deterministic-ordering, filtering, row/byte-limit, partial-failure, and cancellation tests.
  • Include overlapping findings across recipes.
  • Add a representative bounded-allocation/performance test for aggregate execution.
  • Run the complete net8.0 and net9.0 test suites.
  • Update CLI help, README examples, completion metadata, and full-repository audit workflow guidance.
  • Add the required bilingual changelog.d/unreleased/ fragment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    .NETPull requests that update .NET codedocumentationImprovements or additions to documentationenhancementNew feature or requestperformancetests

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions