Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -540,6 +540,39 @@ AgentExperience.NET is a **preview**. It is not production ready, and public API
ships, in which the MCP SDK lifts two library floors (DI abstractions 10.0.13, `Microsoft.Extensions.AI.Abstractions`
10.11.0) inside the tool only. Guide: [CLI and MCP](docs/guide/cli-and-mcp.md).

### One living lesson per situation (story 28.6)

- **`ExperienceConsolidationService.ConsolidateAsync(authorization, scope, options, ct)`** (Core, registered by the
one-call setup; `provider.GetAgentExperienceLessons().ConsolidateAsync(identity)` from the lessons view) merges the
validated records of one scope that failed the same way and were fixed the same way into one consolidated record.
Only records the library finalized and can vouch for are sources: `Origin == Finalized`, a verifying provenance
signature when signing is on, verified, with a library-written, confirmed lesson; never a hand-written record, never
another scope's or a grant's. The grouping key is exact: the set of resolved failure signature keys (story 28.2) and
the working attempt's tool sequence, plus the values of the argument keys `ExperienceConsolidationOptions.ArgumentKeys`
names (the recipe's by default from the lessons view), so `retry(delay=0)` and `retry(delay=30)` stay two lessons. A
group merges once it holds records from `MinimumSources` (3) distinct runs. Deterministic and explicit: no model,
nothing runs it automatically.
- **The consolidated record** renders as its most recent source does (its content is copied from that source only),
with the lesson `Verified in N independent runs (latest yyyy-MM-dd). Failed first with <class> from <tool> in K of
them. Worked: …` (template version 1). Its counters are verified evidence only, deduplicated by independence key: one
validation per source, plus the `Verified` updates in the sources' histories; host-trusted evidence and raised
counters are never carried. `ReadConfidenceAsync(VerifiedOnly)` counts them as verified when signing is on and the
record's signature (over the producer and the source list) verifies. Its ID is derived from its source IDs, so a
re-run is idempotent and a later verified run is folded in by a new record that supersedes the old one. It is marked
in its provenance (`ExperienceConsolidation.ProvenanceSource`; `Provenance.ExposedTo` lists its sources, a signed
claim), so no store, schema, port or contract changes.
- **Its sources are superseded** through the ordinary lifecycle: re-read right before the record is created (a changed
one is left out), then one compare-and-set commit each; a source that changed in between is left untouched and
reported. Until then the record and its sources coexist: `ExperienceConsolidationGroupResult.Complete` and the
result's `IncompleteGroups` say so, and the next pass finishes. A record already at the derived ID is adopted only when
its signature, producer, sources, lesson and starting counters match. Groups beyond 256 sources, or whose records kept
changing, are reported `Deferred`.
- **Erasure does not reach a copy**: `FindConsolidatedContainingAsync` (service and lessons view) finds the consolidated
records that list a source, to erase with it; the next pass leaves an erased source out. Contesting or revoking a
consolidated record works as for any record; its sources stay superseded. The lessons view shows
`ExperienceLesson.ConsolidatedFrom` ("consolidated from N" on the HTML page), and the benefit estimate treats a
consolidated lesson as a new lesson: holdout rows that named its sources stay as they are.

## 0.1.0-preview.17

This preview measures whether injected lessons help. An opt-in holdout withholds the block on a deterministic share of
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,9 @@ stored (`ListAsync(identity)` with your own identity in production): status, con
whether retrieval would use it and why not, and the block an agent there would be shown for it.
`ExperienceLessonsHtml.Render(page)` turns that into one self-contained HTML page. The library maps no endpoint; serve
it behind your own authorization, because it discloses the lessons. See
[Seeing what it learned](docs/guide/deployment.md#seeing-what-it-learned).
[Seeing what it learned](docs/guide/deployment.md#seeing-what-it-learned). When many runs learned the same thing,
`ConsolidateAsync(identity)` on the same view merges them into one lesson that says how many runs confirmed it (see
[Consolidating repeated lessons](docs/guide/lifecycle.md#consolidating-repeated-lessons)).

## Choosing a reflector

Expand Down
9 changes: 9 additions & 0 deletions docs/guide/deletion-and-retention.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ the KL-2 boundary in [Known limits and documented boundaries](../known-limits.md

## Deleting a record

> **Consolidated lessons copy content.** A [consolidated lesson](lifecycle.md#consolidating-repeated-lessons) carries a
> copy of its most recent source's task, attempts, outcome and reflection, and lists every source's ID. Erasing a
> source does **not** reach a consolidated record made from it. Before erasing a record, call
> `FindConsolidatedContainingAsync` (on `ExperienceConsolidationService`, or `GetAgentExperienceLessons()` with the
> identity) and erase the records it returns too; the next consolidation pass leaves the erased source out.

```csharp
var store = new PostgresExperienceRecordStore(dataSource);

Expand Down Expand Up @@ -353,6 +359,9 @@ below shrink to the derived search data, and that page says exactly what remains
- **Exported telemetry.** Spans and metrics this library emitted carry record IDs; erasing a record does not
retract them. The erasure's own telemetry adds nothing that was erased (see [below](#telemetry)).
- **External artifacts a record merely named.** Tickets, logs, commits: the library never held them.
- **Consolidated records made from the erased one.** A consolidated record copies its latest source's content and
lists every source's ID; erase it too (`FindConsolidatedContainingAsync` finds it, see
[above](#deleting-a-record)).
- **The dead heap tuple — until `VACUUM`, the erased text is still in this database.** The tombstone is written
with an `UPDATE`, and an `UPDATE` in PostgreSQL writes a new row version and leaves the old one in the heap.
Until `VACUUM` reclaims it, the previous version of the record row still carries the task summary, the lesson,
Expand Down
5 changes: 4 additions & 1 deletion docs/guide/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,10 @@ each lesson, newest first, it gives:
page's `InjectionsTracked` is `true`); and not tracked (`InjectionsTracked` is `false`: a store that does not track
injections, or a PostgreSQL schema whose application role has not been granted `0026`'s table yet), which the HTML
shows as "not tracked" rather than "never". .NET, Python and Go agents all record their injections (the ports from
[contract 1.4](../../contract/README-1.4.md)), so in a mixed fleet "never" means no agent in the scope was shown it.
[contract 1.4](../../contract/README-1.4.md)), so in a mixed fleet "never" means no agent in the scope was shown it;
- `ConsolidatedFrom`, for a lesson a consolidation pass merged (`ConsolidateAsync(identity)` on the same view): how
many source records it was merged from, each now superseded by it. See
[Consolidating repeated lessons](lifecycle.md#consolidating-repeated-lessons).

`ExperienceLessonsQuery` narrows it: `Statuses`, `Limit` (the store's bounds, 1 to 500, default 50) and
`IncludeIneligible` (default `true`). The page's `HasMore` says whether more lessons match than it shows (or may: the
Expand Down
104 changes: 104 additions & 0 deletions docs/guide/lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,110 @@ cannot abort on a pre-`0006` `Superseded` event that has no replacement — one
store never applied Core's table. New and updated rows are checked from that moment on. The script's header carries
the reconciliation query and the `VALIDATE CONSTRAINT` statements to run once it comes back empty.

## Consolidating repeated lessons

Every verified run becomes its own record, so fifty runs that hit the same failure and fixed it the same way would be
fifty lessons, each starting at confidence 2/3 and competing for the block's slots. A consolidation pass turns them
into one lesson that says how many runs confirmed it. It is explicit and deterministic: nothing runs it automatically,
no model is involved, and it is never on the request path.

```csharp
// The one-call setup: the lessons view runs it in the identity's own scope.
var result = await provider.GetAgentExperienceLessons().ConsolidateAsync(identity);

// Or Core directly, with your own options.
var result = await consolidation.ConsolidateAsync(
authorization,
scope,
new ExperienceConsolidationOptions { MinimumSources = 3, ArgumentKeys = new Dictionary<string, IReadOnlyList<string>> { ["retry"] = ["delay"] } },
cancellationToken);
```

**Which records are sources.** Only records the library finalized and can still vouch for: in exactly the scope
asked for (never another scope's, never one a sharing grant lends), `Validated` or `Reinforced`, verified,
`Origin == Finalized`, and, when [provenance signing](confidence.md#signing-provenance) is on, carrying a signature that
verifies (or listed in `TrustUnsignedRecordIds`), with a library-written lesson whose content is confirmed. That is the
rule confidence verification applies to a run's record. A record written through the store port by hand is never a
source, whatever it claims. The pass reads at most 500 such records, newest first (`MayHaveMore` says when there were
more).

**What is one situation.** Two sources are one situation when they hold the same set of *resolved*
[failure signatures](injection.md#recall-at-the-moment-of-failure) (the exact `ExperienceFailureSignature.Key`) and
their working attempt (the final one, ended without an error: the attempt the block's `Worked:` line names) called the
same tools in the same order, with equal values for the argument keys `ArgumentKeys` names. Name the keys your blocks
show (the lessons view defaults to the keys `options.Recipe` made visible), so `retry(delay=0)` and `retry(delay=30)`
stay two lessons. A group is merged once it holds at least two records from `MinimumSources` (default 3) distinct runs.

**The consolidated record.** A new record whose ID is derived from its source IDs
(`ExperienceConsolidation.ConsolidatedIdFor`), so re-running a pass finds the record it made. Its task, summary,
attempts, outcome, environment and reflection fields are its most recent source's (nothing is copied from the older
ones), so its `Tried:` and `Worked:` lines and recipe render as that source's did. Its lesson reads `Verified in N
independent runs (latest 2026-10-07). Failed first with TimeoutException from run_tests in N of them. Worked: run_tests
→ run_tests.` (template version 1). It is marked in its provenance, so no store, schema, port or contract changes:
`Provenance.Source` is `AgentExperience.Consolidation`, its reflection's producer `AgentExperience.Consolidation/1`
(signed content), and `Provenance.ExposedTo` lists its source records (at most 256) at the revision the pass read them.
`ExperienceConsolidation.IsConsolidated` and `SourcesOf` read it back. Its `Origin` stays `HostWritten` and its
`SourceRunId` is a derived identifier no run has: it vouches for no run. With provenance signing it is signed like a
finalized record, and since the signed claims cover `ExposedTo`, its source list cannot be changed without the
signature failing.

**Its counters are verified evidence only**, deduplicated by
[independence key](confidence.md#independence-is-keyed-and-the-database-owns-the-key) rather than summed by record: each
source's own validation, never more than the one finalization sets, under its run and closed round; and every counted
update admitted `Verified` in each source's history and in each merged consolidated record's own. Host-trusted evidence
and evidence with no recorded admission are never carried, and neither is any counter someone raised in the store. So a
run that produced one source and was also counted as reuse of another counts once. The record is created at the score
those counters explain, as finalization creates every record; your confidence engine scores it from its first evidence
on. Reading it back: `ReadConfidenceAsync` with `ExcludeHostTrusted` reports what is stored. With `VerifiedOnly`, a
consolidated record counts its initial counters as verified only when signing is configured and its signature verifies
(it then covers the producer and the source list), capped at one per listed source; without signing it is treated like
any record written without finalization, and its initial counters are left out.

**Supersession.** Right before the record is created, the pass reads its sources again and leaves out any that changed
since it first read them (the group is re-evaluated; if they keep changing the group is reported `Deferred`). The
record is created as a `Candidate` and validated, and then each source is moved to `Superseded` by it through
`CommitAsync`, one commit per source, at the revision and status the pass read. A source that changed in between (a
contradiction, a revocation, a reinforcement) is left untouched and reported with its `StaleRevision` or
`StatusMismatch`; each commit is the store's own transaction, so a source is either superseded or untouched, never
half. Each source's history ends with the supersession that names the consolidated record and the producer
`AgentExperience.Consolidation/1`.

**The window where both exist.** Between the record's validation and the last supersession, the consolidated record and
its sources are all eligible, and retrieval can return both. A pass that stops in that window (a crash, a refused
commit, a source that changed) leaves them coexisting until the next pass: its group's `Complete` is `false` and the
result's `IncompleteGroups` counts it. Every event ID and timestamp is derived, so an interrupted pass, or two passes at
once, converge on the same record, and the next pass finishes the supersessions (`ExperienceConsolidationGroupOutcome.Existing`).
A record already stored under the derived ID is adopted only when it is a consolidated record this library would
accept as a source (producer, signature, confirmed content) over the same sources, with the same lesson and the same
starting counters the pass computed; otherwise the group is `Refused` and nothing is superseded.

**Reversal and later runs.** A consolidated record is an ordinary record: contest or revoke it like any other. Its
sources stay superseded (the transition table never moves a superseded record back), so withdrawing a consolidated
lesson withdraws the situation's lesson. A later verified run in the same situation is finalized into an ordinary
record; the next pass folds it in, creating a record over the larger source set (a new derived ID) that supersedes the
old consolidated record and the new one. A source the old record lists is carried over only when it is still a trusted
source of the same situation that the old record superseded: one revoked or erased since is left out of the list, the
run count and the counters.

**Erasure does not reach a copy.** A consolidated record copies its most recent source's content and lists every
source's ID, so erasing a source leaves that copy in place. Before erasing a record, call
`FindConsolidatedContainingAsync` (on the service, or on `GetAgentExperienceLessons()` with the identity), which
follows the source's supersession chain and scans the scope's 500 most recent records, and erase what it returns too
(see [Deletion and retention](deletion-and-retention.md#deleting-a-record)).

**Limits and deferral.** A consolidated record lists at most 256 sources. A group with more takes the consolidated
records' sources and the newest records that fit, and reports the rest in `Deferred`; once a consolidated record lists
256 sources, further runs of the same situation stay their own records, and their group is reported with the outcome
`Deferred`.

**Measurement.** Exposure rows and holdout assignments that named a source stay as they are. The
[benefit estimate](injection.md#measuring-benefit-the-holdout) is per lesson, so a consolidated record is a new lesson
with no measurements of its own yet. The lessons view shows a consolidated lesson's `ConsolidatedFrom` ("consolidated
from N" on the HTML page).

**Not done here.** The Python and Go ports read, render and move consolidated records like any other, but do not run a
pass.

## In Python and Go

Since contract 1.16 ([what it adds](../../contract/README-1.16.md)) both ports move a stored record through this
Expand Down
13 changes: 13 additions & 0 deletions src/AgentExperience.Core/Confidence/ProvenanceSigning.cs
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,19 @@ private static byte[] Guarded(Func<byte[]> encode)
}
}

/// <summary>
/// One tool call argument value's canonical encoding, exactly as the content digest encodes it: so two values that
/// sign alike (<c>1.5</c> and <c>1.50</c>, an enum and its name) encode alike here too. Consolidation keys an
/// allowlisted argument by it.
/// </summary>
/// <exception cref="ArgumentException">The value has no JSON form, or is nested too deeply.</exception>
internal static byte[] EncodeArgumentValue(object? value) => Guarded(() =>
{
using var buffer = new MemoryStream();
WriteValue(buffer, ToJson(value), depth: 0);
return buffer.ToArray();
});

private static byte[] EncodeClaims(ExperienceRecord record)
{
using var buffer = new MemoryStream();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ public static class AgentExperienceCoreServiceCollectionExtensions
{
/// <summary>
/// Registers the sanitizer, the in-memory capture service, the default reflector, the lifecycle
/// service, and the finalization service as singletons.
/// service, the finalization service and the consolidation service (it runs only when the host
/// calls it) as singletons.
/// </summary>
/// <remarks>
/// <para>
Expand Down Expand Up @@ -160,6 +161,12 @@ internal static IServiceCollection AddAgentExperienceCoreWithIndexing(
provider.GetService<ExperienceFinalizationOptions>(),
provider.GetService<IExperienceRunStore>()));

// Story 28.6: on demand only; nothing resolves or runs it unless the host does.
services.TryAddSingleton(provider => new ExperienceConsolidationService(
provider.GetRequiredService<IExperienceRecordStore>(),
provider.GetRequiredService<ExperienceLifecycleService>(),
provider.GetService<TimeProvider>()));

return services;
}

Expand Down
Loading
Loading