Performance: read paths fully deserialize all event payloads eagerly on every GetFrom* call

[AGiorgetti]

Summary

Every GetFrom* read path calls doc.ToCommit(_serializer), which fully deserializes the entire commit and all event payloads — even when the caller only needs iteration or metadata.

What ToCommit does on every document

• Deserializes the whole MongoCommit object from BsonDocument.
• Iterates every stored event.
• Deserializes each EventMessage payload.
• Materializes the event set into an array.

Why this matters

• Global checkpoint scans pay full payload materialization cost even when the caller only needs a cursor over commits.
• This cost grows with event count and payload size.
• Allocation pressure will likely dominate long sequential reads (especially for large event counts per commit).
• This is the primary bottleneck for read-heavy workloads.

Baseline (net10, 2026-05-20)

Benchmark	Mean
ReadFromEventStoreAsync(CommitsToWrite=10000)	120.443 ms
ReadFromStream(CommitsToWrite=10000)	148.004 ms

Proposed investigation

• Benchmark small vs large payloads.
• Benchmark 1 / 100 / 1000 events per commit to isolate per-event cost.
• Compare global reads vs per-stream reads to isolate the effect.
• Measure allocation pressure via BenchmarkDotNet MemoryDiagnoser.
• Evaluate lazy/deferred deserialization for the global checkpoint scan path.

Reference

See docs/Performance-Investigation.md → Finding #2 for full context.

[AGiorgetti]

We ran the serializer experiment behind this issue and the result was not strong enough to keep it.

What we tried:

• Introduced an alternate "optimized" commit-envelope serializer that bypassed the normal MongoCommit driver-registry path.
• Kept event payload serialization on the existing IDocumentSerializer path.
• Benchmarked the same read scenarios with both implementations.

What we learned:

• The optimized path reduced allocations somewhat, especially in the heavy case.
• It did not produce a meaningful throughput improvement on the read path.
• In the small case it was actually a bit slower/noisier, and in the large case the latency difference was effectively negligible.
• A follow-up attempt to remove the sync LINQ projection also did not move the benchmark in a useful way.

Conclusion:

• The gain was not large enough to justify the added code complexity, extra serializer mode, additional tests/docs, and the maintenance burden of keeping two envelope implementations in sync.
• We reverted the experiment and kept the original design.

So the issue remains valid from a profiling standpoint, but the attempted fix was the wrong tradeoff. If we revisit this area, the next step should be a simpler, more targeted investigation into the real hot cost inside read-time event payload materialization rather than adding another serialization abstraction.

[AGiorgetti]

We'll implement minor safe optimization refactorings to avoid some Linq allocations and use LoggerMessage for logging.