Skip to content

fix(sleep): thread-safe backend cache + redact exports - #251

Open
WODE25500 wants to merge 5 commits into
microsoft:mainfrom
WODE25500:fix/sleep-hardening
Open

fix(sleep): thread-safe backend cache + redact exports#251
WODE25500 wants to merge 5 commits into
microsoft:mainfrom
WODE25500:fix/sleep-hardening

Conversation

@WODE25500

Copy link
Copy Markdown
Contributor

Sleep-cycle hardening.

  • Guard CliBackend _cache/_tokens with a lock on the opt-in parallel replay path (SKILLOPT_SLEEP_WORKERS>1); the model call stays outside the lock so parallel workers still overlap.
  • Redact report.json before staging (redact_secrets).
  • Redact harvest --output and --json exports (_redact_deep).
  • Add tests for CliBackend caching/thread-safety.

- Guard CliBackend _cache/_tokens with a lock on the opt-in parallel replay
  path (SKILLOPT_SLEEP_WORKERS>1) so a concurrent miss cannot corrupt state
  or lose the token cost metric; the model call stays outside the lock so
  parallel workers still overlap.
- Redact report.json (redact_secrets) before staging.
- Redact harvest --output and --json exports (_redact_deep).
- Add tests for CliBackend caching/thread-safety.
@Yif-Yang

Copy link
Copy Markdown
Contributor

report.json is now correctly passed through the mapping-aware redactor, but the export-redaction and thread-safety fixes are still incomplete in three places.

  1. Harvest output uses _redact_deep(payload), while _redact_deep() only recurses into values and loses the mapping-key context. As a result, _redact_deep({"api_key": "plain-secret", "nested": {"token": "other"}}) returns both secrets unchanged. Please use the existing mapping-key-aware redact_secrets(payload) at every structured output boundary.
  2. write_staging() redacts report.json but writes caller-provided report_md verbatim. Edit content/rationale can therefore still expose credentials. Please redact the Markdown before writing it as well.
  3. Not all cache/token access uses the new lock: the Pi and OpenCode overrides still inspect and pop _cache directly, and tokens_used() reads _tokens without the lock. More importantly, parallel replay_one() attributes one call's tokens from a shared global before/after total, so overlapping workers can charge another worker's tokens to the wrong result. Please route cache/token access through locked helpers, prevent a failed caller from deleting another caller's successful cache entry, and use call-local accounting for per-result tokens.

Please add boundary-level tests for both harvest output/file and report.md, including nested api_key/token mappings. The concurrency tests should use barriers/events to force overlapping misses and assert exact call/cache/token outcomes, including a concurrent Pi/OpenCode empty-result versus successful-result case. The current immediate echo test does not guarantee overlap and can pass without exercising these races.

Address maintainer review on microsoft#251:
- Use redact_secrets (mapping-key aware) instead of _redact_deep for harvest
  --output/--json and handoff exports, so nested api_key/token mappings are
  redacted (not just bare string leaves).
- Redact report_md before writing it alongside report.json.
- Add boundary tests (nested api_key/token + report.md).
Address maintainer review on microsoft#251 (thread-safety):
- Add _cache_get/_cache_pop/_cache_pop_if locked helpers and route the Pi and
  OpenCode _cached_call overrides through them (they previously read/pop the
  cache outside the lock).
- tokens_used() now reads _tokens under the lock.
- Popping a failed entry is conditional (_cache_pop_if): a failed caller only
  drops its own empty value, never another worker's just-stored success.
- Add tests: barrier-forced overlapping misses stay consistent, and pop-if
  does not delete a successful entry.
Address maintainer review on microsoft#251 (last thread-safety item):
- Record each model call's token delta on the calling thread (thread-local),
  so parallel replay_one() charges its own cost instead of a before/after
  global total that an overlapping worker inflates.
- replay_one reads backend.token_delta() (falling back to the text-length
  heuristic for backends that don't track tokens).
- Add tests for call-local and thread-isolated token deltas.
Address maintainer review on microsoft#251 (deepen thread-safety):
- _cached_call no longer caches empty (transient-failure) results and prefers a
  concurrently cached success, so an empty/duplicate cannot clobber or delete
  another worker's successful entry.
- Add a Pi subclass-level concurrency test (barrier-forced empty-vs-success)
  asserting the success survives.
@WODE25500

WODE25500 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

Yifan Yang (@Yif-Yang) — thank you for the thorough review and guidance! I've addressed the feedback:

  • Structured outputs now use the mapping-key-aware redact_secrets (not _redact_deep, which loses the key context); report_md is also redacted before writing.
  • Thread-safety hardened: cache/token access goes through locked helpers (_cache_get/_cache_pop/_cache_pop_if); tokens_used() is locked; a failed caller only conditionally pops its own empty value (_cache_pop_if), never another worker's success; the base no longer caches empty (transient-failure) results.
  • Switched to call-local token accounting (thread-local delta) so parallel replay no longer misattributes tokens through a global before/after total.
  • Added tests: nested api_key/token redaction, report.md, barrier-forced concurrency, pop-if not deleting a success, Pi subclass empty-vs-success, and thread-isolated token deltas — all pass.

Thanks again for the detailed review!

@WODE25500

WODE25500 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

Yifan Yang (@Yif-Yang) — thank you for the careful review and guidance. In fact I've done almost all of these submissions through DSH, which is exactly why I have a bold idea. Across your recent PR reviews I noticed that you consistently apply a set of quality baselines: fail-closed handling, structure-aware and boundary-consistent redaction, thread-safety with call-local accounting, validating against real contracts, PR hygiene, and so on. I'd like to distill that into a reusable review-standards: a general quality standard plus a pre-PR self-check CLI that contributors run before submitting, so they spend less time on rework and you receive fewer low-quality PRs — a win-win. I'd credit you as the original source of these standards. Looking forward to hearing from you whenever you get a chance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants