Skip to content

feat(cache): skip a fill that started before an eviction - #233

Draft
lucataco wants to merge 1 commit into
mainfrom
lucataco/cache-generation-fencing
Draft

lucataco wants to merge 1 commit into
mainfrom
lucataco/cache-generation-fencing

Conversation

@lucataco

@lucataco lucataco commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #232, which is merged. This PR is one commit on main.

Why

After #232, a fill that began before the source reported a key gone can finish after the eviction and write the old value back. The value lasts until the next refresh, at most the fresh TTL, because that refresh evicts it again.

What changed

  • A random generation per eviction. Each eviction replaces the key's generation, cache:gen:<name>:<key>, with a random value in the same script that deletes the data and freshness keys.
  • Compare before writing. A fill reads the generation before it fetches and writes through a compare-and-set script only if the generation is unchanged. A shadow write compares against the shadow's own generation.
  • Limits. A fill older than 2 minutes is not written. Generations expire after 4 minutes, which outlives any fill that could still be written.
  • Observability. A skipped fill sets cache.fill_discarded on its span to generation_changed, generation_read_failed, or max_age.

Cost

  • Every fill on every cache changes. Each fill adds one GET per backend, and the write becomes an EVAL instead of a MULTI pipeline.
  • Each eviction runs an EVAL per backend.
  • Each evicted key leaves one generation key. It has a 26-byte value and a 4-minute TTL.
  • Redis topology. The scripts touch several keys of one entry, so they need standalone or Sentinel Redis, not Redis Cluster.

Keep this a draft until the extra Redis work per fill has been measured. Then decide whether closing the late-fill window is worth that cost.

Verification

  • Tests that failed before their fixes:
    • a late fill after an eviction
    • a late fill after the generation key expires
    • the generation TTL bound
    • the discard span attribute
  • The tests from fix(cache): stop serving a value after the source reports it gone #232 are kept.
  • Package checks: go test -race -shuffle=on ./cache passes, and golangci-lint v2.1.5 reports no issues.
  • End to end against the API service with this code, on a local Compose stack: after the source reported a key gone, the API never served the key again. Redis showed a random generation with a TTL just under 4 minutes.

Base automatically changed from lucataco/cache-evict-missing to main November 3, 2026 16:14
A fill that began before the source reported a key gone can finish after the
eviction and write the old value back. That value lasts until the next refresh.

Each eviction now writes a random generation for the key. A fill reads the
generation before it fetches and writes through a compare-and-set script only
if the generation is unchanged. Shadow writes compare against the shadow's own
generation. Fills older than two minutes are not written, and generations
expire after four minutes. Skipped fills set cache.fill_discarded on the span.
@lucataco
lucataco force-pushed the lucataco/cache-generation-fencing branch from 5b3e0f9 to e6a5f74 Compare November 3, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant