Skip to content

fix(resource): atomic single-shot deprovision on DELETE — close double-deprovision TOCTOU#236

Merged
mastermanas805 merged 7 commits into
masterfrom
fix/resource-delete-atomic-deprovision-guard-2026-06-04
Jun 3, 2026
Merged

fix(resource): atomic single-shot deprovision on DELETE — close double-deprovision TOCTOU#236
mastermanas805 merged 7 commits into
masterfrom
fix/resource-delete-atomic-deprovision-guard-2026-06-04

Conversation

@mastermanas805

Copy link
Copy Markdown
Member

What (design-review follow-up to #229)

A multi-role review panel found a real TOCTOU in the merged idempotent-DELETE fix (#229). It used a resource.Status == "deleted" pre-read before SoftDeleteResource + deprovision — check-then-act: two concurrent DELETEs both read status='active', both pass the pre-read, and both fire the destructive Deprovision RPC against the same backend. That's the truehomie-db DROP incident class (an active customer DB torn down by a path that re-ran teardown). My #229 test was sequential, so it missed the concurrent case.

Fix

Make the soft-delete itself the authoritative atomic guard:

// SoftDeleteResourceIfActive
UPDATE resources SET status = 'deleted' WHERE id = $1 AND status != 'deleted'  // returns RowsAffected

The handler deprovisions only when this call won the transition (deleted==true). Any racer or sequential retry gets deleted==false → idempotent {"ok":true,"already_deleted":true} with no second deprovision. The redundant status pre-read is removed (the rows-affected check subsumes it and is race-safe). SoftDeleteResource is unchanged for the ~18 provision-rollback callers (always a just-created row).

Coverage

Symptom:        two concurrent DELETEs both fire destructive deprovision (truehomie class)
Enumeration:    resource.Delete handler + SoftDeleteResource callers (rg, 18 rollback sites untouched)
Sites found:    1 (the DELETE handler)
Sites touched:  1 handler + 1 new guarded model fn
Coverage test:  models.TestSoftDeleteResourceIfActive_Branches (won/no-op/error, 100%); existing
                TestResourceDelete_Idempotent_DoubleDelete now hits the rows-affected !deleted branch
Live verified:  go test vs real Postgres+Redis; all changed handler lines covered (count≥1)

Follow-up (tracked for the integration suite): a 16-way concurrent -race handler test asserting the provisioner receives exactly one Deprovision RPC.

🤖 Generated with Claude Code

…e-deprovision TOCTOU

Design-review follow-up to #229. The idempotent-DELETE fix used a
`resource.Status == "deleted"` PRE-READ before SoftDeleteResource +
deprovision — a check-then-act TOCTOU: two concurrent DELETEs both read
status='active', both pass the pre-read, and BOTH fire the destructive
provisioner Deprovision / storage Deprovision RPC against the same backend.
That is the truehomie-db DROP incident class (an active customer DB dropped
via a path that re-ran teardown).

Fix: make the soft-delete the authoritative atomic guard. New
models.SoftDeleteResourceIfActive does `UPDATE ... SET status='deleted'
WHERE id=$1 AND status != 'deleted'` and returns RowsAffected>0. The handler
deprovisions ONLY when this call won the transition (deleted==true); any
racer or sequential retry gets deleted==false → idempotent
{"ok":true,"already_deleted":true} with NO second deprovision. Removes the
redundant pre-read (the rows-affected check subsumes it and is race-safe).
SoftDeleteResource is kept unchanged for the ~18 provision-rollback callers
that always operate on a just-created row.

Tests: models.TestSoftDeleteResourceIfActive_Branches (sqlmock: won / no-op /
error — 100% of the new fn). The existing TestResourceDelete_Idempotent_
DoubleDelete now exercises the rows-affected !deleted branch via a sequential
retry (200 already_deleted, no re-deprovision). All changed handler lines
covered. Verified vs real Postgres+Redis.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mastermanas805 mastermanas805 enabled auto-merge (squash) June 3, 2026 20:12
@mastermanas805 mastermanas805 merged commit be32484 into master Jun 3, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant