Skip to content

feat: add webhook coverage gate observability - #232

Merged
ecarreras merged 3 commits into
mainfrom
feat/webhooks-coverage-gate
Oct 3, 2026
Merged

ecarreras merged 3 commits into
mainfrom
feat/webhooks-coverage-gate

Conversation

@giscebot

@giscebot giscebot commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • add measurable cross-source coverage with explicit both, imap_only, webhook_only, and IMAP-eligible denominator
  • expose mean matching delay and an admin-only bounded exception queue
  • show coverage and actionable exceptions in the webhook dashboard
  • preserve payload minimization: no raw webhook or comment bodies are returned

Why

Canary rollout needs an evidence-based gate. The previous dashboard showed match totals but not the denominator or the events requiring investigation.

Validation

  • pytest -q — 392 passed on the complete stack
  • dashboard npm test -- --run — 60 passed
  • dashboard npm run build — passed

Rollback

This PR is observability-only. Reverting it removes the new aggregate/exception views without changing ingestion or dispatch.

Requested by: @ecarreras

Refs #191

TASK-85718

Co-authored-by: Eduard Carreras <ecarreras@gisce.net>
@giscebot
giscebot added this pull request to stack #235 October 3, 2026 10:22
@giscebot
giscebot requested a review from ecarreras October 3, 2026 10:23
@giscebot giscebot self-assigned this Oct 3, 2026
@giscebot giscebot changed the title feat/webhooks coverage gate feat: add webhook coverage gate observability Oct 3, 2026
giscebot and others added 2 commits October 3, 2026 15:36
Co-authored-by: Eduard Carreras <ecarreras@gisce.net>
Co-authored-by: Eduard Carreras <ecarreras@gisce.net>
@ecarreras
ecarreras merged commit d7df4a2 into main Oct 3, 2026
3 checks passed
@ecarreras
ecarreras deleted the feat/webhooks-coverage-gate branch October 3, 2026 13:58
@giscebot

giscebot commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

Post-merge blocker before #233: the coverage population is not comparable.

webhook_shadow_receipts is pruned to webhook_retention_days, but the new summary counts every historical ingest_receipts email row with a non-null event_key. Those IMAP receipts are not pruned. It also counts source-specific fallbacks such as email:<message-id> as imap_eligible, even though by definition they cannot match a webhook canonical key.

A minimal SQL reproduction with one current event present in both sources, one old IMAP receipt, and one current email:<message-id> fallback returns:

both=1, imap_eligible=3, ratio=33%
imap_only=[old canonical event, email fallback]

The comparable retained event has 100% coverage. As implemented, the ratio will drift downward with unrelated/old IMAP history and the exception queue will contain entries that no webhook could ever resolve, so this cannot safely gate the canary rollout.

Before merging #233, I recommend using one explicit comparison window (bounded by webhook retention and ideally by the shadow-rollout start), restricting eligibility to event families/actions with canonical identities shared by both transports, and applying exactly the same predicate to /summary and /exceptions. The exception query should also honor the planned grace period so normal delivery skew is not reported as actionable. Regression tests should prove that old IMAP rows and email:* fallbacks do not change the ratio or appear as imap_only, while a current unmatched canonical event appears only after the grace window.

The existing CI is green, but the added tests only cover an empty IMAP denominator and therefore do not exercise this failure mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants