Skip to content

Show each custom-alert rule's last-fired time on the rule cards (#3360) - #3370

Merged
erikdarlingdata merged 1 commit into
devfrom
feature/3360-custom-alert-last-fired
Sep 12, 2026
Merged

Show each custom-alert rule's last-fired time on the rule cards (#3360)#3370
erikdarlingdata merged 1 commit into
devfrom
feature/3360-custom-alert-last-fired

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What

Part of #3285. The Alert Rules list cards (wwwroot/js/pages/alert-rules.js) showed name, description, metric/threshold/scope chips, and a version line — but not when the rule last fired. This adds a per-rule "last fired …" / "never fired" line to each card, fed by a bounded backend read folded into the existing list summary.

The correlation (the load-bearing part)

A custom-alert fire writes a config.config_alert_log row with metric_name = 'Custom:<rule_id>' (the immutable id — CustomAlertEvaluator.MetricNameFor). A resolve / teardown writes a different metric_name: '<display name> Resolved' (DeliverResolveAsync / WriteTeardownResolutionAsync).

CustomAlertRuleStore.ListSql now LEFT JOINs each rule to MAX(alert_time) over a grouped config_alert_log subquery, keyed on the exact fire key 'Custom:' || id:

  • The LIKE 'Custom:%' is only a pre-filter to shrink the grouped set; the = 'Custom:' || id join key is what actually correlates. A resolve row '<name> Resolved' can never equal 'Custom:<numeric id>', so a resolve is never counted as a fire, and one rule's fires never bleed into another's. Even a contrived 'Custom:<id> Resolved' row (which passes the LIKE) does not join.
  • Every fire delivery writes a Custom:<id> row regardless of mute / channel (the deliver path records the incident even when muted), so MAX(alert_time) means "the last time the rule's condition fired (the incident was recorded)" — the intended meaning. The read is deliberately not filtered on alert_sent / notification_type.
  • One round-trip: the grouped subquery scans the fire rows once and joins to the whole rule set — never a per-rule query. config_alert_log's index leads with server_id, so this deliberately cross-server aggregate scans rather than seeks; that is acceptable on this cold, operator-triggered list render over the retention-bounded history table (the per-(server, metric) cooldown seeds still seek the index).

last_fired (ISO-8601 UTC, naive like updated_at, or null) flows through CustomAlertRuleSummary and BuildRuleSummariesNode, so both GET /api/alerts and the MCP list_custom_alert_rules tool get it — they share both the summary type and the node builder, so it is a clean parity add (the MCP tool's description is updated to match).

Frontend

alert-rules.js renders "Last fired <relTime>" (reusing the shared util.js relTime helper) or "Never fired", with a small CSS touch so a fired rule reads in --fg (draws the eye to active rules) and "never fired" is dimmed.

Tests

  • Live (CustomAlertLastFiredLiveTests, gated on DARLING_TEST_PG): fire rows count (MAX), the natural '<name> Resolved' row is excluded, an adversarial 'Custom:<id> Resolved' row is excluded (proving the exact join, not the LIKE, correlates), cross-rule no-bleed, never-fired → null, all resolved in one ListAsync.
  • Pure (DarlingWebEndpointsTests): the last_fired wire mapping (a value + JSON null when never fired), plus the existing summary-shape test updated for the new field.

Validated locally against a throwaway PostgreSQL 18.4 + TimescaleDB cluster (live test green; full custom-alert live family green). Service build: 0 Warning(s) / 0 Error(s).

Not touched

CHANGELOG.md (folded post-wave), other lanes, the predicate/scope model.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WejwpgWF5Xfm3fAmoEbFz4

The Alert Rules list cards showed name, description, metric/threshold/scope
chips, and a version line, but not when the rule last fired. Add a per-rule
"last fired ..." / "never fired" to the card, fed by a bounded backend read
folded into the existing list summary.

Backend: CustomAlertRuleStore.ListSql LEFT JOINs each rule to MAX(alert_time)
over the config_alert_log rows keyed on the rule's IMMUTABLE fire key
'Custom:<id>' (CustomAlertEvaluator.MetricNameFor). The correlation is that
exact fire key only: a resolve/teardown row carries a different metric_name
('<name> Resolved'), which can never equal 'Custom:<numeric id>', so a resolve
is never counted as a fire and one rule's fires never bleed into another's.
The LIKE 'Custom:%' is only a pre-filter; the '= Custom:<id>' join key is what
correlates, so even a 'Custom:<id> Resolved' row (which passes the LIKE) does
not join. Every fire delivery writes a 'Custom:<id>' row regardless of
mute/channel, so MAX means "the last time the rule's condition fired" and the
read is deliberately not filtered on alert_sent/notification_type. It is one
round-trip: the grouped subquery scans the fire rows once and joins the whole
rule set (never a per-rule query). last_fired flows through CustomAlertRuleSummary
and BuildRuleSummariesNode, so the /api/alerts list AND the MCP
list_custom_alert_rules tool (which share both types) get it for free.

Frontend: alert-rules.js renders "Last fired <relTime>" (reusing util.js's
shared relTime helper) / "Never fired", styled to draw the eye to active rules.

Tests: a live correlation test (fire counts, natural + adversarial resolve rows
excluded, cross-rule no-bleed, never-fired null, one-read shape) and a pure
wire-shape test for the last_fired field (value + JSON null). Validated against
a local PostgreSQL 18.4 + TimescaleDB throwaway cluster.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WejwpgWF5Xfm3fAmoEbFz4
@claude

claude Bot commented Sep 12, 2026

Copy link
Copy Markdown

Reviewed the diff (Darling-only: CustomAlertRuleStore.cs, DarlingWebEndpoints.cs, the MCP tool description, alert-rules.js, editor.css, plus live + pure tests).

Correctness — the join logic checks out. ListSql's LEFT JOIN correlates on exact equality (f.metric_name = 'Custom:' || r.id), not the LIKE 'Custom:%' pre-filter, so:

  • A resolve row ('<name> Resolved', written by DeliverResolveAsync/WriteTeardownResolutionAsync) never collides, because every write path on that route always appends the " Resolved" suffix to the sanitized name — there's no code path that writes a bare metric_name equal to a rule's display name.
  • A same-length adversarial row ('Custom:<id> Resolved') passes the LIKE but correctly fails the exact-equality join — and this is explicitly covered by the new live test.
  • r.id is bigint; 'Custom:' || r.id relies on Postgres's implicit text || anynonarray concat, which is valid and already exercised by the live test.
  • Column ordinals in ListAsync (reader.GetDateTime(7) for the new last_fired) line up correctly with the new SELECT list.
  • CustomAlertRuleSummary is a positional record with a new trailing LastFired member; the only non-test construction site (CustomAlertRuleStore.ListAsync) was updated, and both test call sites use named arguments, so no silent positional-arg mismatch risk.

Parity — this looked like it might need a Lite twin at first glance (CONTRIBUTING.md's Two-Store Parity section), but Lite.Tests/CrossAppMcpToolInventoryPinTests.cs already documents list_custom_alert_rules and the rest of the custom-alert-rule tool family as Darling-only by architecture (central-store feature; Lite has no twin store to port to), so this is correctly Darling-only and there's no parity gap.

Frontendalert-rules.js continues the el()/text discipline (no innerHTML), and relTime()/parseUtc() are reused as-is, consistent with how updated_at is already rendered, so the naive-UTC timestamp handling isn't a new risk.

Minor, non-blocking note: the new grouped subquery filters config_alert_log on metric_name LIKE 'Custom:%' without a server_id predicate, so (per the doc comment) it's a full scan of the whole cross-server, retention-bounded table on every list read rather than an index seek. The PR description/docstring already calls this out explicitly as an accepted tradeoff for a cold, operator-triggered read, and the page isn't polled — just flagging it in case the retention window or fleet size grows to where that stops being true.

No blocking issues found.

@erikdarlingdata
erikdarlingdata merged commit 8b3496b into dev Sep 12, 2026
8 checks passed
@erikdarlingdata
erikdarlingdata deleted the feature/3360-custom-alert-last-fired branch September 12, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant