Skip to content

orb_signals grows unbounded, and adding retention would silently corrupt two existing surfaces #9783

Description

@JSONbored

The gap

orb_signals is absent from RETENTION_POLICY (src/db/retention.ts:16-70), while both its siblings have rules: orb_pr_outcomes at 90 days and orb_webhook_events at 14. Every self-host instance exports into it hourly (src/selfhost/orb-collector.ts), so it grows without bound.

It is also a public data source, and has been since before the weekly fleet trend: computeFleetAnalytics reads it for the fleetAccuracy headline on /fairness (src/orb/analytics.ts:266-283), and #9775 added a second reader for the weekly series.

Why this is not a one-line addition

Adding { table: "orb_signals", column: "received_at", days: 90 } looks obvious and would silently change two figures that are already published:

1. listFleetInstances counts lifetime signals. src/orb/fleet-admin.ts:19-27:

(SELECT COUNT(*) FROM orb_signals s WHERE s.instance_id = i.instance_id) AS signalCount

That is an unbounded cumulative count, presented in the owner-facing fleet view as "every instance that has ingested signals". Pruning turns it into "signals in the retention window" with no change of label — the reader has no way to tell.

2. /v1/internal/fleet/analytics?days= takes an arbitrary window. src/api/routes.ts:4686-4689 defaults to 90 but accepts anything, and computeFleetAnalytics filters received_at >= cutoff from that parameter. With a 90-day prune, ?days=365 would return 90 days of data while reporting a 365-day window — the denominator and the stated window disagreeing, which is exactly the class of defect #9676 / #9718 / #9768 have been correcting on the public surface.

This is the same shape as #9474: orb_pr_outcomes could not simply be pruned either, because it feeds a cumulative public counter. That was solved by folding expiring rows into orb_outcome_rollups before deletion, with an explicit ordering constraint documented at retention.ts:17-23. orb_signals needs the same kind of thought, not a copied line.

Options

A. Prune at 90 days, and make both readers honest. Add the rule; change signalCount to a windowed count with a label saying so; clamp the analytics days parameter to the retention window (or return the effective window alongside the result so a caller can see it was capped). Smallest change that keeps every published number true.

B. Prune at 90 days, preserving the cumulative count via a rollup. Mirror #9474: fold expiring rows into a durable per-instance total before deleting, so signalCount stays genuinely lifetime. More work, and it only pays off if that count is actually load-bearing for anyone.

C. Leave it unbounded, and say so. Document that orb_signals is retained indefinitely by design because two surfaces depend on unbounded history. Honest, but the table only grows, and it is the one ingest path open to any instance before a human registers it (src/orb/ingest.ts stores a stranger's signals; registration only gates whether they count).

I lean A: the cumulative signalCount reads like a diagnostic rather than a number anyone depends on, and B's rollup is real machinery to preserve it. But C is defensible if fleet history is considered worth keeping whole — in which case the growth should at least be a known, stated choice rather than an omission.

Note on scope

Whichever is chosen, the analytics days clamp is worth doing regardless, because the window/denominator mismatch it allows is already reachable today for any days value larger than the data actually retained upstream.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions