feat: add Deleted counter to usage metrics#2253
Draft
niemyjski wants to merge 1 commit into
Draft
Conversation
Comment on lines
+1480
to
+1483
| catch (Exception ex) | ||
| { | ||
| _logger.LogWarning(ex, "Failed to increment deleted usage metrics"); | ||
| } |
- Add Deleted property to UsageInfo and UsageHourInfo models - Add EventsDeleted counter to AppDiagnostics - Add IncrementDeletedAsync to UsageService with cache key, save, and GetUsageAsync support - Update EventController.DeleteModelsAsync to track deleted event counts - Update ResetProjectDataWorkItemHandler to track deleted events on project reset - Update Svelte org/project usage pages to chart Deleted series - Update Angular org/project manage controllers to chart Deleted series - Update generated TypeScript API types - Add comprehensive UsageService tests for deleted metrics - Add EventController integration tests for delete usage tracking Does not change plan limits, billing, or admission control. Does not instrument retention cleanup, bot cleanup, or orphan cleanup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
77a4c37 to
bbd89a4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Until now there was no way to see how many events were explicitly deleted via the API, project reset, or soft-delete cleanup. This makes it impossible to distinguish "events never arrived" from "events were intentionally removed" when analyzing usage. This PR adds a first-class
Deletedcounter alongside the existingTotal,Blocked,Discarded, andTooBigcounters -- without touching billing, plan limits, or admission control.What changed
Backend
UsageInfo/UsageHourInfo-- addedlong Deletedproperty to both records (backwards-compatible as ES stores it as a numeric type).AppDiagnostics-- addedEventsDeletedas aCounter<long>for OTel metrics.UsageService-- newIncrementDeletedAsync(organizationId, projectId, eventCount)that pipelines all Redis operations viaTask.WhenAll; cache key pattern:usage:{bucket}:{orgId}[:projectId]:deleted. Save/flush/read logic updated to include the Deleted bucket (mirrors the other counters exactly).EventController-- injectsUsageService;DeleteModelsAsyncgroups loaded events once (singleGroupBy), callsbase.DeleteModelsAsync, then callsIncrementDeletedAsyncper org/project group inside a try/catch so a tracking failure never blocks the delete response.ResetProjectDataWorkItemHandler-- callsIncrementDeletedAsyncwith the raw event count returned by the delete query (no truncation cast).CleanupDataJob-- injectsUsageService;RemoveProjectsAsyncinstruments deleted;RemoveStacksAsyncaccepts atrackDeletedUsageflag (true for soft-delete cleanup, false for retention enforcement) with proportional per-project distribution usinglongarithmetic.RemoveOrganizationAsyncintentionally does NOT track -- the org is being hard-deleted so the data has no consumer.Frontend (Svelte 5)
api.ts,schemas.ts) updated to includedeleted: number.Deletedseries added to charts, dead variable alias removed.Frontend (Angular legacy)
Deletedseries added at the correct index.Test coverage
UsageServiceTestsEventControllerTestsCleanupDataJobTestsResetProjectDataWorkItemHandlerTestsExisting tests updated to assert
Deleted == 0where appropriate.Key design decisions
RemoveStacksAsyncis called withtrackDeletedUsage=falsefor retention enforcement paths. Only explicit user-initiated deletes and soft-delete cleanup show up asDeletedusage.longnotint. High-volume deployments can delete millions of events;intwould overflow. All cache reads useGetAsync<long>.RemoveStacksAsyncuses purelongarithmetic; the last group absorbs any remainder to prevent event count loss.Deletedis a display-only counter. It does not affectGetEventsLeftAsync, plan enforcement, or Stripe metering.