Repository navigation
Add ADR052 on storing analytics in S3 Tables - #263
Merged
Merged
Conversation
theseanything
commented
Sep 8, 2026
stephencdaly
reviewed
Sep 8, 2026
theseanything
force-pushed
the
ADR-business-analytics-storage-options
branch
from
September 22, 2026 10:54
268c86c to
f75b3f2
Compare
theseanything
changed the title
We want to measure submissions over time and the number of live forms, keeping the data for at least two years. Existing tools (OTel and Classic CloudWatch metrics, GA4, Splunk, Logs Insights, the runner database) fall short on accuracy, cost, cardinality, retention or ownership. S3 Tables queried with Athena was prototyped and is very cheap, but relies on unfamiliar services, at-least-once delivery and slow queries. A separate Aurora PostgreSQL cluster is familiar, gives exactly-once writes and low-latency queries, and can later serve per-form analytics in forms-admin. This record accepts Aurora and sets out the trade-offs.
theseanything
force-pushed
the
ADR-business-analytics-storage-options
branch
from
September 22, 2026 10:57
f75b3f2 to
62858f1
Compare
cadmiumcat
reviewed
Sep 22, 2026
Aurora's always-on cost and maintenance is too much for a few small tables behind an internal dashboard. S3 Tables is cheap, fully managed and already prototyped. Also records storing metrics in the forms-admin database as a rejected option, following PR discussion.
theseanything
changed the title
stephencdaly
approved these changes
Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We want to measure submissions over time and the number of live forms, keeping the data for at least two years.
This proposes storing those metrics in S3 Tables and querying them with Athena. It is very cheap, fully managed and already prototyped, with events taken from the structured logs the applications already emit.
Existing tools (OTel and Classic CloudWatch metrics, GA4, Splunk, Logs Insights, the runner database) fall short on accuracy, cost, cardinality, retention or ownership. A separate Aurora cluster and the forms-admin database were also considered, but are more cost, effort or maintenance than the current requirement needs.