feat: cleanup orphan script (CM-904)

[ulemons]

Add Temporal Scripts for Orphaned Segment Aggregates Cleanup

Summary

Added automated cleanup scripts to remove orphaned segment aggregates that reference deleted members and organizations.

Changes

• New Temporal Workflows: Created cleanupMemberSegmentsAgg and cleanupOrganizationSegmentAgg workflows to identify and delete orphaned records in memberSegmentAggs and organizationSegmentAggs tables
• Scheduled Execution: Both scripts run every 5 minutes via Temporal schedules
• Batch Processing: Processes records in configurable batches (default 500) to prevent database overload
• Logging: Comprehensive logging for monitoring cleanup operations and tracking deleted records

Technical Details

• Workflows query for segment aggregates where the referenced member/organization no longer exists
• Uses BUFFER_ONE overlap policy to prevent concurrent executions
• Includes retry logic with exponential backoff for reliability
• Runs on the script-executor task queue

---

Note

Medium Risk
Automates recurring deletions in production tables and adds new scheduled Temporal workflows; risk is mainly around query correctness/performance and unintended data removal if orphan detection is wrong.

Overview
Adds two new Temporal cleanup workflows to delete orphan rows in memberSegmentsAgg and organizationSegmentsAgg (records whose referenced member/org no longer exists), processing in batches with chunked deletes and continueAsNew until drained.

Introduces DAL helpers to query/delete these orphans plus an orphanCleanupRuns tracking table API, and wires new activities/exports and Temporal schedules (daily at midnight) so the worker automatically runs and records progress/status.

^{Written by Cursor Bugbot for commit 374cb58. This will update automatically on new commits. Configure here.}

[skwowet]

humm, probably not a good idea to pass the table name like this - we usually use an interface and reference it where needed

[ulemons]

ok here maybe the name can be confusing, this will not query a different table, it will just add the table name in a field of the orphans table so that we know it came from memberAggs or orgAggs or any other table in future, i will probably change the field name

[cursor]

Cumulative counts reset on each continueAsNew iteration

Medium Severity

The totalOrphansFound, totalOrphansDeleted, and startTime variables are initialized to zero at the start of each workflow function, which resets them after every continueAsNew call. Since updateOrphanCleanupRun uses direct assignment rather than increment, each batch overwrites the previous counts with only the current batch's values. The final orphanCleanupRuns record will only reflect the last batch's statistics, not cumulative totals across all batches. Similarly, executionTimeMs on completion only measures the final empty-results query, not total execution time.

Additional Locations (1)

• services/apps/script_executor_worker/src/workflows/cleanup/segments-agg.ts#L103-L157

 

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.