Run Puma in cluster mode, and give the nightly purge container a deadline

[openipc-ai]

Two pieces of housekeeping:

config/puma.rb — MRI executes Ruby on one core per process, so a single Puma process with sixteen threads still serves every request from one core while the rest of the machine idles. Production now boots two workers (overridable through WEB_CONCURRENCY); development stays in single mode. No preload_app!: phased restarts (SIGUSR1) replace workers one at a time with the listener kept open — a restart without dropped requests — and they only work when each worker boots the app itself.

deploy/purge-snapshots.sh — the rails runner performing PurgeImagesJob can deadlock at process exit while draining the :async adapter's thread pool (ActiveStorage still enqueues PurgeJobs during destroy despite the inline purge). When that happens the container never exits, --rm never fires, one hung container accumulates per night holding a MySQL connection open, and set -e keeps the rest of the nightly script (download retention, orphan reap, shard-directory cleanup) from running at all. The purge work itself finishes in seconds, before the hang, so the container now gets a ten-minute deadline and a forced cleanup, and a failure logs a warning instead of aborting the script.

Verified: full test suite green (359 runs, 0 failures), ruby -c clean, rubocop adds no offenses to the touched files.

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Enable Puma clustering and bound nightly purge runtime

✨ Enhancement 🐞 Bug fix ⚙️ Configuration changes 🕐 10-20 Minutes

AI Description

• Enables two Puma workers in production while preserving phased restarts and development single
 mode.
• Bounds nightly snapshot purges and continues subsequent retention cleanup after failures.

Diagram

graph TD
  Env["Production Env"] --> Puma["Puma Config"] --> Workers["Puma Workers"]
  Cron["Nightly Cron"] --> Script["Purge Script"] --> Container["Purge Container"] --> Removal["Forced Removal"] --> Cleanup["Later Cleanup"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Fix asynchronous purge shutdown

• ➕ Eliminates the underlying thread-pool exit deadlock
• ➕ Allows the purge container to terminate naturally
• ➖ Requires deeper Rails and ActiveStorage lifecycle changes
• ➖ Does not protect the nightly workflow from unrelated hangs

2. Derive workers from CPU count

• ➕ Automatically scales Puma processes to each production host
• ➕ Can maximize CPU utilization without manual configuration
• ➖ May multiply memory and database connection usage unexpectedly
• ➖ Provides less predictable capacity than a conservative default

Recommendation: Keep the PR's approach: two workers are a conservative production default with an explicit override, while the purge deadline provides operational containment without blocking later maintenance. Investigating synchronous purge execution can be a follow-up, but a deadline should remain as defense against future hangs.

Files changed (2) +20 / -12

Bug fix (1) +10 / -2

purge-snapshots.shBound and clean up nightly purge containers +10/-2

Bound and clean up nightly purge containers

• Runs the snapshot purge container under a ten-minute timeout with a stable name and forced cleanup. Purge failures now emit a warning instead of preventing subsequent retention and storage maintenance.

deploy/purge-snapshots.sh

Other (1) +10 / -10

puma.rbEnable production Puma cluster mode +10/-10

Enable production Puma cluster mode

• Defaults production to two Puma workers, supports WEB_CONCURRENCY overrides, and keeps development in single-process mode. Workers boot the application independently so phased restarts remain available without preload_app!.

config/puma.rb

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Cleanup kills concurrent purge ✓ Resolved 🐞 Bug ☼ Reliability

Description

Every invocation force-removes the globally named container even when its own docker run failed
because another purge already owns that name. An overlapping or interrupted run can therefore be
killed while the new invocation skips snapshot purging and continues with later cleanup steps.

Code

deploy/purge-snapshots.sh[53]

+docker rm -f openipc-purge-snapshots >/dev/null 2>&1 || true

Evidence

The script documents a cron invocation but acquires no lock. Lines 46-52 assign every run the same
name and convert name collisions or other launch failures into warnings; line 53 then force-removes
whichever container currently owns that name, while lines 55-97 continue the remaining retention and
cleanup operations.

deploy/purge-snapshots.sh[6-15]
deploy/purge-snapshots.sh[46-53]
deploy/purge-snapshots.sh[55-73]
deploy/purge-snapshots.sh[75-97]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The purge script uses one global container name and unconditionally force-removes it. If invocations overlap or a prior invocation left a container behind, one run can remove another run's container and then continue without performing the purge.
## Issue Context
Use per-invocation container identity, such as a unique name or `--cidfile`, and clean up only the container created by that invocation. Alternatively, serialize the entire script with a lock and handle stale containers explicitly before starting the purge.
## Fix Focus Areas
- deploy/purge-snapshots.sh[39-53]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[openipc-ai]

/review

[qodo-free-for-open-source-projects]

Code review by qodo was updated up to the latest commit 1401ca7