fix(whatsmeow): reuse pooled authDB connection in StartClient instead of leaking a new pool per call - #174
Open
fmedeiros95 wants to merge 2 commits into
Conversation
sqlstore.New() abria um *sql.DB novo, sem limites de pool e nunca fechado, toda vez que StartClient rodava (conexao inicial, reconexao manual, restart apos LoggedOut, loop de startup). Em producao isso vazou o max_connections do Postgres apos ~2 dias com poucas dezenas de instancias reconectando. Troca por sqlstore.NewWithDB(w.authDB, ...), reaproveitando o pool ja existente e limitado (SetMaxOpenConns/SetMaxIdleConns/etc, configurado em pkg/config/config.go). Como NewWithDB nao roda Upgrade sozinho como o New(), a chamada de Upgrade(ctx) foi movida para o call site.
Reviewer's guide (collapsed on small PRs)Reviewer's GuideThis PR changes StartClient to reuse the existing, bounded Postgres authDB connection pool instead of opening and leaking a new *sql.DB pool on every call, while preserving the automatic schema upgrade behavior; SQLite handling is left unchanged. Sequence diagram for StartClient Postgres container initializationsequenceDiagram
participant caller
participant whatsmeowService
participant sqlstore
participant Postgres
caller->>whatsmeowService: StartClient(cd)
alt PostgresAuthDB configured
whatsmeowService->>sqlstore: NewWithDB(authDB, postgres, dbLog)
sqlstore-->>whatsmeowService: container
whatsmeowService->>sqlstore: container.Upgrade(context.Background)
sqlstore->>Postgres: apply schema migrations using authDB pool
sqlstore-->>whatsmeowService: upgrade result
else SQLite
whatsmeowService->>sqlstore: New(context.Background, sqlite, dsn, dbLog)
sqlstore-->>whatsmeowService: container
end
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Consider guarding the Postgres branches with a nil check on
w.authDBso thatNewWithDBdoesn’t panic if the shared connection wasn’t initialized as expected. - The explanatory comment added above the Postgres
NewWithDBcall is quite long and in Portuguese; you might shorten it and align the language with the rest of the codebase for consistency.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider guarding the Postgres branches with a nil check on `w.authDB` so that `NewWithDB` doesn’t panic if the shared connection wasn’t initialized as expected.
- The explanatory comment added above the Postgres `NewWithDB` call is quite long and in Portuguese; you might shorten it and align the language with the rest of the codebase for consistency.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
- Bail out early with a clear log if PostgresAuthDB is configured but authDB was never initialized, instead of letting NewWithDB receive a nil *sql.DB. - Shorten the explanatory comment above NewWithDB and switch it to English to match the rest of the codebase.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
StartClient(pkg/whatsmeow/service/whatsmeow.go) opens a brand new*sql.DBconnection pool on every call, via:sqlstore.Newinternally doessql.Open(dialect, address)— a fresh, completely unbounded pool (noSetMaxOpenConns/SetMaxIdleConns/SetConnMaxLifetime), independent from the properly-configured GORM pool set up inpkg/config/config.go. Thecontainervariable is local toStartClientand is never closed (container.Close()is not called anywhere in the file), so every call leaks the entire underlying connection pool.StartClientis called repeatedly during normal operation, not just once per process lifetime:LoggedOutevent auto-restart (whatsmeow.go, "restart client")CONNECT_ON_STARTUPPOST /instance/connect) — note this path only callsclient.Disconnect()on the oldwhatsmeow.Client, it never closes the oldsqlstore.Container/pool before creating a new oneIn production, with only a couple of WhatsApp instances reconnecting periodically (WhatsApp forces session logout for various reasons on its own, no user action required), this accumulated ~95 idle Postgres connections over about 2 days and exhausted
max_connections, taking down the whole app withremaining connection slots are reserved for roles with the SUPERUSER attribute— including breaking evolution-go's own ability to open new connections, so instances got stuck in a reconnect-fail loop.Fix
whatsmeowServicealready holdsauthDB *sql.DB— the same GORM-backed, properly pool-limited connection injected viaNewWhatsmeowService.sqlstoreexposesNewWithDB(db *sql.DB, dialect string, log)exactly for reusing an existing connection instead of opening a new one. This PR switches the Postgres branches ofStartClientto:NewWithDBdoesn't runUpgradeautomatically likeNewdoes, so theUpgradecall is made explicitly right after, preserving the original behavior (schema stays up to date) while reusing the bounded pool instead of leaking a new one per call.The SQLite branches are untouched —
w.sqliteDBwraps a different file (dbdata/users.dbvs. the whatsmeow store'sdbdata/main.db), so blindly reusing it there would point at the wrong database; fixing that path would need separate investigation and is out of scope for this fix.Testing
CGO_ENABLED=1 go build ./cmd/evolution-go— builds cleanly.POSTGRES_AUTH_DB,max_connections=100shared with other apps on the same server). Before the fix,pg_stat_activityshowed theevolutionrole accumulating idle connections on every instance reconnect/restart, with recurringFailed to create container: failed to upgrade database: ... remaining connection slots are reserved for roles with the SUPERUSER attributein the logs. After deploying the fix and restarting, idle connections for theevolutionrole stayed at 1-2 across multiple reconnect cycles, with no recurrence of the pool-exhaustion error.Summary by Sourcery
Bug Fixes: