Skip to content

Split a comma-joined host list into a sequence for asyncpg - #2267

Open
afonsojanu wants to merge 1 commit into
tortoise:developfrom
afonsojanu:fix/asyncpg-multi-host-dsn-1924
Open

Split a comma-joined host list into a sequence for asyncpg#2267
afonsojanu wants to merge 1 commit into
tortoise:developfrom
afonsojanu:fix/asyncpg-multi-host-dsn-1924

Conversation

@afonsojanu

Copy link
Copy Markdown

Description

A postgres DB URL with more than one host, e.g. postgres://user:pass@host1,host2:5432/db, is libpq's own multi-host syntax for primary/replica failover. urlparse reads the whole comma-joined string back as a single hostname, so asyncpg.connect() ended up being asked to resolve host1,host2 as one DNS name and failed with a plain socket.gaierror instead of trying each host in turn.

expand_db_url now splits the hostname into a list when it contains a comma and the backend is one of postgres/postgresql/asyncpg, since that's the form asyncpg's own host parameter accepts for exactly this case (it documents accepting "a sequence" of hosts, tried in order). psycopg is left untouched: it forwards its kwargs straight to libpq, which already parses the joined string correctly on its own, so splitting it there would just make libpq see a Python list where it expects the original string.

Motivation and Context

Fixes #1924

How Has This Been Tested?

Added test_postgres_multi_host to tests/backends/test_db_url.py, covering all four postgres-family schemes (postgres, postgresql, asyncpg, psycopg) against the same multi-host URL, asserting the asyncpg-family schemes get a list and psycopg keeps the joined string. Confirmed via git stash that it fails against the unmodified code (asserts a list where the code still returns the joined string) and passes with the fix.

  • python -m pytest tests/backends/test_db_url.py tests/test_connection.py: 59 passed.
  • ruff check and ruff format --check: clean.
  • mypy on the changed file: 0 errors.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added the changelog accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

A postgres DB URL with more than one host, e.g. host1,host2:5432, is
libpq's own multi-host connection string syntax for primary/replica
failover. urlparse reads the whole comma-joined string back as a
single hostname, so asyncpg.connect() ended up being asked to resolve
"host1,host2" as one DNS name and failed with a plain gaierror instead
of trying each host in turn.

psycopg forwards its kwargs straight through to libpq, which already
understands the joined string on its own, so only the asyncpg path
needed the split into an actual list of hosts, which is the form
asyncpg's own host parameter accepts for this exact case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Postgres connection string (dsn) with multi-host does not work

1 participant