Skip to content

Fix the defects the agents page turned up - #22

Merged
AlexeyShalaev merged 6 commits into
masterfrom
fix/agents-page-findings
Sep 6, 2026
Merged

Fix the defects the agents page turned up#22
AlexeyShalaev merged 6 commits into
masterfrom
fix/agents-page-findings

Conversation

@AlexeyShalaev

Copy link
Copy Markdown
Member

Writing docs/agents.md against the source turned up a list of defects. This is that list,
worked through in order of how much damage each one does to a caller. Every one was
reproduced before it was touched; the reproductions are below.

Nothing here is breaking in the API sense — no name was removed, no signature narrowed, no
default changed. One behaviour does change for the better and is called out under finding 3.


1. The guides and the home page documented a settings API that does not exist

What was wrong. docs/index.md, docs/guide/quickstart.md, docs/guide/configuration.md
and docs/guide/advanced.md all called BaseRedisSettings(host=…, port=…, retry_enabled=…, cluster_mode=…, ssl=True, decode_responses=…). The model is grouped — connection=,
cluster=, pool=, retry=, ssl=, response= — forbids extras, and requires
key_prefix. The README was already right; those four pages were not.

How I know.

>>> BaseRedisSettings(host='localhost', port=6379, decode_responses=True)
ValidationError: 4 validation errors for BaseRedisSettings
key_prefix   Field required
host         Extra inputs are not permitted
port         Extra inputs are not permitted
decode_responses  Extra inputs are not permitted

What I changed. The doc, not the code — the README, docs/agents.md and the model all
agree, and it is the four pages that drifted. Every snippet now passes the group models.
While rewriting them I also corrected the prose that was wrong for the same reason:
env_prefix="REDIS_" alone does not produce REDIS_HOST for a nested group (it needs
env_nested_delimiter), retry.enabled=True with the default max_attempts=0 retries
nothing, ssl.cert_reqs is mandatory once SSL is on, a cluster node string needs an
explicit port, and health_check_interval is redis-py's per-connection ping interval
rather than the check_*_redis_health function.

To check the result rather than eyeball it, I extracted every BaseRedisSettings(...) call
from every fenced Python block in docs/ and README.md and evaluated it: 37 construct
successfully. The three that do not are the deliberate # WRONG example on the agents page
and two snippets using socket.TCP_KEEPIDLE, which exists on Linux and not on the macOS I
ran this from.

2. docs/index.md claimed features and versions the package does not have

What was wrong. OpenTelemetry instrumentation (it is Prometheus), an [instrumentation]
extra (it is [metrics]), redis>=7.1.0 (>=4.5.0,<9.0.0) and "Requirements: Python 3.11+"
(requires-python = ">=3.10", and the classifiers list 3.10).

What I changed. All four, plus the same OpenTelemetry line in zensical.toml's
site_description, which feeds the site's meta description.

3. retry_async_connection could not fail startup

What was wrong. It raised only when connect_func raised, and check_async_redis_health
returns False rather than raising. Against an unreachable Redis the loop ran its three
attempts, took the if branch nowhere, never slept, and returned normally — so
get_redis_with_health_check handed the container a client that cannot reach Redis, while
its own docstring said "Application startup will be blocked until Redis is available or max
retries reached".

How I know. Against a dead port:

INFO  Creating single Redis client
WARNING  Async Redis health check failed
WARNING  Async Redis health check failed
WARNING  Async Redis health check failed
RETURNED NORMALLY after 10.21s -- startup NOT failed

Three health checks, no backoff log line between them, no exception.

What I changed. A falsy result is now a failed attempt like any other: it is retried with
the same exponential backoff and, on the last attempt, raised as a builtin ConnectionError
naming the service. An attempt that raised still propagates its own exception, so a caller
catching a specific error from connect_func keeps catching it. Since the function can now
raise, get_redis_with_health_check closes the client it built before letting the error out.

Is it breaking? It is a behaviour change, and worth being plain about: an application
that today starts with an unreachable Redis and fails on first use will now fail at startup.
That is what the docstring promised, it is the point of asking for a health check, and the
new check_health_on_startup=False (finding 4) is the supported way to keep the old
behaviour. It goes in as fix: because the code was not doing what it documented.

Note that the raised error is Python's ConnectionError, not redis.exceptions.ConnectionError,
so it is not caught by except RedisError. That is now a row in the agents page's error table.

4. AsyncRedisProvider.get_redis was unreachable through a container

What was wrong. get_redis and get_redis_with_health_check both provided
AsyncRedisClient, so dishka kept the second and dropped the first. The docstring's "two
Redis client options" were one, with no way to ask for the other.

How I know.

FACTORY (Redis | RedisCluster, component=None) <- AsyncRedisProvider.get_redis
FACTORY (Redis | RedisCluster, component=None) <- AsyncRedisProvider.get_redis_with_health_check

and resolving AsyncRedisClient ran three health checks, i.e. the second factory.

What I changed. The choice moved to where it can actually be made — the constructor.
AsyncRedisProvider() registers the health-checking factory as before;
AsyncRedisProvider(check_health_on_startup=False) registers the other one. Both methods
are still on the class and still public; only the registration changed, so the default
behaviour through a container is exactly what it was.

5. Provider order was load-bearing and silent

What was wrong. AsyncRedisProvider provides RedisMetricsProtocol | None as None.
Dishka lets the last provider of a type win, so registering it after a metrics provider
overrode a real metrics implementation with None and produced a plain, uninstrumented
client with no error anywhere.

How I know.

order (AsyncRedisProvider, AppProvider): metrics = Metrics
order (AppProvider, AsyncRedisProvider): metrics = NoneType

What I changed. Dishka's last-wins rule is not something this library can change, so I
added the way out: AsyncRedisProvider(provide_default_metrics=False) registers no default
and leaves the type to your provider, whatever the order. Registering AsyncRedisProvider()
first still works and is still what the docs show first. The class docstring and the agents
page now both say which orders are safe.

6. redis_pool_size was always 0 on the async client

What was wrong. redis_client_kit/aio/instrumented.py read pool._all_connections.

How I know. On redis-py 8.1:

>>> sorted(a for a in vars(redis.asyncio.ConnectionPool(...)) if a.startswith('_'))
['_available_connections', '_connection_kwargs', '_event_dispatcher', '_in_use_connections', ...]
>>> hasattr(pool, '_all_connections')
False

getattr(pool, "_all_connections", []) therefore returned [] on every command and the
gauge sat at 0. The sync client was already correct.

What I changed. The async client sums _available_connections and _in_use_connections
the way the sync one does. Two tests cover it, one of them against a real
redis.asyncio.ConnectionPool so a future rename in redis-py breaks the test rather than
the gauge.

7. A translated transport error was counted as a success

What was wrong. In all four instrumented clients the except RuntimeError branch sat
ahead of the except Exception branch that sets status = "error" and calls
record_error. A uvloop closed-transport RuntimeError translated into redis-py's
ConnectionError — and every other RuntimeError — was therefore counted in
redis_commands_total{status="success"} and never appeared in
redis_connection_errors_total.

How I know. Reading the branch order, then the test: with the old file in place,
record_error is called 0 times and record_command gets status="success" for a
RuntimeError("… the handler is closed").

What I changed. The RuntimeError branch now records the error it is about to raise
before raising it — under ConnectionError when it translates, under RuntimeError when it
does not — and sets status = "error" either way. The translation itself and the exception a
caller sees are unchanged. Six tests, three async and three sync.

8. Smaller things

  • make test-unit failed on a clean checkout. make install ran uv sync --group dev,
    which installs no extras, so collection died with ModuleNotFoundError: No module named 'pydantic' in tests/unit/settings/. CI has always used --all-extras; the Makefile and
    CONTRIBUTING.md now say the same thing.
  • mask_redis_kwargs was public by name and in no __all__. redis_client_kit/utils.py
    had no __all__ at all; it now lists its four public helpers. mask_redis_kwargs stays out
    of the root package, where it has never been and where docs/agents.md does not claim it is.
  • settings/redis.py declared __all__ = ["BaseRedisSettings"] while the package
    re-exports all seven models, so from redis_client_kit.settings.redis import RedisSSLSettings
    is not an export as far as a strict type checker is concerned. It lists all seven now.
  • The README's documentation list pointed at .../reference/api/, which is not a page.
    The reference is at .../reference/.

Coverage

providers/redis.py and providers/utils.py were on the coverage omit list and had no tests
at all — findings 3, 4 and 5 all lived in code nothing exercised. They now have two test
files and came off the omit list; both are at 100% and 96%.

tests/unit/providers/test_providers_init.py replaced entries in sys.modules without
restoring them, which left the new provider tests looking at a stale module object when they
ran after it. It uses monkeypatch.setitem/delitem now, which restores on teardown. The
same pattern is still in tests/unit/settings/test_settings_import_error.py and
tests/unit/metrics/test_metrics_init.py; nothing depends on it today, so I left them alone.

docs/agents.md

Rules 15, 16, 17, 19 and 20 stated five of the defects above as rules a caller has to obey,
which is the failure mode that page exists to prevent. They now describe the fixed behaviour.
The Dishka section gained the two constructor arguments, the error table gained the
provider's ConnectionError, and the caveat under the documentation map — the one warning
that the guide pages' constructor calls were wrong — is gone, because they are not any more.

Verification

uv sync --frozen --all-extras --group dev; uv.lock is byte-identical to origin/master.

$ make check
uv run ruff check .
All checks passed!
uv run ruff format --check .
54 files already formatted
uv run mypy redis_client_kit
Success: no issues found in 25 source files

$ make test-unit
132 passed, 2 deselected, 1 warning in 0.41s

$ make test-integration
2 passed, 132 deselected, 1 warning in 1.39s

$ make test
TOTAL   562   2   99%
Required test coverage of 90% reached. Total coverage: 99.64%
134 passed, 1 warning in 2.05s

Docs build clean: zensical build --clean reports "No issues found", and
scripts/emit_markdown.py writes 6 Markdown pages with 1 declined, as before.

One thing I did not touch

make docs-build, make docs-serve and make clean shell out to python, which does not
exist on a machine that only has python3make docs-build dies with
/bin/sh: python: command not found before it reaches zensical. CI is unaffected because
the Docs workflow runs cp CHANGELOG.md docs/changelog.md itself rather than going through
make. It is unrelated to anything on this list, so it is not in this pull request.

Alex Shalaev and others added 6 commits September 6, 2026 21:15
…nc pool

The async instrumented client read pool._all_connections, which redis-py's
asyncio ConnectionPool does not have (it keeps _available_connections and
_in_use_connections), so redis_pool_size was always 0. It now sums the two
containers, the way the sync client already did.

Both clients also had the except RuntimeError branch ahead of the branch that
sets status = "error" and calls record_error. A uvloop closed-transport error
translated into redis-py's ConnectionError, and every other RuntimeError, was
therefore counted in redis_commands_total{status="success"} and never reached
redis_connection_errors_total. The RuntimeError branch now records the error it
raises before raising it.
…client choice real

Three problems in one place.

retry_async_connection only ever raised when connect_func itself raised, and
check_async_redis_health returns False rather than raising. Against a dead Redis
the loop ran its three attempts with no backoff sleep and returned normally, so
get_redis_with_health_check handed the container a client that cannot reach
Redis while its docstring promised the opposite. A falsy result is now a failed
attempt like any other: it is retried with the same backoff and, on the last
attempt, raised as ConnectionError. An attempt that raised still propagates its
own error. The client is closed if the retries run out, since the failure path
can now leave one behind.

get_redis and get_redis_with_health_check both provided AsyncRedisClient, so
dishka kept only the second and the "two Redis client options" in the docstring
were one. The choice is now made where it belongs, at construction:
AsyncRedisProvider(check_health_on_startup=False) registers the other one. Both
methods are still there and still public.

The default RedisMetricsProtocol | None of None silently overrode a metrics
provider registered before AsyncRedisProvider, leaving a plain client and no
error. Registering AsyncRedisProvider first still works; provider order no
longer has to be right, because provide_default_metrics=False turns the default
off.

The providers had no tests at all: none of this was covered. The new files cover
the retry helper and the provider's registration and startup behaviour.
tests/unit/providers/test_providers_init.py replaced entries in sys.modules
without restoring them, which made those new tests see a stale module; it uses
monkeypatch now.
…dules export

make install ran `uv sync --group dev`, which installs no extras, so on a clean
checkout `make test-unit` died collecting tests/unit/settings with
ModuleNotFoundError: No module named 'pydantic'. CI has always used
--all-extras; the Makefile and CONTRIBUTING now say the same thing.

redis_client_kit/utils.py had no __all__ at all, and settings/redis.py declared
only BaseRedisSettings while the package re-exports all seven models, so a
direct `from redis_client_kit.settings.redis import RedisSSLSettings` is not an
export as far as a strict type checker is concerned. Both now list what they
define. mask_redis_kwargs stays out of the root package, where it has never
been.

The README pointed at .../reference/api/, which is not a page; the reference is
at .../reference/.

providers/redis.py and providers/utils.py came off the coverage omit list now
that they have tests.
…hat ships

Every BaseRedisSettings(...) call on docs/index.md and the three guide pages
passed flat keywords -- host=, port=, retry_enabled=, cluster_mode=, ssl=True,
decode_responses= -- to a model that has been grouped for some time, forbids
extras and requires key_prefix. All of them raised ValidationError, so nothing a
reader copied off those pages ran. They now pass the group models, and the
prose around them says what the model actually enforces: max_attempts=0 retries
nothing, ssl.cert_reqs is mandatory once SSL is on, a cluster node needs a port,
health_check_interval is redis-py's ping interval.

The environment-variable section promised REDIS_HOST from env_prefix="REDIS_"
alone; nested groups need env_nested_delimiter, which the example now sets.

docs/index.md also advertised OpenTelemetry instrumentation and an
[instrumentation] extra, neither of which exists -- it is Prometheus and
[metrics] -- and claimed redis>=7.1.0 (actually >=4.5.0,<9.0.0) and Python 3.11+
(actually 3.10+). zensical.toml carried the same OpenTelemetry line.

The Dishka section now shows the provider constructor and says which
registration orders work.
…alse

Rules 15 to 17, 19 and 20 stated the defects fixed earlier in this branch as
things a caller has to work around, which is exactly the failure mode the page
exists to avoid. They now describe what the code does: the provider registers
one client factory chosen by check_health_on_startup, its startup health check
raises ConnectionError instead of falling through, provide_default_metrics=False
takes provider order out of the metrics question, single-node pool gauges are
real, and a translated closed-transport error is counted as an error under the
type the caller catches.

The Dishka section gains the constructor arguments, the Errors table gains the
provider's ConnectionError, and the caveat under the documentation map is gone
now that the pages it warned about are correct.
@AlexeyShalaev

Copy link
Copy Markdown
Member Author

One correction on top of this, in its own commit.

The PR notes that pre-existing table cells contain unescaped pipes inside type names and that those rows are broken. They are not. I checked the deployed page: across every table on agents.md, each row's cell count matches its header, the three rows containing int | None, Redis | RedisCluster and redis.retry.Retry | None included. A pipe inside a code span does not split a row in this renderer.

Escaping it does cause a visible defect, though: the backslash is content inside a code span, so \\| renders as a literal backslash on the page. That is why the escaping was removed org-wide earlier this week. The new provide_default_metrics row had picked it up, so it is unescaped now and matches the rest of the file.

@AlexeyShalaev
AlexeyShalaev merged commit df19ab5 into master Sep 6, 2026
6 checks passed
@AlexeyShalaev
AlexeyShalaev deleted the fix/agents-page-findings branch September 6, 2026 18:45
AlexeyShalaev added a commit that referenced this pull request Sep 6, 2026
…s as errors (#23)

The work landed in #22. Its squash subject lost the Conventional Commit prefix --
my mistake on the merge, not the author's -- so release-please skipped the merge
and these fixes would never have reached a release. This commit carries the
record. It changes no code: #22 is already on master.

* retry_async_connection could not fail startup. It raised only when connect_func
  raised, while the health check returned False instead of raising, so the loop
  spun three times with no backoff and handed back a client that could not reach
  Redis. A falsy result is a retried failure now and the last attempt raises.
  Reproduced against a dead port.
* AsyncRedisProvider.get_redis was unreachable: it and get_redis_with_health_check
  provided the same dishka key and the second won. The choice moved to a
  constructor flag and both methods are reachable.
* Provider order was load-bearing and silent: the Redis provider supplied a None
  metrics collector, so registered after a metrics provider it overrode it and you
  got a plain client with no error. A flag turns the default off.
* The async instrumented client read pool._all_connections, which redis-py 8 does
  not have, so the pool-size gauge was always zero. It sums the two attributes the
  sync client already used.
* In all four clients the RuntimeError branch preceded the one that sets an error
  status, so a uvloop transport error translated to ConnectionError was counted as
  a success and never recorded.
* The three guide pages and the docs home documented a settings API that does not
  exist -- every snippet raised a validation error -- along with OpenTelemetry
  support, a wrong redis floor, a wrong Python floor and an extra that is not
  declared. All corrected against the code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant