Fix the defects the agents page turned up - #22
Conversation
…nc pool
The async instrumented client read pool._all_connections, which redis-py's
asyncio ConnectionPool does not have (it keeps _available_connections and
_in_use_connections), so redis_pool_size was always 0. It now sums the two
containers, the way the sync client already did.
Both clients also had the except RuntimeError branch ahead of the branch that
sets status = "error" and calls record_error. A uvloop closed-transport error
translated into redis-py's ConnectionError, and every other RuntimeError, was
therefore counted in redis_commands_total{status="success"} and never reached
redis_connection_errors_total. The RuntimeError branch now records the error it
raises before raising it.
…client choice real Three problems in one place. retry_async_connection only ever raised when connect_func itself raised, and check_async_redis_health returns False rather than raising. Against a dead Redis the loop ran its three attempts with no backoff sleep and returned normally, so get_redis_with_health_check handed the container a client that cannot reach Redis while its docstring promised the opposite. A falsy result is now a failed attempt like any other: it is retried with the same backoff and, on the last attempt, raised as ConnectionError. An attempt that raised still propagates its own error. The client is closed if the retries run out, since the failure path can now leave one behind. get_redis and get_redis_with_health_check both provided AsyncRedisClient, so dishka kept only the second and the "two Redis client options" in the docstring were one. The choice is now made where it belongs, at construction: AsyncRedisProvider(check_health_on_startup=False) registers the other one. Both methods are still there and still public. The default RedisMetricsProtocol | None of None silently overrode a metrics provider registered before AsyncRedisProvider, leaving a plain client and no error. Registering AsyncRedisProvider first still works; provider order no longer has to be right, because provide_default_metrics=False turns the default off. The providers had no tests at all: none of this was covered. The new files cover the retry helper and the provider's registration and startup behaviour. tests/unit/providers/test_providers_init.py replaced entries in sys.modules without restoring them, which made those new tests see a stale module; it uses monkeypatch now.
…dules export make install ran `uv sync --group dev`, which installs no extras, so on a clean checkout `make test-unit` died collecting tests/unit/settings with ModuleNotFoundError: No module named 'pydantic'. CI has always used --all-extras; the Makefile and CONTRIBUTING now say the same thing. redis_client_kit/utils.py had no __all__ at all, and settings/redis.py declared only BaseRedisSettings while the package re-exports all seven models, so a direct `from redis_client_kit.settings.redis import RedisSSLSettings` is not an export as far as a strict type checker is concerned. Both now list what they define. mask_redis_kwargs stays out of the root package, where it has never been. The README pointed at .../reference/api/, which is not a page; the reference is at .../reference/. providers/redis.py and providers/utils.py came off the coverage omit list now that they have tests.
…hat ships Every BaseRedisSettings(...) call on docs/index.md and the three guide pages passed flat keywords -- host=, port=, retry_enabled=, cluster_mode=, ssl=True, decode_responses= -- to a model that has been grouped for some time, forbids extras and requires key_prefix. All of them raised ValidationError, so nothing a reader copied off those pages ran. They now pass the group models, and the prose around them says what the model actually enforces: max_attempts=0 retries nothing, ssl.cert_reqs is mandatory once SSL is on, a cluster node needs a port, health_check_interval is redis-py's ping interval. The environment-variable section promised REDIS_HOST from env_prefix="REDIS_" alone; nested groups need env_nested_delimiter, which the example now sets. docs/index.md also advertised OpenTelemetry instrumentation and an [instrumentation] extra, neither of which exists -- it is Prometheus and [metrics] -- and claimed redis>=7.1.0 (actually >=4.5.0,<9.0.0) and Python 3.11+ (actually 3.10+). zensical.toml carried the same OpenTelemetry line. The Dishka section now shows the provider constructor and says which registration orders work.
…alse Rules 15 to 17, 19 and 20 stated the defects fixed earlier in this branch as things a caller has to work around, which is exactly the failure mode the page exists to avoid. They now describe what the code does: the provider registers one client factory chosen by check_health_on_startup, its startup health check raises ConnectionError instead of falling through, provide_default_metrics=False takes provider order out of the metrics question, single-node pool gauges are real, and a translated closed-transport error is counted as an error under the type the caller catches. The Dishka section gains the constructor arguments, the Errors table gains the provider's ConnectionError, and the caveat under the documentation map is gone now that the pages it warned about are correct.
|
One correction on top of this, in its own commit. The PR notes that pre-existing table cells contain unescaped pipes inside type names and that those rows are broken. They are not. I checked the deployed page: across every table on Escaping it does cause a visible defect, though: the backslash is content inside a code span, so |
…s as errors (#23) The work landed in #22. Its squash subject lost the Conventional Commit prefix -- my mistake on the merge, not the author's -- so release-please skipped the merge and these fixes would never have reached a release. This commit carries the record. It changes no code: #22 is already on master. * retry_async_connection could not fail startup. It raised only when connect_func raised, while the health check returned False instead of raising, so the loop spun three times with no backoff and handed back a client that could not reach Redis. A falsy result is a retried failure now and the last attempt raises. Reproduced against a dead port. * AsyncRedisProvider.get_redis was unreachable: it and get_redis_with_health_check provided the same dishka key and the second won. The choice moved to a constructor flag and both methods are reachable. * Provider order was load-bearing and silent: the Redis provider supplied a None metrics collector, so registered after a metrics provider it overrode it and you got a plain client with no error. A flag turns the default off. * The async instrumented client read pool._all_connections, which redis-py 8 does not have, so the pool-size gauge was always zero. It sums the two attributes the sync client already used. * In all four clients the RuntimeError branch preceded the one that sets an error status, so a uvloop transport error translated to ConnectionError was counted as a success and never recorded. * The three guide pages and the docs home documented a settings API that does not exist -- every snippet raised a validation error -- along with OpenTelemetry support, a wrong redis floor, a wrong Python floor and an extra that is not declared. All corrected against the code.
Writing
docs/agents.mdagainst the source turned up a list of defects. This is that list,worked through in order of how much damage each one does to a caller. Every one was
reproduced before it was touched; the reproductions are below.
Nothing here is breaking in the API sense — no name was removed, no signature narrowed, no
default changed. One behaviour does change for the better and is called out under finding 3.
1. The guides and the home page documented a settings API that does not exist
What was wrong.
docs/index.md,docs/guide/quickstart.md,docs/guide/configuration.mdand
docs/guide/advanced.mdall calledBaseRedisSettings(host=…, port=…, retry_enabled=…, cluster_mode=…, ssl=True, decode_responses=…). The model is grouped —connection=,cluster=,pool=,retry=,ssl=,response=— forbids extras, and requireskey_prefix. The README was already right; those four pages were not.How I know.
What I changed. The doc, not the code — the README,
docs/agents.mdand the model allagree, and it is the four pages that drifted. Every snippet now passes the group models.
While rewriting them I also corrected the prose that was wrong for the same reason:
env_prefix="REDIS_"alone does not produceREDIS_HOSTfor a nested group (it needsenv_nested_delimiter),retry.enabled=Truewith the defaultmax_attempts=0retriesnothing,
ssl.cert_reqsis mandatory once SSL is on, a cluster node string needs anexplicit port, and
health_check_intervalis redis-py's per-connection ping intervalrather than the
check_*_redis_healthfunction.To check the result rather than eyeball it, I extracted every
BaseRedisSettings(...)callfrom every fenced Python block in
docs/andREADME.mdand evaluated it: 37 constructsuccessfully. The three that do not are the deliberate
# WRONGexample on the agents pageand two snippets using
socket.TCP_KEEPIDLE, which exists on Linux and not on the macOS Iran this from.
2.
docs/index.mdclaimed features and versions the package does not haveWhat was wrong. OpenTelemetry instrumentation (it is Prometheus), an
[instrumentation]extra (it is
[metrics]),redis>=7.1.0(>=4.5.0,<9.0.0) and "Requirements: Python 3.11+"(
requires-python = ">=3.10", and the classifiers list 3.10).What I changed. All four, plus the same OpenTelemetry line in
zensical.toml'ssite_description, which feeds the site's meta description.3.
retry_async_connectioncould not fail startupWhat was wrong. It raised only when
connect_funcraised, andcheck_async_redis_healthreturns
Falserather than raising. Against an unreachable Redis the loop ran its threeattempts, took the
ifbranch nowhere, never slept, and returned normally — soget_redis_with_health_checkhanded the container a client that cannot reach Redis, whileits own docstring said "Application startup will be blocked until Redis is available or max
retries reached".
How I know. Against a dead port:
Three health checks, no backoff log line between them, no exception.
What I changed. A falsy result is now a failed attempt like any other: it is retried with
the same exponential backoff and, on the last attempt, raised as a builtin
ConnectionErrornaming the service. An attempt that raised still propagates its own exception, so a caller
catching a specific error from
connect_funckeeps catching it. Since the function can nowraise,
get_redis_with_health_checkcloses the client it built before letting the error out.Is it breaking? It is a behaviour change, and worth being plain about: an application
that today starts with an unreachable Redis and fails on first use will now fail at startup.
That is what the docstring promised, it is the point of asking for a health check, and the
new
check_health_on_startup=False(finding 4) is the supported way to keep the oldbehaviour. It goes in as
fix:because the code was not doing what it documented.Note that the raised error is Python's
ConnectionError, notredis.exceptions.ConnectionError,so it is not caught by
except RedisError. That is now a row in the agents page's error table.4.
AsyncRedisProvider.get_rediswas unreachable through a containerWhat was wrong.
get_redisandget_redis_with_health_checkboth providedAsyncRedisClient, so dishka kept the second and dropped the first. The docstring's "twoRedis client options" were one, with no way to ask for the other.
How I know.
and resolving
AsyncRedisClientran three health checks, i.e. the second factory.What I changed. The choice moved to where it can actually be made — the constructor.
AsyncRedisProvider()registers the health-checking factory as before;AsyncRedisProvider(check_health_on_startup=False)registers the other one. Both methodsare still on the class and still public; only the registration changed, so the default
behaviour through a container is exactly what it was.
5. Provider order was load-bearing and silent
What was wrong.
AsyncRedisProviderprovidesRedisMetricsProtocol | NoneasNone.Dishka lets the last provider of a type win, so registering it after a metrics provider
overrode a real metrics implementation with
Noneand produced a plain, uninstrumentedclient with no error anywhere.
How I know.
What I changed. Dishka's last-wins rule is not something this library can change, so I
added the way out:
AsyncRedisProvider(provide_default_metrics=False)registers no defaultand leaves the type to your provider, whatever the order. Registering
AsyncRedisProvider()first still works and is still what the docs show first. The class docstring and the agents
page now both say which orders are safe.
6.
redis_pool_sizewas always 0 on the async clientWhat was wrong.
redis_client_kit/aio/instrumented.pyreadpool._all_connections.How I know. On redis-py 8.1:
getattr(pool, "_all_connections", [])therefore returned[]on every command and thegauge sat at 0. The sync client was already correct.
What I changed. The async client sums
_available_connectionsand_in_use_connectionsthe way the sync one does. Two tests cover it, one of them against a real
redis.asyncio.ConnectionPoolso a future rename in redis-py breaks the test rather thanthe gauge.
7. A translated transport error was counted as a success
What was wrong. In all four instrumented clients the
except RuntimeErrorbranch satahead of the
except Exceptionbranch that setsstatus = "error"and callsrecord_error. A uvloop closed-transportRuntimeErrortranslated into redis-py'sConnectionError— and every otherRuntimeError— was therefore counted inredis_commands_total{status="success"}and never appeared inredis_connection_errors_total.How I know. Reading the branch order, then the test: with the old file in place,
record_erroris called 0 times andrecord_commandgetsstatus="success"for aRuntimeError("… the handler is closed").What I changed. The
RuntimeErrorbranch now records the error it is about to raisebefore raising it — under
ConnectionErrorwhen it translates, underRuntimeErrorwhen itdoes not — and sets
status = "error"either way. The translation itself and the exception acaller sees are unchanged. Six tests, three async and three sync.
8. Smaller things
make test-unitfailed on a clean checkout.make installranuv sync --group dev,which installs no extras, so collection died with
ModuleNotFoundError: No module named 'pydantic'intests/unit/settings/. CI has always used--all-extras; the Makefile andCONTRIBUTING.mdnow say the same thing.mask_redis_kwargswas public by name and in no__all__.redis_client_kit/utils.pyhad no
__all__at all; it now lists its four public helpers.mask_redis_kwargsstays outof the root package, where it has never been and where
docs/agents.mddoes not claim it is.settings/redis.pydeclared__all__ = ["BaseRedisSettings"]while the packagere-exports all seven models, so
from redis_client_kit.settings.redis import RedisSSLSettingsis not an export as far as a strict type checker is concerned. It lists all seven now.
.../reference/api/, which is not a page.The reference is at
.../reference/.Coverage
providers/redis.pyandproviders/utils.pywere on the coverage omit list and had no testsat all — findings 3, 4 and 5 all lived in code nothing exercised. They now have two test
files and came off the omit list; both are at 100% and 96%.
tests/unit/providers/test_providers_init.pyreplaced entries insys.moduleswithoutrestoring them, which left the new provider tests looking at a stale module object when they
ran after it. It uses
monkeypatch.setitem/delitemnow, which restores on teardown. Thesame pattern is still in
tests/unit/settings/test_settings_import_error.pyandtests/unit/metrics/test_metrics_init.py; nothing depends on it today, so I left them alone.docs/agents.md
Rules 15, 16, 17, 19 and 20 stated five of the defects above as rules a caller has to obey,
which is the failure mode that page exists to prevent. They now describe the fixed behaviour.
The Dishka section gained the two constructor arguments, the error table gained the
provider's
ConnectionError, and the caveat under the documentation map — the one warningthat the guide pages' constructor calls were wrong — is gone, because they are not any more.
Verification
uv sync --frozen --all-extras --group dev;uv.lockis byte-identical toorigin/master.Docs build clean:
zensical build --cleanreports "No issues found", andscripts/emit_markdown.pywrites 6 Markdown pages with 1 declined, as before.One thing I did not touch
make docs-build,make docs-serveandmake cleanshell out topython, which does notexist on a machine that only has
python3—make docs-builddies with/bin/sh: python: command not foundbefore it reaches zensical. CI is unaffected becausethe Docs workflow runs
cp CHANGELOG.md docs/changelog.mditself rather than going throughmake. It is unrelated to anything on this list, so it is not in this pull request.