around_call: a ContextVar token cannot cross the yield on three of four RPC kinds, and a failing teardown replaces a successful response

[AlexeyShalaev]

Measured on 0.1.1 with grpcio 1.83.1, against an in-process server with all four RPC kinds. The interceptor is the shortest thing anyone would write with this seam, a request id on the outgoing call:

class TokenAcrossYield(AsyncAroundClientInterceptor):
    async def around_call(self, call: ClientCall):
        token = REQUEST_ID.set("req-42")
        try:
            yield
        finally:
            REQUEST_ID.reset(token)

--- an around_call that sets a ContextVar before the yield and resets it after
    unary_unary    caller got the response
    unary_stream   caller got ValueError: <Token ...> was created in a different Context
    stream_unary   caller got the response
        loop exception handler: Task exception was never retrieved: ValueError("<Token ...> was created in a different Context")
    stream_stream  caller got ValueError: <Token ...> was created in a different Context
--- the same interceptor on a stream the server fails mid-way
    unary_stream   caller got UNAVAILABLE: died at item 2
        loop exception handler: Task exception was never retrieved: ValueError("<Token ...>")
--- any teardown that raises, per RPC kind
    unary_unary    caller got RuntimeError: the teardown itself failed
    unary_stream   caller got RuntimeError: the teardown itself failed
    stream_unary   caller got the response
        loop exception handler: Task exception was never retrieved: RuntimeError('the teardown itself failed')
    stream_stream  caller got RuntimeError: the teardown itself failed

Two things are wrong here, and they compound.

The teardown runs in a different context than the setup. For a response stream _run_rpc closes the around context from _closing_stream, in whichever task drains the stream; for a streaming request it closes it from _spawn_background. contextvars refuses a reset of a token minted in another context, so the pair that is the normal way to scope anything for the duration of a call — a request id, a correlation id, an OpenTelemetry context — cannot be written against this seam for three of the four RPC kinds. AsyncClientTracingInterceptor already knows this: its docstring says the span is made current only while the call is created because "detaching the OpenTelemetry context there fails". That warning lives on the one interceptor that hit it, while AsyncAroundClientInterceptor's own docstring, the interceptor rows of the API table and rule 13 of the agents page all describe around_call as code that runs before and after the whole RPC, with nothing about the context boundary.

A teardown failure changes the outcome of a call that already finished. The last block is the general case, with no contextvars in it: an exception from anything after the yield reaches the caller as that exception for unary-unary, unary-stream and stream-stream — a call the server answered OK is delivered to application code as RuntimeError — and for stream-unary it is swallowed into the loop's exception handler. Interceptors are observability; a metrics push or a log write that fails must not take the response with it, and the same mistake must not have four different outcomes.

What I think it needs, in this order:

1. Teardown exceptions never change the call's outcome. Whatever the around generator raises after the yield is logged at ERROR with the method on it and dropped, in all four paths — same as an after_* failure would be treated in any observability chain. An exception raised before the yield keeps its current meaning: refusing the call.
2. Setup and teardown in the same context, if it can be had. asyncio.Task takes context= since 3.11, so the finalizer could close the around context in the very contextvars.Context the setup ran in, which would make the token pair above work on every kind. If that turns out to be wrong for the streaming paths (the teardown deliberately runs in the consumer's task so a cancellation is seen), then say so instead: rule 13 and the AsyncAroundClientInterceptor docstring both need the sentence "the code after the yield may run in a different context than the code before it, so do not carry a ContextVar token or an OpenTelemetry context across the yield", plus the pattern that does work.
3. A test per RPC kind for both, since this is exactly the class of bug that shows up in one kind and not the others.

Lab: probe.py and interceptors_lab.py in https://github.com/bedrock-python/bedrock-python.github.io/tree/docs/production-python-series/docs/blog/lab/2026-09-07-grpc-interceptors-and-streams.

[AlexeyShalaev]

Reproduced both blocks on 0.1.1 with grpcio 1.83.1, running your probe.py unchanged against an editable checkout — same three ValueErrors, same four different outcomes for one raising teardown. Thank you for the lab; having the four kinds side by side is what makes the second half of this obvious.

What is actually wrong

One around_call generator, four places its teardown is stepped from:

kind	teardown runs in
unary-unary	the same coroutine as the setup — the interceptors task
unary-stream, stream-stream	_closing_stream, i.e. whichever task drains the response
stream-unary	_spawn_background(...), an ensure_future task whose context is a copy

contextvars compares Context objects by identity, so every row but the first refuses reset. And because the teardown sits in three different callers, an exception out of it has three different fates: it replaces the response, it replaces the stream's last __anext__, or it lands in the loop's exception handler. Both symptoms are the same root cause — the generator has no scope of its own — so I want to fix them in one place rather than patch four call sites.

What I am going to do

1. A teardown never changes the outcome. The __aexit__ moves behind a single private object that owns the generator and closes it exactly once. Anything the generator raises after the yield is logged at ERROR with the method on it and dropped, for all four kinds, in that one place. CancelledError, KeyboardInterrupt and SystemExit keep propagating — a cancelled task is not an interceptor bug, and the kit already draws that line in the tracing layer. Raising before the yield keeps meaning "refuse the call", untouched.

2. Setup and teardown in one context, where they are not already. asyncio.Task(context=...) enters the Context object it is given rather than copying it, so the same object can be entered by the setup task and later by the teardown task; I measured it, the token pair survives. For unary-stream, stream-unary and stream-stream the scope therefore takes a copy_context() of its own and steps the setup, the RPC creation and the teardown in it.

The RPC creation has to be inside it too: pinning only __aenter__/__aexit__ would make the token legal and the value invisible to every layer below, which is the whole point of setting it. That also matters for what I am not doing — unary-unary keeps running with no task at all, because its setup, call and teardown are already one coroutine and therefore already one context. A task per call per around layer on the hottest path in the kit, to buy behaviour that is already correct, is not a trade I want to make. The four kinds end up observationally identical; only unary-unary gets there for free.

What I rejected

• Documenting the constraint instead (your option 2b). Cheapest, and I would take it if the mechanism did not exist — but it does, and a ContextVar set/reset pair is the canonical use of a seam like this. Documenting "you cannot do the obvious thing" is the answer when there is no other answer.
• Pinning a context for all four kinds, for symmetry. Uniform in the source, and it costs every unary-unary call an extra task per around layer. The canonical chain has four of them.
• Swallowing teardown errors only where they reach the caller. Leaves stream-unary dropping them into the loop handler, which is the "four outcomes for one mistake" you are objecting to.
• Context.run without a task. It cannot drive a coroutine; anything built on it is a hand-rolled Task.

Tests

Both behaviours, per RPC kind, against the live server bench the repo already parametrizes over all four — plus the seam-level unit tests. Each fails on master: the contextvars ones with the ValueError you quoted, the teardown ones by the response never arriving (or, for stream-unary, by nothing being logged).

The tracing interceptor

Its docstring says around_call is unusable because "detaching the OpenTelemetry context there fails". After this that reason is gone, and the Note has to say so — a stale rule on the one layer that hit the problem is how this stayed invisible. Whether AsyncClientTracingInterceptor should then be rewritten onto around_call is a separate question with its own risk, and I would rather not answer it in the same PR that changes the seam under it. I will correct the docstring and say why the implementation stays.

Docs: the AsyncAroundClientInterceptor docstring and the module preamble, the "After it" bullet in the interceptors guide, and the agents page — the interceptor row, plus a rule of its own for the around contract, since neither of the two guarantees has anywhere to live today.

[AlexeyShalaev]

Shipped in #20, both items, as one change: they were one bug. A single around_call generator had no scope of its own, so it was being stepped from four different places, and everything else follows from that.

_AroundScope owns the generator now.

Teardown exceptions. __aexit__ happens in exactly one place, which logs whatever it raises at ERROR against the method and drops it — same treatment on all four kinds, whether the call succeeded or failed. CancelledError, KeyboardInterrupt and SystemExit still propagate; a task being torn down around the teardown is not the teardown failing, and swallowing that would strand a cancel your code asked for. Raising before the yield is untouched: it still refuses the call.

One context across the yield. asyncio.Task(context=...) enters the context object it is handed rather than copying it, so for the three kinds whose teardown finishes in another task the scope pins one contextvars.Context and steps the setup, the RPC creation and the teardown in it. The RPC creation is inside it on purpose — pinning only the two __a*__ calls would have made your token pair legal and made what it sets invisible to every layer below, which is the point of setting it. Unary-unary pins nothing and pays nothing: its three phases are one coroutine already, and they were never the problem.

Your probe.py, unmodified, against the branch:

--- an around_call that sets a ContextVar before the yield and resets it after
    unary_unary    caller got the response
    unary_stream   caller got the response
    stream_unary   caller got the response
    stream_stream  caller got the response
--- the same interceptor on a stream the server fails mid-way
    unary_stream   caller got UNAVAILABLE: died at item 2
--- any teardown that raises, per RPC kind
    unary_unary    caller got the response       (+ ERROR: around_call teardown failed for /lab.Reports/Get)
    unary_stream   caller got the response       (+ ERROR: around_call teardown failed for /lab.Reports/Stream)
    stream_unary   caller got the response       (+ ERROR: around_call teardown failed for /lab.Reports/Upload)
    stream_stream  caller got the response       (+ ERROR: around_call teardown failed for /lab.Reports/Chat)

No loop-exception-handler lines left, and the mid-stream failure still arrives as the status it is.

Nothing to change on your side. TokenAcrossYield as you wrote it is now a supported interceptor on every kind, and so is the OpenTelemetry attach / detach version of it. What is worth knowing is the price, which is documented rather than hidden: the three pinned kinds cost an asyncio.Task or two per call, per around layer. If you have a layer where that matters and which never touches the outcome — one that only rewrites the call details — intercept remains the cheaper seam, as the context and timeout layers already use it.

Your point 3 is covered twice: per kind at the seam in tests/unit/interceptors/test_base.py, driving the chain from a task of its own because that boundary is the entire bug, and per kind against a live server in tests/integration/test_around_call.py. Against master's base.py they fail 8 of 9 and 7 of 8 respectively — the single case that passes either way is contextvars on unary-unary, which is the one row that already worked in your table.

AsyncClientTracingInterceptor you were right to point at. Its reason for avoiding the seam is no longer true and the docstring said so as if it were permanent; it now states the reason that does hold, which is cost rather than correctness — with opentelemetry-api installed and no SDK configured the layer does nothing at all, while an around scope would be opened per call regardless. Rewriting it onto around_call is a separate change and I did not want it under the same diff as the seam it sits on.

Docs: docs/agents.md has the seam's table row, the paragraph under it and rule 14 for the two guarantees (14–20 shift down by one); the interceptors guide has the corrected "after the yield" bullet and a Scoping a value to one call section with the pattern and its cost; the README and the base module preamble say the same in fewer words. This closes when #20 merges.