perf(llm): share one TLS context across OpenAI clients - #6763
Conversation
Each OpenAICompletion builds a sync and an async client, and each client built its own SSLContext, parsing the certifi CA bundle from disk. Agent construction paid that parse twice. Build one context per process and pass it to both clients. Constructing 10 agents drops from 102.1 ms to 2.5 ms. Trust anchors, TLS verification, per-instance client config and the interceptor path are unchanged.
📝 WalkthroughWalkthroughChangesOpenAI TLS client configuration
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR reduces OpenAI-backed Agent/LLM construction overhead by reusing a single process-wide TLS ssl.SSLContext instead of rebuilding (and re-parsing the CA bundle for) separate sync and async OpenAI clients.
Changes:
- Introduces a module-level shared SSL context helper for OpenAI client construction.
- Uses OpenAI SDK
DefaultHttpxClient/DefaultAsyncHttpxClientwithverify=_shared_ssl_context()when no customhttp_client(and no interceptor) is provided.
Suppressed comments (1)
lib/crewai/src/crewai/llms/providers/openai/completion.py:84
- New behavior (sharing a single SSLContext across both sync/async OpenAI clients) isn’t covered by tests. A small unit test could patch httpx.create_ssl_context and assert it’s called once when both clients are built, preventing regressions that reintroduce double CA-bundle parsing.
def _shared_ssl_context() -> ssl.SSLContext:
"""Return one process-wide TLS context for OpenAI clients.
``httpx`` builds a fresh context per client, and each one loads the system
CA bundle from disk. Every completion builds two clients, so agent creation
paid that load twice. An ``ssl.SSLContext`` is safe to share across clients.
"""
global _SHARED_SSL_CONTEXT
if _SHARED_SSL_CONTEXT is None:
_SHARED_SSL_CONTEXT = httpx.create_ssl_context()
return _SHARED_SSL_CONTEXT
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| _SHARED_SSL_CONTEXT: ssl.SSLContext | None = None | ||
|
|
||
|
|
||
| def _shared_ssl_context() -> ssl.SSLContext: | ||
| """Return one process-wide TLS context for OpenAI clients. | ||
|
|
||
| ``httpx`` builds a fresh context per client, and each one loads the system | ||
| CA bundle from disk. Every completion builds two clients, so agent creation | ||
| paid that load twice. An ``ssl.SSLContext`` is safe to share across clients. | ||
| """ | ||
| global _SHARED_SSL_CONTEXT | ||
| if _SHARED_SSL_CONTEXT is None: | ||
| _SHARED_SSL_CONTEXT = httpx.create_ssl_context() | ||
| return _SHARED_SSL_CONTEXT |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai/src/crewai/llms/providers/openai/completion.py`:
- Around line 71-84: Protect the check-and-set initialization in
_shared_ssl_context with a module-level lock. Acquire the lock before checking
_SHARED_SSL_CONTEXT and creating it via httpx.create_ssl_context(), while
allowing subsequent calls to reuse the initialized context.
- Around line 324-338: Add synchronous and asynchronous lifecycle cleanup
methods to OpenAICompletion that close the provider-managed _client and
_async_client via client.close() and async_client.aclose(). Ensure cleanup
handles clients created by _build_client and _build_async_client without
affecting externally supplied clients.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c401c2e6-285e-4eaa-89ee-a51e2f0718cc
📒 Files selected for processing (1)
lib/crewai/src/crewai/llms/providers/openai/completion.py
| _SHARED_SSL_CONTEXT: ssl.SSLContext | None = None | ||
|
|
||
|
|
||
| def _shared_ssl_context() -> ssl.SSLContext: | ||
| """Return one process-wide TLS context for OpenAI clients. | ||
|
|
||
| ``httpx`` builds a fresh context per client, and each one loads the system | ||
| CA bundle from disk. Every completion builds two clients, so agent creation | ||
| paid that load twice. An ``ssl.SSLContext`` is safe to share across clients. | ||
| """ | ||
| global _SHARED_SSL_CONTEXT | ||
| if _SHARED_SSL_CONTEXT is None: | ||
| _SHARED_SSL_CONTEXT = httpx.create_ssl_context() | ||
| return _SHARED_SSL_CONTEXT |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== locate file ==="
git ls-files | grep -F 'lib/crewai/src/crewai/llms/providers/openai/completion.py' || true
echo "=== relevant code ==="
if [ -f lib/crewai/src/crewai/llms/providers/openai/completion.py ]; then
nl -ba lib/crewai/src/crewai/llms/providers/openai/completion.py | sed -n '1,130p'
fi
echo "=== references to _shared_ssl_context / _SHARED_SSL_CONTEXT ==="
rg -n "_shared_ssl_context|_SHARED_SSL_CONTEXT|create_ssl_context|get_transport" lib/crewai/src/crewai/llms/providers/openai/completion.py lib/crewai/src/crewai -S || true
echo "=== lock/imports in this file ==="
rg -n 'import threading|threading\.' lib/crewai/src/crewai/llms/providers/openai/completion.py || trueRepository: crewAIInc/crewAI
Length of output: 294
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== deterministic race behavior in Python threading check-then-set ==="
python3 - <<'PY'
import concurrent.futures
import threading
_shared = None
_counter = 0
def _unprotected():
global _shared, _counter
if _shared is None:
_counter += 1
_shared = object()
return _shared
def _locked():
global _shared, _counter
with _lock:
if _shared is None:
_counter += 1
_shared = object()
return _shared
_lock = threading.Lock()
_thread = _shared
with concurrent.futures.ThreadPoolExecutor(max_workers=32) as e:
res_unprotected = [e.submit(_unprotected) for _ in range(32)]
final_unprotected = sum(r.result() is res_unprotected[0].result() for r in res_unprotected)
_shared = _thread
_counter = 0
with concurrent.futures.ThreadPoolExecutor(max_workers=32) as e:
res_locked = [e.submit(_locked) for _ in range(32)]
final_locked = sum(r.result() is res_locked[0].result() for r in res_locked)
print("unprotected identical_contexts_per_run=", final_unprotected, "counter=", _counter)
print("locked_identical_contexts_per_run=", final_locked, "counter=", _counter)
PYRepository: crewAIInc/crewAI
Length of output: 328
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== relevant code ==="
awk '{printf "%6d\t%s\n", NR, $0}' lib/crewai/src/crewai/llms/providers/openai/completion.py | sed -n '1,120p'
echo "=== references to _shared_ssl_context / _SHARED_SSL_CONTEXT / transports ==="
rg -n "_shared_ssl_context|_SHARED_SSL_CONTEXT|create_ssl_context|get_transport|transport" lib/crewai/src/crewai/llms/providers/openai/completion.py lib/crewai/src/crewai -S || true
echo "=== imports/locks in this file ==="
rg -n 'import threading|threading\.' lib/crewai/src/crewai/llms/providers/openai/completion.py || trueRepository: crewAIInc/crewAI
Length of output: 48145
Synchronize shared context initialization.
_shared_ssl_context() uses an unsynchronized check-then-set, so concurrent first-time client creation can run httpx.create_ssl_context() more than once and pass different contexts. Protect initialization with a module-level lock.
Proposed fix
import ssl
+import threading
from typing import TYPE_CHECKING, Any, ClassVar, Literal, TypedDict
...
_SHARED_SSL_CONTEXT: ssl.SSLContext | None = None
+_SHARED_SSL_CONTEXT_LOCK = threading.Lock()
def _shared_ssl_context() -> ssl.SSLContext:
global _SHARED_SSL_CONTEXT
- if _SHARED_SSL_CONTEXT is None:
- _SHARED_SSL_CONTEXT = httpx.create_ssl_context()
- return _SHARED_SSL_CONTEXT
+ with _SHARED_SSL_CONTEXT_LOCK:
+ if _SHARED_SSL_CONTEXT is None:
+ _SHARED_SSL_CONTEXT = httpx.create_ssl_context()
+ return _SHARED_SSL_CONTEXT📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| _SHARED_SSL_CONTEXT: ssl.SSLContext | None = None | |
| def _shared_ssl_context() -> ssl.SSLContext: | |
| """Return one process-wide TLS context for OpenAI clients. | |
| ``httpx`` builds a fresh context per client, and each one loads the system | |
| CA bundle from disk. Every completion builds two clients, so agent creation | |
| paid that load twice. An ``ssl.SSLContext`` is safe to share across clients. | |
| """ | |
| global _SHARED_SSL_CONTEXT | |
| if _SHARED_SSL_CONTEXT is None: | |
| _SHARED_SSL_CONTEXT = httpx.create_ssl_context() | |
| return _SHARED_SSL_CONTEXT | |
| _SHARED_SSL_CONTEXT: ssl.SSLContext | None = None | |
| _SHARED_SSL_CONTEXT_LOCK = threading.Lock() | |
| def _shared_ssl_context() -> ssl.SSLContext: | |
| """Return one process-wide TLS context for OpenAI clients. | |
| ``httpx`` builds a fresh context per client, and each one loads the system | |
| CA bundle from disk. Every completion builds two clients, so agent creation | |
| paid that load twice. An ``ssl.SSLContext`` is safe to share across clients. | |
| """ | |
| global _SHARED_SSL_CONTEXT | |
| with _SHARED_SSL_CONTEXT_LOCK: | |
| if _SHARED_SSL_CONTEXT is None: | |
| _SHARED_SSL_CONTEXT = httpx.create_ssl_context() | |
| return _SHARED_SSL_CONTEXT |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/crewai/src/crewai/llms/providers/openai/completion.py` around lines 71 -
84, Protect the check-and-set initialization in _shared_ssl_context with a
module-level lock. Acquire the lock before checking _SHARED_SSL_CONTEXT and
creating it via httpx.create_ssl_context(), while allowing subsequent calls to
reuse the initialized context.
| elif "http_client" not in client_config: | ||
| client_config["http_client"] = DefaultHttpxClient( | ||
| verify=_shared_ssl_context() | ||
| ) | ||
| return OpenAI(**client_config) | ||
|
|
||
| def _build_async_client(self) -> Any: | ||
| client_config = self._get_client_params() | ||
| if self.interceptor: | ||
| transport = AsyncHTTPTransport(interceptor=self.interceptor) | ||
| client_config["http_client"] = httpx.AsyncClient(transport=transport) | ||
| elif "http_client" not in client_config: | ||
| client_config["http_client"] = DefaultAsyncHttpxClient( | ||
| verify=_shared_ssl_context() | ||
| ) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
provider='lib/crewai/src/crewai/llms/providers/openai/completion.py'
rg -n -C 8 \
'Default(Httpx|AsyncHttpx)Client|def (close|aclose|__del__)|_client|_async_client' \
"$provider"
fd -i -t f 'base_llm.py' . \
-x rg -n -C 8 'def (close|aclose|__del__)' {}Repository: crewAIInc/crewAI
Length of output: 13915
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
provider='lib/crewai/src/crewai/llms/providers/openai/completion.py'
printf 'OpenAI provider lifecycle/cleanup symbols:\n'
rg -n -C 5 \
'class OpenAICompletion|def (close|aclose|__del__|__enter__|__exit__|__aenter__|__aexit__)|del _client|del _async_client|_client|_async_client' \
"$provider" || true
printf '\nBase LLM close/aclose/__del__ symbols across repo:\n'
fd -i -t f -x 'base_llm' . | sed 's#/proc/.*##' | while read -r f; do
if rg -n -C 5 'def (close|aclose|__del__)|__del__|self\.close|async def close' "$f" >/tmp/llm_close_hits.txt 2>/tmp/llm_close_err.txt; then
echo "=== $f ==="
cat /tmp/llm_close_hits.txt
fi
done
printf '\nInstalled SDK version (if present):\n'
python3 - <<'PY'
try:
import openai
from inspect import isclass
print('openai version:', getattr(openai, '__version__', None))
for name in ['SyncHttpxClientWrapper', 'AsyncHttpxClientWrapper', 'DefaultHttpxClient', 'DefaultAsyncHttpxClient']:
mod = openai._base_client
val = getattr(mod, name, None)
print(name, type(val), val)
if val is not None:
for sym in dir(val):
if sym.startswith('_') and sym.endswith('__') and callable(getattr(val, sym, None)):
print(f' {name}.{sym}')
print(' methods:', [m for m in dir(val) if not m.startswith('_')])
print()
except Exception as e:
print(type(e).__name__, e)
PY
printf '\nInstalled source relevant wrappers (if present):\n'
python3 - <<'PY'
try:
import inspect
import openai._base_client as mod
for name in ['SyncHttpxClientWrapper', 'AsyncHttpxClientWrapper']:
cls = getattr(mod, name)
print(f'--- {name} ---')
print(inspect.getfile(cls))
for symbol in ['close', 'aclose', '__del__', '__enter__', '__exit__', '__aenter__', '__aexit__']:
try:
src = inspect.getsource(getattr(cls, symbol))
except Exception as e:
src = f'RETrie {e}'
print(src[:1800])
except Exception as e:
print(type(e).__name__, e)
PYRepository: crewAIInc/crewAI
Length of output: 50373
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
provider='lib/crewai/src/crewai/llms/providers/openai/completion.py'
base_llm='lib/crewai/src/crewai/llms/base_llm.py'
printf 'Files present:\n'
[ -f "$provider" ] && echo "provider: yes" || echo "provider: no"
[ -f "$base_llm" ] && echo "base_llm: yes" || echo "base_llm: no"
printf '\nopenai/completion.py lifecycle/cleanup symbols:\n'
rg -n -C 3 \
'class OpenAICompletion|def (close|aclose|__del__|__enter__|__exit__|__aenter__|__aexit__)|del _client|del _async_client|_client|_async_client' \
"$provider" || true
printf '\nbase_llm.py lifecycle methods/cleanup-ish symbols:\n'
if [ -f "$base_llm" ]; then
rg -n -C 5 \
'class BaseLLM|def (close|aclose|__del__|__init__|__enter__|__exit__|__aenter__|__aexit__)' \
"$base_llm" || true
else
echo "base_llm.py not found"
fi
printf '\nInstalled OpenAI version and wrapper details:\n'
python3 - <<'PY'
import inspect
try:
import openai
print(f'openai.__version__={getattr(openai, "__version__", None)}')
except Exception as e:
print(f'import error: {type(e).__name__} {e}')
raise SystemExit
for name in ['SyncHttpxClientWrapper', 'AsyncHttpxClientWrapper']:
cls = getattr(openai._base_client, name, None)
print(f'--- {name} ---')
print(cls)
print('location:', getattr(cls, '__module__', '?'))
for sym in ['close', 'aclose', '__del__', '__enter__', '__exit__', '__aenter__', '__aexit__']:
value = getattr(cls, sym, None)
print(f'{sym}: {value}')
if value is not None:
try:
src = inspect.getsource(value)
except Exception as s:
src = f'<uninspectable: {type(s).__name__} {s}>'
print(src[:2200])
print()
PYRepository: crewAIInc/crewAI
Length of output: 7461
🌐 Web query:
OpenAI Python v2.30.0 Raw GitHub SyncHttpxClientWrapper AsyncHttpxClientWrapper DefaultHttpxClient DefaultAsyncHttpxClient
💡 Result:
In the OpenAI Python library v2.30.0, the classes SyncHttpxClientWrapper, AsyncHttpxClientWrapper, DefaultHttpxClient, and DefaultAsyncHttpxClient are components of the library's internal HTTP client architecture [1][2]. DefaultHttpxClient and DefaultAsyncHttpxClient: These are the default HTTP client implementations used by the library for synchronous and asynchronous requests, respectively [2][3]. They are pre-configured with specific defaults—such as custom timeouts, connection limits, and redirect behavior—that are optimized for the OpenAI API [2][3]. When initializing an OpenAI client, you are encouraged to use these classes if you need to provide a custom HTTP client (e.g., to configure proxies or custom transports), as doing so ensures the SDK's recommended network settings are preserved [4][5][6]. SyncHttpxClientWrapper and AsyncHttpxClientWrapper: These are internal wrapper classes that inherit from DefaultHttpxClient and DefaultAsyncHttpxClient, respectively [1][2]. Their primary purpose is to manage resource cleanup automatically [2]. Specifically, they implement a del method to ensure that the underlying httpx client is closed if it hasn't been explicitly closed by the user [1][2]. The AsyncHttpxClientWrapper has been the subject of technical discussions regarding its destructor's behavior when managing asyncio tasks [7]. In summary, DefaultHttpxClient and DefaultAsyncHttpxClient are the public-facing classes for user configuration, while SyncHttpxClientWrapper and AsyncHttpxClientWrapper are internal utilities designed to handle lifecycle management and resource cleanup [1][2][3].
Citations:
- 1: https://github.com/openai/openai-python/blob/6ba31bcb/src/openai/_base_client.py
- 2: https://deepwiki.com/openai/openai-python/3.2-http-client-foundation
- 3: https://deepwiki.com/openai/openai-python/7.4-custom-http-clients-and-proxies
- 4: https://pypi.org/project/openai/2.30.0/
- 5: https://developers.openai.com/api/reference/python/
- 6: https://openai-openai-python-73.mintlify.app/concepts/client
- 7: AsyncHttpxClientWrapper creates task in destructor -> prevents usage in other threads openai/openai-python#2440
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
url='https://raw.githubusercontent.com/openai/openai-python/v2.30.0/src/openai/_base_client.py'
tmp=/tmp/openai_v2.30.0_base_client.py
printf '%s\n' "$url"
curl -fsSL "$url" -o "$tmp" || curl -S "$url" -o "$tmp"
python3 - <<'PY' "$tmp"
import ast, sys, textwrap
path = sys.argv[1]
src = open(path, encoding="utf-8").read()
tree = ast.parse(src)
for node in tree.body:
if isinstance(node, (ast.ClassDef, ast.FunctionDef)) and node.name in {
"SyncHttpxClientWrapper", "AsyncHttpxClientWrapper",
"DefaultHttpxClient", "DefaultAsyncHttpxClient",
"_ensure_default_client", "_ensure_default_async_client",
}:
print(f"--- {node.name} lines {node.lineno}-{node.end_lineno} ---")
lines = src.splitlines()
print("\n".join(textwrap.indent(lines[i-1], '| ') for i in range(node.lineno, node.end_lineno+1)))
PYRepository: crewAIInc/crewAI
Length of output: 899
Close the provider-managed OpenAI clients during teardown.
OpenAICompletion creates and stores _client and _async_client here, but the class has no close/aclose lifecycle hooks. The explicit DefaultHttpxClient and DefaultAsyncHttpxClient instances do not get the wrapper destructor cleanup that is only added by SyncHttpxClientWrapper/AsyncHttpxClientWrapper, so discarded providers can retain HTTPX connection pools. Add provider lifecycle cleanup that calls client.close() and async_client.aclose().
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/crewai/src/crewai/llms/providers/openai/completion.py` around lines 324 -
338, Add synchronous and asynchronous lifecycle cleanup methods to
OpenAICompletion that close the provider-managed _client and _async_client via
client.close() and async_client.aclose(). Ensure cleanup handles clients created
by _build_client and _build_async_client without affecting externally supplied
clients.
Source: MCP tools
What this changes
OpenAICompletionbuilds both a sync and an async client in itsmodel_validator(mode="after"). Each client gets its ownssl.SSLContext, and building one parses the whole certifi CA bundle off disk. So everyAgentbacked by an OpenAI-provider LLM pays that parse twice before it has done any work.An
ssl.SSLContextis safe to share between clients, so this builds one per process and hands it to both.Measurements
Constructing 10
Agentobjects, Linux, CPython 3.13, certifi bundle with 119 anchors. Each number is the median of 7 rounds, and I take the worse of two independent passes so a lucky run can't flatter it:cProfileover 40 constructions onmainputs 83% ofAgent()inside_ssl._SSLContext.load_verify_locations, entered twice per agent.What this does not do
This is construction cost, nothing else. No API call gets faster, and a crew that spends its time waiting on the model will not notice. It shows up where agents actually get built: test suites, CLI start, serverless cold start, and services that construct an agent per request. On a box with a smaller CA bundle the saving is smaller.
Behaviour
httpx.create_ssl_context()returns the certifi bundle these clients already used. Moving tossl.create_default_context()would switch to the OS store and quietly change which CAs are trusted, so I did not do that.LLMstill gets its own client objects. Only the context is shared.DefaultHttpxClientandDefaultAsyncHttpxClientare the SDK's own classes, sotimeout,max_retriesand the connection limits keep the values they have today.http_clientinclient_paramsstill wins, and the interceptor path is untouched.Checks
uv run pytest lib/crewai/tests/llms/openai/ -qgives 150 passed, 1 skippeduv run pytest lib/crewai/tests/llms/ -qgives 594 passed, 20 skippeduv run ruff check,uv run ruff format --checkanduv run mypyare clean on the fileThe anthropic, azure and bedrock providers construct clients the same way. I left them alone to keep this to one change, and can follow up if you want it.
Search trajectory behind the change: https://dashboard.weco.ai/share/YEYglLiWzwTkOBvMc9b-oSTyrVi6RN8a