feat(llm): Add A3M Router for intelligent model routing - #6794
Conversation
Add A3M Router as a supported LLM provider in CrewAI:
Changes:
- Added 'a3m' to OPENAI_COMPATIBLE_PROVIDERS for quick setup
- Created dedicated A3MCompletion provider class
- Added example usage in examples/a3m_router_example.py
Key Benefits:
- Automatic model selection based on task complexity
- 70-95% cost savings vs single-model setups
- Support for 47+ LLM providers via A3M
- Built-in fallback handling
Usage:
from crewai.llms import A3MCompletion
agent = Agent(llm=A3MCompletion(model='auto'))
This addresses the gap left by LiteLLM CVE issues and provides
a cost-effective alternative for multi-agent orchestration.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds A3M Router support to CrewAI through a new provider, OpenAI-compatible configuration, public exports, and examples for basic, multi-agent, and cost-tracking workflows. ChangesA3M Router integration
Sequence Diagram(s)sequenceDiagram
participant ExampleScript
participant CrewAI
participant A3MCompletion
participant A3MRouterAPI
ExampleScript->>CrewAI: create agents and tasks
CrewAI->>A3MCompletion: execute routed task
A3MCompletion->>A3MRouterAPI: send completion request
A3MRouterAPI-->>A3MCompletion: return routed response
A3MCompletion-->>CrewAI: return completion result
CrewAI-->>ExampleScript: print result
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/a3m_router_example.py`:
- Around line 101-111: Update example_multi_agent() so each Task—research_task,
writing_task, and review_task—defines an expected_output, and configure the
hierarchical Crew with manager_llm using A3MCompletion(model="auto") or provide
a separate manager_agent. Preserve the existing agents, task assignments, and
hierarchical process.
In `@lib/crewai/src/crewai/llms/providers/__init__.py`:
- Around line 3-5: Expose A3MCompletion from the documented crewai.llms import
path and use that path consistently: update
lib/crewai/src/crewai/llms/providers/__init__.py lines 3-5 to re-export it
through crewai.llms, then adjust the import in
lib/crewai/src/crewai/llms/providers/a3m/completion.py lines 6-10 and both
references in examples/a3m_router_example.py lines 13-15 and 34-35 to match the
selected public path.
In `@lib/crewai/src/crewai/llms/providers/a3m/completion.py`:
- Around line 71-81: Update A3M cost reporting by tracking actual router usage
in the provider implementation and returning those accumulated totals from
get_cost(), including request counts and per-model costs; alternatively remove
get_cost() and the dependent cost-tracking example until metrics are
implemented.
- Around line 47-57: The A3MCompletion initialization bypasses shared provider
configuration by inheriting from OpenAICompletion. Change A3MCompletion to
subclass OpenAICompatibleCompletion, pass the resolved model, base URL, and API
key through kwargs rather than hardcoded defaults, and set provider="a3m" so
_resolve_provider_config applies A3M_API_KEY, A3M_BASE_URL, and the registered
A3M configuration.
- Around line 12-15: Update the quick-start construction around Agent and Task
so Task is instantiated with concrete description and expected_output values
before Agent is created, then pass that Task instance to Agent instead of
referencing the undefined task variable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 92d00237-74c6-476c-9086-9d922ece6482
📒 Files selected for processing (5)
examples/a3m_router_example.pylib/crewai/src/crewai/llms/providers/__init__.pylib/crewai/src/crewai/llms/providers/a3m/__init__.pylib/crewai/src/crewai/llms/providers/a3m/completion.pylib/crewai/src/crewai/llms/providers/openai_compatible/completion.py
| from crewai.llms.providers.a3m import A3MCompletion | ||
|
|
||
| __all__ = ["A3MCompletion"] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Export A3MCompletion from the documented import path.
CrewAI 1.15.10's crewai.llms package exports no provider symbols. This change exports only crewai.llms.providers.A3MCompletion, so every from crewai.llms import A3MCompletion statement fails. (raw.githubusercontent.com)
lib/crewai/src/crewai/llms/providers/__init__.py#L3-L5: Also exportA3MCompletionfromcrewai.llms, or selectcrewai.llms.providersas the public path.lib/crewai/src/crewai/llms/providers/a3m/completion.py#L6-L10: Match the selected public import path.examples/a3m_router_example.py#L13-L15: Match the selected public import path in documentation.examples/a3m_router_example.py#L34-L35: Match the selected public import path in executable code.
📍 Affects 3 files
lib/crewai/src/crewai/llms/providers/__init__.py#L3-L5(this comment)lib/crewai/src/crewai/llms/providers/a3m/completion.py#L6-L10examples/a3m_router_example.py#L13-L15examples/a3m_router_example.py#L34-L35
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/crewai/src/crewai/llms/providers/__init__.py` around lines 3 - 5, Expose
A3MCompletion from the documented crewai.llms import path and use that path
consistently: update lib/crewai/src/crewai/llms/providers/__init__.py lines 3-5
to re-export it through crewai.llms, then adjust the import in
lib/crewai/src/crewai/llms/providers/a3m/completion.py lines 6-10 and both
references in examples/a3m_router_example.py lines 13-15 and 34-35 to match the
selected public path.
| agent = Agent( | ||
| task=Task(task), | ||
| llm=llm | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== file exists and relevant lines =="
if [ -f lib/crewai/src/crewai/llms/providers/a3m/completion.py ]; then
nl -ba lib/crewai/src/crewai/llms/providers/a3m/completion.py | sed -n '1,120p'
else
echo "file not found"
fi
echo
echo "== dependency pins mentioning crewai versions =="
rg -n "crewai|crewAI|1\.15\.10" pyproject.toml uv.lock poetry.lock requirements*.txt setup.py setup.cfg . 2>/dev/null | head -n 200 || true
echo
echo "== local agent/task definitions if available =="
fd -a 'task.py|agent.py' lib/crewai/src 2>/dev/null | sed -n '1,80p' || true
if [ -f lib/crewai/src/crewai/task.py ]; then
nl -ba lib/crewai/src/crewai/task.py | sed -n '1,220p'
fiRepository: crewAIInc/crewAI
Length of output: 231
🌐 Web query:
crewAI 1.15.10 Task expected_output required
💡 Result:
In crewAI version 1.15.10 (as in all versions of the framework), the expected_output parameter is a required attribute when defining a Task [1][2][3]. The expected_output parameter is defined as a string that provides a clear, detailed description of what a successful task completion looks like [1][4][3]. This parameter is critical because it is injected into the agent's prompt, serving as the primary instruction for the LLM to understand when the task is complete and what format or structure the final output should take [3]. Key points regarding expected_output: - Required: It must be provided for every Task instance [1][2]. - Function: It acts as the definition of success for the AI agent [3]. - Type: It is a string (str) [1]. - Implementation: It is defined as a field within the Task class [4][5]. Example usage: task = Task( description="Research the latest trends in AI.", expected_output="A list of 5 bullet points summarizing the most important trends.", agent=researcher)
Citations:
- 1: https://docs.crewai.com/en/concepts/tasks
- 2: https://docs.crewai.com/v1.15.6/en/concepts/tasks
- 3: https://learnixo.io/blog/crew-task-definition
- 4: https://github.com/crewAIInc/crewAI/blob/main/lib/crewai/src/crewai/task.py
- 5: https://github.com/crewAIInc/crewAI/blob/84d57c7a/lib/crewai/src/crewai/task.py
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== candidate files =="
git ls-files | awk '
/^lib\/crewai\/src\/crewai\/llms\/providers\/a3m\/completion\.py$/ ||
/(^|\/)(pyproject\.toml|uv\.lock|poetry\.lock|requirements.*\.txt|setup\.py|setup\.cfg)$/'
echo
echo "== relevant completion.py excerpt =="
python3 - <<'PY'
from pathlib import Path
p = Path("lib/crewai/src/crewai/llms/providers/a3m/completion.py")
if p.exists():
for i, line in enumerate(p.read_text().splitlines(), 1):
if 1 <= i <= 40:
print(f"{i:4}: {line}")
else:
print("file not found")
PY
echo
echo "== local task.py excerpt if present =="
python3 - <<'PY'
from pathlib import Path
p = Path("lib/crewai/src/crewai/task.py")
if p.exists():
for i, line in enumerate(p.read_text().splitlines(), 1):
if 1 <= i <= 220:
print(f"{i:4}: {line}")
else:
print("file not found")
PY
echo
echo "== crewAI dependency mentions =="
python3 - <<'PY'
patterns = ["crewai", "crewAI", "1.15.10"]
for root in ["."]:
for p in Path(root).rglob("*"):
if p.is_file() and p.name in {"pyproject.toml","uv.lock","poetry.lock","requirements.txt","setup.py","setup.cfg"}:
try:
txt = p.read_text(errors="ignore")
except Exception:
continue
hits = [i+1 for i,l in enumerate(txt.splitlines()) if any(x.lower() in l.lower() for x in patterns)]
if hits:
print(f"— {p}")
for i in hits[:5]:
print(f" {i}: {txt.splitlines()[i-1]}")
PYRepository: crewAIInc/crewAI
Length of output: 11294
Make the quick-start code executable.
The example uses Task(task) before task is defined, and Task.expected_output is required. Create the Task with concrete values such as a description and expected_output, then pass that instance to Agent.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/crewai/src/crewai/llms/providers/a3m/completion.py` around lines 12 - 15,
Update the quick-start construction around Agent and Task so Task is
instantiated with concrete description and expected_output values before Agent
is created, then pass that Task instance to Agent instead of referencing the
undefined task variable.
Source: Coding guidelines
| # A3M Router default settings | ||
| default_kwargs: Dict[str, Any] = { | ||
| "model": model, | ||
| "base_url": "http://localhost:8787/v1", | ||
| "api_key": kwargs.get("api_key", "not-needed"), | ||
| } | ||
|
|
||
| # Override with any user-provided kwargs | ||
| default_kwargs.update(kwargs) | ||
|
|
||
| super().__init__(**default_kwargs) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate files =="
git ls-files | rg 'llms/providers/(a3m|openai)/completion\.py|llms/providers.*(a3m|openai|compatible)' || true
echo "== a3m completion =="
if [ -f lib/crewai/src/crewai/llms/providers/a3m/completion.py ]; then
cat -n lib/crewai/src/crewai/llms/providers/a3m/completion.py
fi
echo "== openai completion relevant =="
if [ -f lib/crewai/src/crewai/llms/providers/openai/completion.py ]; then
wc -l lib/crewai/src/crewai/llms/providers/openai/completion.py
sed -n '1,260p' lib/crewai/src/crewai/llms/providers/openai/completion.py | cat -n
fi
echo "== searches for provider/config symbols =="
rg -n "OPENAI_COMPATIBLE_PROVIDERS|OpenAICompatibleCompletion|_resolve_provider_config|A3M_API_KEY|A3M_BASE_URL|a3m|provider" lib/crewai/src/crewai/llms -S || trueRepository: crewAIInc/crewAI
Length of output: 30249
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== openai base_llm model validators/defaults =="
sed -n '250,330p' lib/crewai/src/crewai/llms/providers/openai/completion.py | cat -n
sed -n '888,918p' lib/crewai/src/crewai/llms/providers/openai/completion.py | cat -n
sed -n '250,290p' lib/crewai/src/crewai/llms/base_llm.py | cat -n
echo "== openai compatible class =="
sed -n '1,270p' lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py | cat -n
sed -n '270,295p' lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py | cat -n
echo "== a3m config =="
sed -n '78,110p' lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py | cat -n
echo "== static call graph / imports =="
rg -n "def __init__|model_validator|BaseLLM|OpenAICompatibleCompletion|OPENAI_COMPATIBLE_PROVIDERS|A3MCompletion" lib/crewai/src/crewai/llms/providers/a3m lib/crewai/src/crewai/llms/providers/openai lib/crewai/src/crewai/llms/providers/openai_compatible lib/crewai/src/crewai/llms/base_llm.py -SRepository: crewAIInc/crewAI
Length of output: 21607
Use OpenAICompatibleCompletion so A3M config is applied.
A3MCompletion inherits from OpenAICompletion, so _resolve_provider_config never runs. Lines 50-51 therefore ignore A3M_API_KEY, A3M_BASE_URL, and OPENAI_COMPATIBLE_PROVIDERS["a3m"]. Subclass OpenAICompatibleCompletion, pass explicit values through kwargs, and set provider="a3m" so the shared config validator uses the registered A3M configuration.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/crewai/src/crewai/llms/providers/a3m/completion.py` around lines 47 - 57,
The A3MCompletion initialization bypasses shared provider configuration by
inheriting from OpenAICompletion. Change A3MCompletion to subclass
OpenAICompatibleCompletion, pass the resolved model, base URL, and API key
through kwargs rather than hardcoded defaults, and set provider="a3m" so
_resolve_provider_config applies A3M_API_KEY, A3M_BASE_URL, and the registered
A3M configuration.
The get_cost() method was returning hardcoded zeros, misleading users who expected actual cost tracking. A3M Router's built-in analytics at localhost:8787 provides cost tracking instead. Also removed "70-95% cost savings" claim which lacked evidence.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
lib/crewai/src/crewai/llms/providers/a3m/completion.py (1)
26-56:⚠️ Potential issue | 🟠 MajorRoute A3M through the shared compatible-provider configuration.
A3MCompletionstill inherits fromOpenAICompletion. This bypassesOpenAICompatibleCompletion._resolve_provider_config(). The registereda3msettings cannot provideA3M_API_KEYorA3M_BASE_URL, so the class remains tied to the hard-coded endpoint and fallback key unless every caller overrides them. Inherit fromOpenAICompatibleCompletionand setprovider="a3m"in the shared initialization path.#!/bin/bash set -euo pipefail rg -n \ 'class A3MCompletion|class OpenAICompatibleCompletion|_resolve_provider_config|OPENAI_COMPATIBLE_PROVIDERS|A3M_API_KEY|A3M_BASE_URL' \ lib/crewai/src/crewai/llms/providers/a3m/completion.py \ lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai/src/crewai/llms/providers/a3m/completion.py` around lines 26 - 56, Update A3MCompletion to inherit from OpenAICompatibleCompletion instead of OpenAICompletion, and route initialization through the shared provider configuration by setting provider="a3m". Remove the hard-coded A3M base URL and fallback API key so _resolve_provider_config() can apply the registered A3M_API_KEY and A3M_BASE_URL settings while preserving caller-provided kwargs.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/a3m_router_example.py`:
- Line 135: Update example_cost_tracking() and its docstring to reflect that
get_cost() is no longer used, and ensure the example output directs users to
inspect A3M Router analytics at localhost:8787 instead of implying cost details
are printed. Keep the result output intact.
---
Duplicate comments:
In `@lib/crewai/src/crewai/llms/providers/a3m/completion.py`:
- Around line 26-56: Update A3MCompletion to inherit from
OpenAICompatibleCompletion instead of OpenAICompletion, and route initialization
through the shared provider configuration by setting provider="a3m". Remove the
hard-coded A3M base URL and fallback API key so _resolve_provider_config() can
apply the registered A3M_API_KEY and A3M_BASE_URL settings while preserving
caller-provided kwargs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 002d425f-c6c1-419b-9645-087f7963d68d
📒 Files selected for processing (2)
examples/a3m_router_example.pylib/crewai/src/crewai/llms/providers/a3m/completion.py
|
|
||
| crew = Crew(agents=[agent], tasks=[task]) | ||
| result = crew.kickoff() | ||
| print(f"Result: {result}") |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Keep the cost-tracking example aligned with its output.
Line 135 prints only the crew result. example_cost_tracking() no longer shows cost information after get_cost() removal. Rename the example and docstring, or print a clear instruction to inspect A3M Router analytics at localhost:8787.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/a3m_router_example.py` at line 135, Update example_cost_tracking()
and its docstring to reflect that get_cost() is no longer used, and ensure the
example output directs users to inspect A3M Router analytics at localhost:8787
instead of implying cost details are printed. Keep the result output intact.
CodeRabbit review: Tasks in example_multi_agent() were missing expected_output and hierarchical crew lacked manager_llm. Also removed remaining '70-95% cost savings' claim from docstring.
1 participant
Summary
Add A3M Router as a supported LLM provider for CrewAI multi-agent systems.
Motivation
CrewAI agents currently rely on LiteLLM or direct provider integrations. A3M Router provides:
Changes
Added A3M to OpenAI-compatible providers
Created dedicated A3MCompletion provider
Added example usage
Usage
Benefits
Testing
Related