Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
![Python](https://img.shields.io/badge/python-3.10%20%E2%80%93%203.13-green)
![License](https://img.shields.io/badge/license-MIT-blue)

[CrewAI](https://www.crewai.com) tools for the [Decodo](https://decodo.com) Web Scraping API. Give your agents live web pages and search results.
[Decodo Web Data API](https://help.decodo.com/docs/web-data-api-introduction) tools for [CrewAI](https://www.crewai.com). Give your agents live web pages and search results.

| Tool | What it does |
| --- | --- |
Expand All @@ -15,7 +15,7 @@
## Requirements

- Python 3.10 to 3.13
- A Decodo Web Scraping API key
- A Decodo Web Data API key. Basic authentication tokens from older plans don't work with these tools.

## Installation

Expand All @@ -25,7 +25,7 @@ pip install decodo-crewai

## Credentials

Copy your API key from your Web Scraping API subscription on the [Decodo dashboard](https://dashboard.decodo.com/web-data/playground) and set it as an environment variable:
Copy your API key from your Web Data API subscription on the [Decodo dashboard](https://dashboard.decodo.com/web-data/playground) and set it as an environment variable:

```bash
export DECODO_API_KEY="<api_key>"
Expand Down Expand Up @@ -88,7 +88,7 @@ Constructor options:

| Option | Default | Description |
| --- | --- | --- |
| `api_key` | `DECODO_API_KEY` | Decodo Web Scraping API key. |
| `api_key` | `DECODO_API_KEY` | Decodo Web Data API key. |
| `max_chars` | `45000` | Markdown longer than this is truncated to keep the page within the agent's context. |

### DecodoSearchTool
Expand All @@ -112,6 +112,8 @@ Arguments the agent fills in. Pass either `product_id` or `query`:
| `query` | `str`, optional | Search the store's products instead. |
| `zip_code` | `str`, optional | US ZIP code for prices and availability at that delivery location, e.g. `10001`. |

Target pages are rendered in a headless browser and can take close to a minute.

### DecodoSERPTool

Arguments the agent fills in:
Expand Down
2 changes: 1 addition & 1 deletion examples/research_crew/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ DECODO_API_KEY=<decodo_api_key>
OPENAI_API_KEY=<openai_api_key>
```

Get the Decodo API key from your Web Scraping API subscription on the [Decodo dashboard](https://dashboard.decodo.com/web-data/playground).
Get the Decodo API key from your Web Data API subscription on the [Decodo dashboard](https://dashboard.decodo.com/web-data/playground).

The crew uses CrewAI's default model, `gpt-4.1-mini`. To use another model, set `MODEL`, e.g. `MODEL=gpt-4.1`. Other providers need their CrewAI extra and key, e.g. `pip install "crewai[anthropic]"`, `ANTHROPIC_API_KEY` and `MODEL=anthropic/<model>`.

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ build-backend = "hatchling.build"
[project]
name = "decodo-crewai"
version = "0.0.0"
description = "Decodo Web Scraping API tools for CrewAI agents."
description = "Decodo Web Data API tools for CrewAI agents."
readme = "README.md"
license = { file = "LICENSE" }
requires-python = ">=3.10,<3.14"
Expand Down
4 changes: 2 additions & 2 deletions src/decodo_crewai/_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
class DecodoBaseTool(BaseTool):
"""Shared Decodo client and error handling for the Decodo tools.

Get an API key from your Web Scraping API subscription on the Decodo dashboard:
Get an API key from your Web Data API subscription on the Decodo dashboard:
https://dashboard.decodo.com/web-data/playground
"""

Expand All @@ -35,7 +35,7 @@ class DecodoBaseTool(BaseTool):
default_factory=lambda: [
EnvVar(
name=API_KEY_ENV_VAR,
description="API key for the Decodo Web Scraping API",
description="API key for the Decodo Web Data API",
required=True,
)
]
Expand Down
2 changes: 1 addition & 1 deletion src/decodo_crewai/ecommerce_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ class DecodoEcommerceTool(DecodoBaseTool):
"""Get product details or search results from Amazon, Walmart or Target, parsed as JSON.

Args:
api_key: Decodo Web Scraping API key. Defaults to the ``DECODO_API_KEY`` environment variable.
api_key: Decodo Web Data API key. Defaults to the ``DECODO_API_KEY`` environment variable.
"""

name: str = "Decodo Ecommerce"
Expand Down
4 changes: 2 additions & 2 deletions src/decodo_crewai/search_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ class DecodoSearchArgs(BaseModel):


class DecodoSearchTool(DecodoBaseTool):
"""Search Google or Amazon with the Decodo Web Scraping API and return the parsed results as JSON.
"""Search Google or Amazon with the Decodo Web Data API and return the parsed results as JSON.

Args:
api_key: Decodo Web Scraping API key. Defaults to the ``DECODO_API_KEY`` environment variable.
api_key: Decodo Web Data API key. Defaults to the ``DECODO_API_KEY`` environment variable.
"""

name: str = "Decodo Search"
Expand Down
2 changes: 1 addition & 1 deletion src/decodo_crewai/serp_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ class DecodoSERPTool(DecodoBaseTool):
"""Search Google or Bing, or ask Google AI Mode, and return the parsed results as JSON.

Args:
api_key: Decodo Web Scraping API key. Defaults to the ``DECODO_API_KEY`` environment variable.
api_key: Decodo Web Data API key. Defaults to the ``DECODO_API_KEY`` environment variable.
"""

name: str = "Decodo SERP"
Expand Down
4 changes: 2 additions & 2 deletions src/decodo_crewai/web_scraper_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ class DecodoWebScraperArgs(BaseModel):


class DecodoWebScraperTool(DecodoBaseTool):
"""Scrape any public web page with the Decodo Web Scraping API and return it as Markdown.
"""Scrape any public web page with the Decodo Web Data API and return it as Markdown.

Args:
api_key: Decodo Web Scraping API key. Defaults to the ``DECODO_API_KEY`` environment variable.
api_key: Decodo Web Data API key. Defaults to the ``DECODO_API_KEY`` environment variable.
max_chars: Markdown longer than this is truncated, to keep the page within the agent's context.
"""

Expand Down
Loading