Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ async def main() -> None:
'some': 'input',
}

# Start an Actor and wait for it to finish.
# Start an Actor and waits for it to finish.
call_result = await actor_client.call(run_input=run_input)
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ def main() -> None:
'some': 'input',
}

# Start an Actor and wait for it to finish.
# Start an Actor and waits for it to finish.
call_result = actor_client.call(run_input=run_input)
7 changes: 4 additions & 3 deletions website/versioned_docs/version-2.5/01_introduction/index.mdx
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
---
id: introduction
title: Overview
sidebar_label: Overview
slug: /
description: The official Python library to access the Apify API, with automatic retries, async support, and comprehensive API coverage.
description: "The official Python library to access the Apify API, with automatic retries, async support, and comprehensive API coverage."
---

import Tabs from '@theme/Tabs';
Expand All @@ -23,7 +24,7 @@ The client simplifies interaction with the Apify platform by providing:

## Prerequisites

`apify-client` requires Python 3.11 or higher. Python is available for download on the [official website](https://www.python.org/downloads/). Check your current Python version by running:
`apify-client` requires Python 3.10 or higher. Python is available for download on the [official website](https://www.python.org/downloads/). Check your current Python version by running:

```bash
python --version
Expand All @@ -39,7 +40,7 @@ pip install apify-client

## Quick example

The following example shows how to run an Actor and retrieve its results:
Here's an example showing how to run an Actor and retrieve its results:

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
id: quick-start
title: Quick start
description: Get started with the Apify API client for Python by running an Actor and retrieving results from its dataset.
sidebar_label: Quick start
description: "Get started with the Apify API client for Python by running an Actor and retrieving results from its dataset."
---

import Tabs from '@theme/Tabs';
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: asyncio-support
title: Asyncio support
description: Use the async client for non-blocking API calls with Python asyncio.
---

import Tabs from '@theme/Tabs';
Expand All @@ -11,12 +10,10 @@ import ApiLink from '@theme/ApiLink';

import AsyncSupportExample from '!!raw-loader!./code/01_async_support.py';

The package provides an asynchronous version of the client, <ApiLink to="class/ApifyClientAsync">`ApifyClientAsync`</ApiLink>, which allows you to interact with the Apify API using Python's standard async/await syntax. This enables you to perform non-blocking operations, see the Python [asyncio documentation](https://docs.python.org/3/library/asyncio-task.html) for more information. This is useful for applications that need to perform multiple API operations concurrently or integrate with other async frameworks.
The package provides an asynchronous version of the client, <ApiLink to="class/ApifyClientAsync">`ApifyClientAsync`</ApiLink>, which allows you to interact with the Apify API using Python's standard async/await syntax. This enables you to perform non-blocking operations, see the Python [asyncio documentation](https://docs.python.org/3/library/asyncio-task.html) for more information.

The following example shows how to run an Actor asynchronously and stream its logs while it is running:
The following example demonstrates how to run an Actor asynchronously and stream its logs while it is running:

<CodeBlock className="language-python">
{AsyncSupportExample}
</CodeBlock>

For the full async client API, see the <ApiLink to="class/ApifyClientAsync">`ApifyClientAsync`</ApiLink> reference.
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: single-and-collection-clients
title: Single and collection clients
description: Understand the two types of resource clients, single-resource and collection clients.
---

import Tabs from '@theme/Tabs';
Expand All @@ -15,7 +14,7 @@ import CollectionSyncExample from '!!raw-loader!./code/02_collection_sync.py';
import SingleAsyncExample from '!!raw-loader!./code/02_single_async.py';
import SingleSyncExample from '!!raw-loader!./code/02_single_sync.py';

The Apify client provides two types of resource clients: single-resource clients for managing an individual resource, and collection clients for listing or creating resources.
The Apify client interface is designed to be consistent and intuitive across all of its components. When you call specific methods on the main client, you create specialized clients to manage individual API resources. There are two main types of clients:

- <ApiLink to="class/ActorClient">`ActorClient`</ApiLink> - Manages a single resource.
- <ApiLink to="class/ActorCollectionClient">`ActorCollectionClient`</ApiLink> - Manages a collection of resources.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: nested-clients
title: Nested clients
description: Access related resources directly through nested client methods.
---

import Tabs from '@theme/Tabs';
Expand All @@ -13,7 +12,7 @@ import ApiLink from '@theme/ApiLink';
import NestedAsyncExample from '!!raw-loader!./code/03_nested_async.py';
import NestedSyncExample from '!!raw-loader!./code/03_nested_sync.py';

Nested clients let you access related resources directly from a parent resource client, without manually constructing new client instances.
In some cases, the Apify client provides nested clients to simplify working with related collections. For example, you can easily manage the runs of a specific Actor without having to construct multiple endpoints or client instances manually.

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: error-handling
title: Error handling
description: Handle API errors with the ApifyApiError exception and automatic data parsing.
---

import Tabs from '@theme/Tabs';
Expand All @@ -14,22 +13,6 @@ import ErrorSyncExample from '!!raw-loader!./code/04_error_sync.py';

When you use the Apify client, it automatically extracts all relevant data from the endpoint and returns it in the expected format. Date strings, for instance, are seamlessly converted to Python `datetime.datetime` objects. If an error occurs, the client raises an <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink>. This exception wraps the raw JSON errors returned by the API and provides additional context, making it easier to debug any issues that arise.

## Error subclasses

The Apify client provides dedicated error subclasses based on the HTTP status code of the failed response, so you can branch on error kind without inspecting `status_code` or `type`:

| Status | Subclass |
|---|---|
| 400 | <ApiLink to="class/InvalidRequestError">`InvalidRequestError`</ApiLink> |
| 401 | <ApiLink to="class/UnauthorizedError">`UnauthorizedError`</ApiLink> |
| 403 | <ApiLink to="class/ForbiddenError">`ForbiddenError`</ApiLink> |
| 404 | <ApiLink to="class/NotFoundError">`NotFoundError`</ApiLink> |
| 409 | <ApiLink to="class/ConflictError">`ConflictError`</ApiLink> |
| 429 | <ApiLink to="class/RateLimitError">`RateLimitError`</ApiLink> |
| 5xx | <ApiLink to="class/ServerError">`ServerError`</ApiLink> |

All subclasses inherit from <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink>, so an existing `except ApifyApiError` handler still catches every API error. Catch a specific subclass when you want to react differently to, for example, a missing resource or a rate-limit:

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
<CodeBlock className="language-python">
Expand All @@ -42,5 +25,3 @@ All subclasses inherit from <ApiLink to="class/ApifyApiError">`ApifyApiError`</A
</CodeBlock>
</TabItem>
</Tabs>

For a complete list of error classes, see the <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink> reference.
9 changes: 4 additions & 5 deletions website/versioned_docs/version-2.5/02_concepts/05_retries.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: retries
title: Retries
description: Configure automatic retries with exponential backoff for failed requests.
---

import Tabs from '@theme/Tabs';
Expand All @@ -13,23 +12,23 @@ import ApiLink from '@theme/ApiLink';
import RetriesAsyncExample from '!!raw-loader!./code/05_retries_async.py';
import RetriesSyncExample from '!!raw-loader!./code/05_retries_sync.py';

The Apify client automatically retries requests that fail due to:
When dealing with network communication, failures can occasionally occur. The Apify client automatically retries requests that fail due to:

- Network errors
- Internal errors in the Apify API (HTTP status codes 500 and above)
- Rate limit errors (HTTP status code 429)

By default, the client retries a failed request up to 4 times. The retry intervals use an exponential backoff strategy:
By default, the client will retry a failed request up to 8 times. The retry intervals use an exponential backoff strategy:

- The first retry occurs after approximately 500 milliseconds.
- The second retry occurs after approximately 1,000 milliseconds, and so on.

You can customize this behavior using the following options in the <ApiLink to="class/ApifyClient">`ApifyClient`</ApiLink> constructor:

- `max_retries`: Defines the maximum number of retry attempts.
- `min_delay_between_retries`: Sets the minimum delay between retries as a `timedelta`.
- `min_delay_between_retries_millis`: Sets the minimum delay between retries (in milliseconds).

Retries with exponential backoff help reduce the load on the server and increase the chances of a successful request.
Retries with exponential backoff are a common strategy for handling network errors. They help to reduce the load on the server and increase the chances of a successful request.

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: logging
title: Logging
description: Configure debug logging to inspect API requests and responses.
---

import Tabs from '@theme/Tabs';
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: convenience-methods
title: Convenience methods
description: Use high-level methods for running Actors, waiting for results, and accessing data.
---

import Tabs from '@theme/Tabs';
Expand All @@ -15,9 +14,8 @@ import CallSyncExample from '!!raw-loader!./code/07_call_sync.py';

The Apify client provides several convenience methods to handle actions that the API alone cannot perform efficiently, such as waiting for an Actor run to finish without running into network timeouts. These methods simplify common tasks and enhance the usability of the client.

- <ApiLink to="class/ActorClient#call">`ActorClient.call`</ApiLink> - Starts an Actor and waits for it to finish, handling network timeouts internally. Waits indefinitely by default, or up to the specified `wait_duration`.
- <ApiLink to="class/ActorClient#start">`ActorClient.start`</ApiLink> - Starts an Actor and immediately returns the Run object without waiting for it to finish.
- <ApiLink to="class/RunClient#wait_for_finish">`RunClient.wait_for_finish`</ApiLink> - Waits for an already-started run to reach a terminal status.
- <ApiLink to="class/ActorClient#call">`ActorClient.call`</ApiLink> - Starts an Actor and waits for it to finish, handling network timeouts internally.
- <ApiLink to="class/ActorClient#start">`ActorClient.start`</ApiLink> - Explicitly waits for an Actor run to finish with customizable timeouts.

Additionally, storage-related resources offer flexible options for data retrieval:

Expand All @@ -36,7 +34,3 @@ Additionally, storage-related resources offer flexible options for data retrieva
</CodeBlock>
</TabItem>
</Tabs>

:::tip
The `call()` method polls internally and may take a long time for long-running Actors. Use `start()` and `wait_for_finish()` separately if you need more control over the waiting behavior.
:::
51 changes: 4 additions & 47 deletions website/versioned_docs/version-2.5/02_concepts/08_pagination.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: pagination
title: Pagination
description: Paginate through large result sets using page models or generator-based iteration.
---

import Tabs from '@theme/Tabs';
Expand All @@ -12,22 +11,18 @@ import ApiLink from '@theme/ApiLink';

import PaginationAsyncExample from '!!raw-loader!./code/08_pagination_async.py';
import PaginationSyncExample from '!!raw-loader!./code/08_pagination_sync.py';
import IterateItemsAsyncExample from '!!raw-loader!./code/08_iterate_items_async.py';
import IterateItemsSyncExample from '!!raw-loader!./code/08_iterate_items_sync.py';
import IterateCollectionAsyncExample from '!!raw-loader!./code/08_iterate_collection_async.py';
import IterateCollectionSyncExample from '!!raw-loader!./code/08_iterate_collection_sync.py';

Most methods named `list` or `list_something` in the Apify client return a page model — a [Pydantic](https://docs.pydantic.dev/latest/) model such as <ApiLink to="class/ListOfActors">`ListOfActors`</ApiLink>, <ApiLink to="class/ListOfDatasets">`ListOfDatasets`</ApiLink>, or <ApiLink to="class/ListOfRequests">`ListOfRequests`</ApiLink>. Unstructured dataset items use the <ApiLink to="class/DatasetItemsPage">`DatasetItemsPage`</ApiLink> dataclass instead. All page models share a consistent interface for working with paginated data and expose the following fields:
Most methods named `list` or `list_something` in the Apify client return a <ApiLink to="class/ListPage">`ListPage`</ApiLink> object. This object provides a consistent interface for working with paginated data and includes the following properties:

- `items` - The main results you're looking for.
- `total` - The total number of items available.
- `offset` - The starting point of the current page.
- `count` - The number of items in the current page.
- `limit` - The maximum number of items per page.

Some methods, such as `list_keys` or `list_head`, paginate differently. Regardless, the primary results are always stored under the `items` field, and the `limit` field can be used to control the number of results returned.
Some methods, such as `list_keys` or `list_head`, paginate differently. Regardless, the primary results are always stored under the items property, and the limit property can be used to control the number of results returned.

The following example shows how to fetch all items from a dataset using pagination:
The following example demonstrates how to fetch all items from a dataset using pagination:

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
Expand All @@ -42,42 +37,4 @@ The following example shows how to fetch all items from a dataset using paginati
</TabItem>
</Tabs>

The page model interface offers several key benefits. Its consistent structure ensures predictable results for most `list` methods, providing a uniform way to work with paginated data. It also offers flexibility, allowing you to customize the `limit` and `offset` parameters to control data fetching according to your needs. Additionally, it provides scalability, enabling you to efficiently handle large datasets through pagination. This approach ensures efficient data retrieval while keeping memory usage under control, making it ideal for managing and processing large collections.

## Generator-based iteration

For collection clients, the `iterate` method returns an iterator that lazily fetches as many pages as needed
to retrieve every item matching the filters. For dataset, key-value store and request queue clients, the
matching helpers are `iterate_items`, `iterate_keys` and `iterate_requests`. They handle pagination
automatically, so you don't need to manage offsets, limits or cursors yourself.

The example below iterates over every Actor owned by the current user using a collection client's `iterate`
method:

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
<CodeBlock className="language-python">
{IterateCollectionAsyncExample}
</CodeBlock>
</TabItem>
<TabItem value="SyncExample" label="Sync client">
<CodeBlock className="language-python">
{IterateCollectionSyncExample}
</CodeBlock>
</TabItem>
</Tabs>

The next example uses `iterate_items` on a dataset client to stream items past a given offset:

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
<CodeBlock className="language-python">
{IterateItemsAsyncExample}
</CodeBlock>
</TabItem>
<TabItem value="SyncExample" label="Sync client">
<CodeBlock className="language-python">
{IterateItemsSyncExample}
</CodeBlock>
</TabItem>
</Tabs>
The <ApiLink to="class/ListPage">`ListPage`</ApiLink> interface offers several key benefits. Its consistent structure ensures predictable results for most `list` methods, providing a uniform way to work with paginated data. It also offers flexibility, allowing you to customize the `limit` and `offset` parameters to control data fetching according to your needs. Additionally, it provides scalability, enabling you to efficiently handle large datasets through pagination. This approach ensures efficient data retrieval while keeping memory usage under control, making it ideal for managing and processing large collections.
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
id: streaming-resources
title: Streaming resources
description: Stream large datasets, key-value store records, and logs without loading them into memory.
---

import Tabs from '@theme/Tabs';
Expand All @@ -23,7 +22,7 @@ Supported streaming methods:

These methods return a raw, context-managed `impit.Response` object. The response must be consumed within a with block to ensure that the connection is closed automatically, preventing memory leaks or unclosed connections.

The following example shows how to stream the logs of an Actor run incrementally:
The following example demonstrates how to stream the logs of an Actor run incrementally:

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
Expand All @@ -38,4 +37,4 @@ The following example shows how to stream the logs of an Actor run incrementally
</TabItem>
</Tabs>

Streaming is ideal for processing large logs, datasets, or files incrementally without downloading them entirely into memory.
Streaming offers several key benefits. It ensures memory efficiency by loading only a small portion of the resource into memory at any given time, making it ideal for handling large data. It enables real-time processing, allowing you to start working with data immediately as it is received. With automatic resource management, using the `with` statement ensures that connections are properly closed, preventing memory leaks or unclosed connections. This approach is valuable for processing large logs, datasets, or files on the fly without the need to download them entirely.
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ async def main() -> None:
actor_client = apify_client.actor('username/actor-name')
runs_client = actor_client.runs()

# List the last 10 runs of the Actor.
# List the last 10 runs of the Actor
actor_runs = (await runs_client.list(limit=10, desc=True)).items

# Select the last run of the Actor that finished with a SUCCEEDED status.
last_succeeded_run_client = actor_client.last_run(status='SUCCEEDED')
# Select the last run of the Actor that finished with a SUCCEEDED status
last_succeeded_run_client = actor_client.last_run(status='SUCCEEDED') # ty: ignore[invalid-argument-type]

# Get dataset
actor_run_dataset_client = last_succeeded_run_client.dataset()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ def main() -> None:
actor_client = apify_client.actor('username/actor-name')
runs_client = actor_client.runs()

# List the last 10 runs of the Actor.
# List the last 10 runs of the Actor
actor_runs = runs_client.list(limit=10, desc=True).items

# Select the last run of the Actor that finished with a SUCCEEDED status.
last_succeeded_run_client = actor_client.last_run(status='SUCCEEDED')
# Select the last run of the Actor that finished with a SUCCEEDED status
last_succeeded_run_client = actor_client.last_run(status='SUCCEEDED') # ty: ignore[invalid-argument-type]

# Get dataset
actor_run_dataset_client = last_succeeded_run_client.dataset()
Expand Down
Loading