From 90639e8c4561c9ea87bc838e34c7f2d9fbbf2783 Mon Sep 17 00:00:00 2001 From: Marco Vinciguerra Date: Thu, 30 Jul 2026 17:06:14 +0200 Subject: [PATCH] docs(pdf): explain page limit configuration --- api-reference/endpoint/extract.mdx | 3 +- api-reference/endpoint/scrape.mdx | 3 +- api-reference/endpoint/search.mdx | 3 +- sdks/javascript.mdx | 4 +-- sdks/python.mdx | 4 +-- services/crawl.mdx | 2 +- services/extract.mdx | 2 +- services/scrape.mdx | 51 ++++++++++++++++++++++++++++++ services/search.mdx | 2 +- 9 files changed, 64 insertions(+), 10 deletions(-) diff --git a/api-reference/endpoint/extract.mdx b/api-reference/endpoint/extract.mdx index b48d5ad..143fd4c 100644 --- a/api-reference/endpoint/extract.mdx +++ b/api-reference/endpoint/extract.mdx @@ -45,7 +45,8 @@ Exactly one of `url`, `html`, or `markdown` must be supplied as the source. Optional processing configuration for URL input. Omit this field to use the 25-page PDF cap. You may also send `{"type":"pdf"}` and omit `maxPages`; it defaults to `25`. Set `maxPages` only to override the default, using `1`–`500`, or `-1` for no page limit. PDF processing costs 1 credit - per page actually processed. + per page actually processed. See [Configure PDF page + limits](/services/scrape#configure-pdf-page-limits) for examples. diff --git a/api-reference/endpoint/scrape.mdx b/api-reference/endpoint/scrape.mdx index 0bbfb4c..ef7e2f4 100644 --- a/api-reference/endpoint/scrape.mdx +++ b/api-reference/endpoint/scrape.mdx @@ -42,7 +42,8 @@ Returns markdown, HTML, links, images, summary, JSON extraction, branding, or sc Optional processing configuration. Omit this field to use the 25-page PDF cap. You may also send `{"type":"pdf"}` and omit `maxPages`; it defaults to `25`. Set `maxPages` only to override the default, using `1`–`500`, or `-1` for no page limit. PDF processing costs 1 credit per page - actually processed. + actually processed. See [Configure PDF page limits](/services/scrape#configure-pdf-page-limits) + for examples. diff --git a/api-reference/endpoint/search.mdx b/api-reference/endpoint/search.mdx index 00a512f..02cc846 100644 --- a/api-reference/endpoint/search.mdx +++ b/api-reference/endpoint/search.mdx @@ -47,7 +47,8 @@ Replaces the v1 `searchscraper` endpoint. Returns the top pages with their conte Optional processing configuration. Omit this field to use the 25-page PDF cap. You may also send `{"type":"pdf"}` and omit `maxPages`; it defaults to `25`. Set `maxPages` only to override the default, using `1`–`500`, or `-1` for no page limit. PDF processing costs 1 credit per page - actually processed. This does not change `allowedTypes`. + actually processed. See [Configure PDF page limits](/services/scrape#configure-pdf-page-limits) + for examples. This does not change `allowedTypes`. diff --git a/sdks/javascript.mdx b/sdks/javascript.mdx index 8680b67..8ba88d1 100644 --- a/sdks/javascript.mdx +++ b/sdks/javascript.mdx @@ -241,7 +241,7 @@ if (res.status === "success") { | `timeRange` | `string` | No | `"past_hour"`, `"past_24_hours"`, `"past_week"`, `"past_month"`, `"past_year"` | | `locationGeoCode` | `string` | No | Two-letter country code (e.g. `"us"`) | | `allowedTypes` | `string[]` | No | Non-empty MIME allowlist. Omit it for all supported types; `"all"` and `"*"` are not accepted | -| `processors` | `object[]` | No | Omit for the 25-page PDF cap. `{ type: "pdf" }` also defaults to 25; set `maxPages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed | +| `processors` | `object[]` | No | Omit for the 25-page PDF cap. `{ type: "pdf" }` also defaults to 25; set `maxPages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed. [Configuration examples](/services/scrape#configure-pdf-page-limits) | | `fetchConfig` | `FetchConfig` | No | Fetch configuration | @@ -304,7 +304,7 @@ await sgai.crawl.delete(crawlId); | `includePatterns` | `string[]` | No | URL patterns to include | | `excludePatterns` | `string[]` | No | URL patterns to exclude | | `allowedTypes` | `string[]` | No | Non-empty MIME allowlist; omit it to allow every supported type | -| `processors` | `object[]` | No | Omit for the 25-page PDF cap. `{ type: "pdf" }` also defaults to 25; set `maxPages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed | +| `processors` | `object[]` | No | Omit for the 25-page PDF cap. `{ type: "pdf" }` also defaults to 25; set `maxPages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed. [Configuration examples](/services/scrape#configure-pdf-page-limits) | | `fetchConfig` | `FetchConfig` | No | Fetch configuration | ### `sgai.monitor.*` diff --git a/sdks/python.mdx b/sdks/python.mdx index ab83da3..5005d90 100644 --- a/sdks/python.mdx +++ b/sdks/python.mdx @@ -271,7 +271,7 @@ if res.status == "success": | `location_geo_code` | `str` | No | Two-letter country code (e.g. `"us"`, `"it"`) | | `time_range` | `str` | No | `"past_hour"`, `"past_24_hours"`, `"past_week"`, `"past_month"`, `"past_year"` | | `allowed_types` | `list[str]` | No | Non-empty MIME allowlist. Omit it for all supported types; `"all"` and `"*"` are not accepted | -| `processors` | `list[PdfProcessor]` | No | Omit for the 25-page PDF cap. `PdfProcessor()` also defaults to 25; set `max_pages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed | +| `processors` | `list[PdfProcessor]` | No | Omit for the 25-page PDF cap. `PdfProcessor()` also defaults to 25; set `max_pages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed. [Configuration examples](/services/scrape#configure-pdf-page-limits) | | `fetch_config` | `FetchConfig` | No | Fetch configuration | ### Crawl @@ -319,7 +319,7 @@ sgai.crawl.delete(crawl_id) | `include_patterns` | `list[str]` | No | URL glob patterns to include | | `exclude_patterns` | `list[str]` | No | URL glob patterns to exclude | | `allowed_types` | `list[str]` | No | Non-empty MIME allowlist; omit it to allow every supported type | -| `processors` | `list[PdfProcessor]` | No | Omit for the 25-page PDF cap. `PdfProcessor()` also defaults to 25; set `max_pages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed | +| `processors` | `list[PdfProcessor]` | No | Omit for the 25-page PDF cap. `PdfProcessor()` also defaults to 25; set `max_pages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed. [Configuration examples](/services/scrape#configure-pdf-page-limits) | | `fetch_config` | `FetchConfig` | No | Fetch configuration | ### Monitor diff --git a/services/crawl.mdx b/services/crawl.mdx index 73a5085..144128a 100644 --- a/services/crawl.mdx +++ b/services/crawl.mdx @@ -134,7 +134,7 @@ curl -X GET "https://v2-api.scrapegraphai.com/api/crawl/:id/pages?limit=50&curso | `includePatterns` / `include_patterns` | array | No | URL patterns to include (e.g. `["/blog/*"]`). | | `excludePatterns` / `exclude_patterns` | array | No | URL patterns to exclude (e.g. `["/admin/*"]`). | | `allowedTypes` / `allowed_types` | array | No | Non-empty MIME allowlist. Omit it for all supported types; `"all"` and `"*"` are invalid. | -| `processors` | array | No | Omit for the 25-page PDF cap. Send it only to override the cap; `maxPages` also defaults to 25 when omitted, and `-1` means unlimited. PDF processing costs 1 credit per page actually processed. | +| `processors` | array | No | Omit for the 25-page PDF cap. Send it only to override the cap; `maxPages` also defaults to 25 when omitted, and `-1` means unlimited. PDF processing costs 1 credit per page actually processed. [Configure PDF page limits](/services/scrape#configure-pdf-page-limits). | | `fetchConfig` / `fetch_config` | object | No | Fetch options (see [Scrape · FetchConfig](/services/scrape#fetchconfig)). | diff --git a/services/extract.mdx b/services/extract.mdx index c144c0c..daa47dd 100644 --- a/services/extract.mdx +++ b/services/extract.mdx @@ -80,7 +80,7 @@ curl -X POST https://v2-api.scrapegraphai.com/api/extract \ | `schema` | object | No | JSON schema describing the desired output shape. In Python you can pass a Pydantic model via `MyModel.model_json_schema()`. | | `mode` | string | No | HTML processing mode: `"normal"`, `"reader"`, `"prune"`. | | `allowedTypes` / `allowed_types` | array | No | Non-empty MIME allowlist for URL input. Omit it for all supported types; `"all"` and `"*"` are invalid. | -| `processors` | array | No | Omit for the 25-page PDF cap. Send it only to override the cap; `maxPages` also defaults to 25 when omitted, and `-1` means unlimited. PDF processing costs 1 credit per page actually processed. | +| `processors` | array | No | Omit for the 25-page PDF cap. Send it only to override the cap; `maxPages` also defaults to 25 when omitted, and `-1` means unlimited. PDF processing costs 1 credit per page actually processed. [Configure PDF page limits](/services/scrape#configure-pdf-page-limits). | | `fetchConfig` / `fetch_config` | object | No | Fetch options (see [Scrape · FetchConfig](/services/scrape#fetchconfig)). | diff --git a/services/scrape.mdx b/services/scrape.mdx index 22d3c3d..2e2ec69 100644 --- a/services/scrape.mdx +++ b/services/scrape.mdx @@ -96,6 +96,57 @@ curl -X POST https://v2-api.scrapegraphai.com/api/scrape \ Get your API key from the [dashboard](https://scrapegraphai.com/dashboard). +## Configure PDF page limits + +PDF page limits are configured inside the `processors` array, not as a top-level request field. +The API uses a 25-page limit when `processors` is omitted or when the PDF processor does not include +`maxPages`. + +| Configuration | PDF pages processed | +|---|---| +| Omit `processors` | Up to 25 pages | +| `{"type": "pdf"}` | Up to 25 pages | +| `{"type": "pdf", "maxPages": 10}` | Up to 10 pages | +| `{"type": "pdf", "maxPages": -1}` | Every page | + +Use an integer from `1` to `500` for a custom limit. The limit is an upper bound, not a fixed +charge: PDF processing costs 1 credit for each page actually processed. For example, a 6-page PDF +with `maxPages: 25` processes 6 pages and costs 6 credits. + + + +```python Python +from scrapegraph_py import PdfProcessor + +res = sgai.scrape( + "https://example.com/document.pdf", + processors=[PdfProcessor(max_pages=10)], +) +``` + +```javascript JavaScript +const res = await sgai.scrape({ + url: "https://example.com/document.pdf", + processors: [{ type: "pdf", maxPages: 10 }], +}); +``` + +```bash cURL +curl -X POST https://v2-api.scrapegraphai.com/api/scrape \ + -H "SGAI-APIKEY: $SGAI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "url": "https://example.com/document.pdf", + "formats": [{"type": "markdown"}], + "processors": [{"type": "pdf", "maxPages": 10}] + }' +``` + + + +Python uses `max_pages`; JavaScript and the REST API use `maxPages`. The same `processors` +configuration is available on Scrape, Extract, Search, and Crawl. + ```json { diff --git a/services/search.mdx b/services/search.mdx index eb59eae..d327a9a 100644 --- a/services/search.mdx +++ b/services/search.mdx @@ -87,7 +87,7 @@ curl -X POST https://v2-api.scrapegraphai.com/api/search \ | `timeRange` / `time_range` | string | No | Recency filter: `"past_hour"`, `"past_24_hours"`, `"past_week"`, `"past_month"`, `"past_year"`. | | `locationGeoCode` / `location_geo_code` | string | No | Two-letter ISO country code for localized results. Curated set (52): `ae`, `ar`, `at`, `au`, `be`, `br`, `ca`, `ch`, `cl`, `cn`, `co`, `cz`, `de`, `dk`, `eg`, `es`, `fi`, `fr`, `gb`, `gr`, `hk`, `hu`, `id`, `ie`, `il`, `in`, `it`, `jp`, `kr`, `mx`, `my`, `ng`, `nl`, `no`, `nz`, `pe`, `ph`, `pk`, `pl`, `pt`, `ro`, `ru`, `sa`, `se`, `sg`, `th`, `tr`, `tw`, `ua`, `us`, `vn`, `za`. | | `allowedTypes` / `allowed_types` | array | No | Optional non-empty MIME allowlist. Omit it to allow every supported type, including PDF. | -| `processors` | array | No | Omit for the 25-page PDF cap. Send it only to override the cap; `maxPages` also defaults to 25 when omitted, and `-1` means unlimited. PDF processing costs 1 credit per page actually processed. | +| `processors` | array | No | Omit for the 25-page PDF cap. Send it only to override the cap; `maxPages` also defaults to 25 when omitted, and `-1` means unlimited. PDF processing costs 1 credit per page actually processed. [Configure PDF page limits](/services/scrape#configure-pdf-page-limits). | | `fetchConfig` / `fetch_config` | object | No | Fetch options (see [Scrape · FetchConfig](/services/scrape#fetchconfig)). |