Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,12 +127,12 @@ jobs:
for delay in 15 30 60 120 240; do
# Not hash-pinned on purpose: this installs the version just published,
# whose hashes exist only on the index, to prove a user's install works.
if python -m pip install --no-cache-dir "${index[@]}" "sdo-sdk==${SDO_EXPECTED_VERSION}"; then
if python -m pip install --no-cache-dir "${index[@]}" "sdo-client==${SDO_EXPECTED_VERSION}"; then
python -c "import os, sdo; assert sdo.__version__ == os.environ['SDO_EXPECTED_VERSION'], sdo.__version__; print('installed', sdo.__version__)"
exit 0
fi
echo "sdo-sdk==${SDO_EXPECTED_VERSION} is not installable yet; retrying in ${delay}s"
echo "sdo-client==${SDO_EXPECTED_VERSION} is not installable yet; retrying in ${delay}s"
sleep "$delay"
done
echo "::error::sdo-sdk==${SDO_EXPECTED_VERSION} never became installable"
echo "::error::sdo-client==${SDO_EXPECTED_VERSION} never became installable"
exit 1
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Before 1.0, a minor version may contain breaking changes; each is marked

## [Unreleased]

The first release of `sdo-sdk`, the official Python SDK for the Scrape.do API
The first release of `sdo-client`, the official Python SDK for the Scrape.do API
(import name `sdo`). Requires Python 3.11 or newer.

### Added
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

Thanks for taking the time to help out — contributions are genuinely welcome, whether that's a bug report, a failing test that pins down an edge case, a docs fix, or a new feature. This guide gets you from a fresh clone to a passing test run, and explains the few conventions we hold the line on.

`sdo` is Scrape.do's official open-source tooling. This repository is the Python SDK: package `sdo-sdk` on PyPI, imported as `import sdo`. It's a thin, well-tested wrapper over the Scrape.do API built on [`httpx`](https://www.python-httpx.org/).
`sdo` is Scrape.do's official open-source tooling. This repository is the Python SDK: package `sdo-client` on PyPI, imported as `import sdo`. It's a thin, well-tested wrapper over the Scrape.do API built on [`httpx`](https://www.python-httpx.org/).

The project is developed Solo+AI: the founder is the sole approver and final reviewer. That means review may be asynchronous, and design discussion happens up front (see [Changing the public surface](#changing-the-public-surface)). Don't let that discourage you — clear, small, well-tested PRs get merged.

Expand Down
10 changes: 5 additions & 5 deletions MIGRATING.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<!-- SPDX-License-Identifier: Apache-2.0 -->
# Migrating `sdo-sdk`
# Migrating `sdo-client`

This document is the authoritative record of breaking changes in the Python SDK
(`sdo-sdk` on PyPI). It exists so that upgrading is never a guessing game: every
(`sdo-client` on PyPI). It exists so that upgrading is never a guessing game: every
removal that lands in a new major version is documented here, with the
deprecation that preceded it and the replacement you should adopt.

Expand All @@ -12,7 +12,7 @@ guidance is never rewritten.

## Compatibility policy

`sdo-sdk` follows [Semantic Versioning](https://semver.org/). Before 1.0, a
`sdo-client` follows [Semantic Versioning](https://semver.org/). Before 1.0, a
minor version may contain breaking changes; each is marked **Breaking** in
[CHANGELOG.md](CHANGELOG.md) and described in this file. Patch versions never
break.
Expand Down Expand Up @@ -101,7 +101,7 @@ here.

### How to upgrade safely

1. Pin a major in your dependency spec, e.g. `sdo-sdk>=1.4,<2`.
1. Pin a major in your dependency spec, e.g. `sdo-client>=1.4,<2`.
2. Enable `error::DeprecationWarning` in CI and fix warnings as they appear, on
your own schedule, while still on the current major.
3. When the next major is published, read its section here, apply the documented
Expand All @@ -115,7 +115,7 @@ No version has been published yet, so there is nothing to migrate from. The
first releases are `0.x`: the additive-and-deprecate intent above is already in
force, but a minor version may still contain a breaking change before `1.0.0`;
each one is marked **Breaking** in the changelog and described here. Patch
versions never break. Pin a minor (`sdo-sdk==0.1.*`) if you need maximum
versions never break. Pin a minor (`sdo-client==0.1.*`) if you need maximum
stability during this period.

---
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Token-only auth, transparent compression, a typed result, and a generic plugin
core — no telemetry, no UI. Every parameter is described in the
[API documentation](https://scrape.do/documentation).

> Distribution name: **`sdo-sdk`** · import name: **`sdo`**. The `sdo`
> Distribution name: **`sdo-client`** · import name: **`sdo`**. The `sdo`
> command-line tool is not on PyPI: install it with Homebrew
> (`brew install scrape-do/tap/sdo`) or Scoop, see
> [sdo-cli](https://github.com/scrape-do/sdo-cli). The PyPI project named `sdo`
Expand All @@ -17,9 +17,9 @@ core — no telemetry, no UI. Every parameter is described in the
## Install

```bash
pip install sdo-sdk
pip install sdo-client
# optional CSS/XPath extraction via .selector():
pip install "sdo-sdk[parse]"
pip install "sdo-client[parse]"
```

Requires Python 3.11+. Brotli decompression comes from `brotli` (1.2.1 or newer), a
Expand All @@ -37,7 +37,7 @@ with Client(token="YOUR_TOKEN") as client:
print(result.status_code) # the TARGET's status code
print(result.cost) # credits this request cost
print(result.text[:200]) # decoded HTML
title = result.selector().css("title::text").get() # needs sdo-sdk[parse]
title = result.selector().css("title::text").get() # needs sdo-client[parse]

# Reuse a config, or build one inline:
cfg = ScrapeConfig(url="https://example.com", super_proxy=True, output="markdown")
Expand All @@ -46,7 +46,7 @@ with Client(token="YOUR_TOKEN") as client:

Runnable programs are in
[`examples/`](https://github.com/scrape-do/sdo-sdk-python/tree/main/examples):
with `sdo-sdk` installed, set `SDO_TOKEN` and run `python examples/quickstart.py`.
with `sdo-client` installed, set `SDO_TOKEN` and run `python examples/quickstart.py`.

`ScrapeConfig` fields mirror the documented Scrape.do parameters. Values that
mean "unset" (`None`, `False`, `0`, `""`, empty list or dict) are not sent;
Expand Down
4 changes: 2 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<!-- SPDX-License-Identifier: Apache-2.0 -->
# Security Policy

`sdo` (`sdo-sdk`, `import sdo`) is the official Python SDK for Scrape.do. It is a
`sdo` (`sdo-client`, `import sdo`) is the official Python SDK for Scrape.do. It is a
token-bearing HTTP client: it holds your Scrape.do API token and attaches it to
every outbound request. The token is the sensitive asset this SDK handles, so we
treat any issue that can leak, log, mishandle, or weaken protection of that token
Expand Down Expand Up @@ -99,7 +99,7 @@ minors. Upgrade to the latest minor to receive security updates.
| Any older minor | :x: |

If you are pinned to an older minor, the remediation for a reported vulnerability
is to upgrade to the patched latest minor (`pip install -U sdo-sdk`).
is to upgrade to the patched latest minor (`pip install -U sdo-client`).

## Triage SLA

Expand Down
2 changes: 1 addition & 1 deletion examples/render_geo_and_extract.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
"""JS rendering + geo targeting + CSS extraction (needs sdo-sdk[parse])."""
"""JS rendering + geo targeting + CSS extraction (needs sdo-client[parse])."""

from sdo import Client, ScrapeConfig

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ requires = ["hatchling==1.32.4"]
build-backend = "hatchling.build"

[project]
name = "sdo-sdk"
name = "sdo-client"
dynamic = ["version"]
description = "Official Python SDK for the Scrape.do web-scraping API"
readme = "README.md"
Expand Down
4 changes: 2 additions & 2 deletions src/sdo/response.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,13 +157,13 @@ def selector(self) -> Any:
"""Return a ``parsel.Selector`` for CSS/XPath extraction (typed ``Any``
so the optional dependency is not needed to resolve the annotation).

``parsel`` is an optional dependency; install ``sdo-sdk[parse]``.
``parsel`` is an optional dependency; install ``sdo-client[parse]``.
"""
try:
from parsel import Selector
except ImportError as exc: # pragma: no cover - optional dep
raise ImportError(
"selector() needs the optional 'parsel' dependency: pip install 'sdo-sdk[parse]'"
"selector() needs the optional 'parsel' dependency: pip install 'sdo-client[parse]'"
) from exc
return Selector(text=self.text)

Expand Down
Loading