Skip to content

README.md

Latest commit

 

History

History
60 lines (38 loc) · 2.77 KB

README.md

File metadata and controls

60 lines (38 loc) · 2.77 KB
Raw

xfetch Python search example

Collect a bounded sample of recent public X/Twitter posts, follow pagination, deduplicate post IDs, and save a JSON export. Maintained for xfetch; the initial code and documentation were drafted with AI and tested offline using synthetic responses.

Requirements

  • Python 3.9 or newer; no third-party packages.
  • An xfetch API key in XFETCH_API_KEY.

Run

Clone this repository, or download search_posts.py directly:

git clone https://github.com/prxusyd/xfetch-python-search-example.git
cd xfetch-python-search-example

In Bash or zsh, enter the key at the hidden input after running read, then export it:

read -r -s XFETCH_API_KEY
export XFETCH_API_KEY

python3 search_posts.py '"AI agents" lang:en -filter:replies' \
  --max-pages 3 \
  --output ai-agent-posts.json

The output filename must not already exist. The script permits one to three requests, each asking for up to 20 posts. It uses GET /v1/search/recent, keeps the same query/limit/sort order across pages, and deduplicates by string post ID within one run.

Read the output

  • data: unique post records, retaining the first version seen in the run.
  • pages_fetched: successfully parsed pages.
  • credits_charged_confirmed: charges reported on those pages, before any local deduplication.
  • stop_reason: no_next_token, page_limit, repeated_token, or error.

Empty or short pages can still carry a continuation token. A missing token ends this pagination sequence, without establishing exhaustive coverage of X or historical data. Inspect created_at for your analysis window.

Errors and repeated tokens stop the run with exit code 1; earlier successful pages remain in the JSON export. The script does not automatically retry. After a timeout, total charges may exceed confirmed usage because a response may have been lost after the server completed the request.

Offline checks

Put test_search_posts.py beside the script and run:

PYTHONDONTWRITEBYTECODE=1 python3 -m unittest -v test_search_posts.py

All HTTP calls are mocked. The checks cover URL encoding, opaque token forwarding, duplicates, empty pages, request limits, repeated tokens, errors, partial exports, and preflight validation. They do not establish live data coverage or production API availability.

API documentation

Read the full tutorial: Search public X/Twitter posts with Python: pagination, deduplication, and JSON export.

This is a code example published by the product's author, not an independent review.