Skip to content

fix(crawlers): Use the HTTP client and session pool of AdaptivePlaywrightCrawler in its sub crawlers - #2451

Draft
Mantisus wants to merge 3 commits into
apify:v2from
Mantisus:adaptive-shared-collaborators
Draft

Mantisus wants to merge 3 commits into
apify:v2from
Mantisus:adaptive-shared-collaborators

Conversation

@Mantisus

Copy link
Copy Markdown
Collaborator

Description

  • Both sub crawlers of AdaptivePlaywrightCrawler use the HTTP client and the session pool of the adaptive crawler, instead of building defaults of their own that nothing entered or closed.
  • Static requests and robots.txt lookups go through the client that the crawler enters and closes on every run.

Issues

Testing

  • Integration test, for static and browser requests, that the sub crawlers send through the HTTP client the crawler closes.

✍️ Drafted by Claude Code

@Mantisus
Mantisus marked this pull request as draft November 15, 2026 17:15
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Nov 15, 2026
@Mantisus Mantisus self-assigned this Nov 15, 2026
@Mantisus
Mantisus force-pushed the adaptive-shared-collaborators branch from ed84b3c to 176b3f8 Compare November 15, 2026 22:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant