Skip to content

feat: opt-in field filtering for scrape_batch and search_engine_batch#143

Open
SuarezPM wants to merge 1 commit into
brightdata:mainfrom
SuarezPM:feat/batch-field-filtering
Open

feat: opt-in field filtering for scrape_batch and search_engine_batch#143
SuarezPM wants to merge 1 commit into
brightdata:mainfrom
SuarezPM:feat/batch-field-filtering

Conversation

@SuarezPM

@SuarezPM SuarezPM commented Jun 3, 2026

Copy link
Copy Markdown

Splitting the field-filtering part out of #140 into its own PR.

Adds an optional fields param to both batch tools so you only get back the
fields you actually need — saves a lot of tokens in agent pipelines.
filter_fields skips prototype-polluting keys and has unit tests.

  • Backward compatible: if fields isn't passed, the output is exactly the
    same as before.
  • scrape_batch also isolates errors per URL now (a failed URL comes back as
    {url, error} instead of breaking the settlement) and keeps
    responseType: 'text'.

Leaving the dedup out of this one — it needs a block-level rewrite to actually
collapse shared boilerplate properly, so that'll come as its own PR.

Optional `fields` param returns only the requested top-level fields, to save
tokens in agent pipelines. Backward compatible: when omitted, output is
unchanged. scrape_batch also isolates per-URL errors (a failed URL resolves
to {url, error} instead of a rejected settlement).
@egoriklok

Copy link
Copy Markdown

Public no-secret MCP Buyer-Agent Readiness Snapshot for brightdata-mcp:

  • Public signal: Public MCP candidate brightdata-mcp: A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.. Matched R1 terms: agent, automation, browser, github, mcp, server, tool.
  • R1 fit: agent, automation, browser, github, mcp, server, tool.
  • Readiness status: public evidence review needed before an autonomous buyer-agent should rely on this surface.
  • Blind spot 1: explicit auth scopes and delegated-permission boundary.
  • Blind spot 2: spend/API cost cap plus approval semantics before paid actions.
  • Blind spot 3: receipt, audit-log, revocation, or dispute evidence for safe buyer-agent use.
  • Single next question: For brightdata-mcp, is there already a documented policy for agent spend/auth limits, receipt evidence, and revocation before a buyer-agent can invoke it?

No secrets, invoice, payment link, delivery link, private endpoint, paid call, or wallet signature; this is only a free public snapshot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants