Skip to content

feat(mcp): add scrape_artist_socials, get_scraper_results, get_artist_posts tools - #1243

Draft
sweetmantech wants to merge 4 commits into
mainfrom
feat/mcp-social-scrape-posts-tools
Draft

sweetmantech wants to merge 4 commits into
mainfrom
feat/mcp-social-scrape-posts-tools

Conversation

@sweetmantech

@sweetmantech sweetmantech commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Before: an agent connected to the Recoup MCP server could read an artist's social profiles but had no way to start a socials scrape, check a scrape run, or read the artist's posts. Those only existed as REST routes.

After: three MCP tools expose them:

  • scrape_artist_socials (write, spends credits): starts a scrape of every social linked to the artist. Checks artist access, gates credits for all linked profiles ($0.05 each + $0.01 per requested post), charges only the runs that started, and registers them as one batch, exactly like POST /api/artist/socials/scrape.
  • get_scraper_results (read): run status, plus the dataset once SUCCEEDED. Same as GET /api/apify/runs/{runId}: requires a credential but is not owner-scoped (chat#1840).
  • get_artist_posts (read): saved posts across the artist's socials with the REST pagination defaults (page 1, limit 20, max 100). Same as GET /api/artists/{id}/posts.

Tracking and merge order

How

  • Shared domain logic for REST and MCP: lib/artist/runArtistSocialsScrape.ts (access check, socials lookup, credit gate, start via startArtistSocialsScrape), lib/apify/getScraperRunResults.ts, and lib/posts/getArtistPosts.ts (posts envelope). The REST handlers and tools only map results to their own response shapes; REST responses are unchanged. The usage label is ARTIST_SOCIALS_SCRAPE_MODEL_ID in lib/const.ts.
  • Tools live in lib/mcp/tools/artistSocials/ and are registered via registerAllArtistSocialsTools, so they are available to API-key callers and to full OAuth (mcp:tools) grants. Full OAuth also runs the existing prepareFullOAuthTool artist gate on artist_account_id.
  • Classified in fullOAuthToolPolicy (scrape is readOnly: false with a credit notice), scripts/mcp-oauth/tool-policy.json (scrape: mcp:write + mcp:spend) and docs/mcp-oauth-tool-qualification.md. Full catalog count 89 → 92.
  • Not added to the limited mcp:read/mcp:write grant (registerOAuthTools), which keeps its existing five-tool contract.

Tests

  • New: tool tests for all three tools, runArtistSocialsScrape, startArtistSocialsScrape, getScraperRunResults, getArtistPosts.
  • pnpm test: 1485 files / 9462 tests pass.
  • tsc -p tsconfig.build.json and pnpm typecheck:mcp-oauth: clean. eslint and prettier clean on changed files.

Generated by Claude Code


Summary by cubic

Adds MCP tools for starting artist social scrapes, checking scrape runs, and listing saved posts—capabilities previously only available through REST.

  • scrape_artist_socials checks artist access, gates the full credit cost, charges only runs that started, and returns an error listing every failure when no run started (nothing was charged); MCP charges use the REST route label so usage groups together.
  • get_scraper_results returns run status and the dataset after success; like the REST route, it requires a credential but is not owner-scoped.
  • get_artist_posts uses the REST pagination defaults.
  • REST and MCP share the scrape orchestration (runArtistSocialsScrape), run-result logic, and posts envelope; REST responses are unchanged.
  • The tools are included in full OAuth policies but not the limited OAuth grant.

Written for commit 2b67e27. Summary will update on new commits.

View guided diff Turn on auto-fix


Generated by Claude Code

…_posts tools

Expose the existing socials scrape, Apify run results and artist posts
REST capabilities as MCP tools so agents can pull an artist's recent
social posts.

- Extract startArtistSocialsScrape and getScraperRunResults so the REST
  handlers and the MCP tools share the same domain logic.
- scrape_artist_socials: artist access check, credit gate for every
  linked profile, charges only runs that started (write, spends credits).
- get_scraper_results: run status plus dataset once SUCCEEDED (read;
  credential required, not owner-scoped, matching chat#1840).
- get_artist_posts: saved posts with the REST pagination defaults (read).
- Classify all three in the full OAuth policy and tool inventory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
api Ready Ready Preview Nov 13, 2026 1:30pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
Pre-merge checks: Passed 1

Check name Status Explanation
Solid & Clean Code Passed Summary: The current head (a23eae9) addresses the main items in the standing FAIL. The scrape orchestration, posts envelope, and route label are now shared. Two minor residuals remain. Neither matches…
Full details: Solid & Clean Code

Explanation

Summary: The current head (a23eae9) addresses the main items in the standing FAIL. The scrape orchestration, posts envelope, and route label are now shared. Two minor residuals remain. Neither matches a hard failure condition in the check. Evidence, checked against the current tree: 1. Scrape orchestration (prior item 1 and 2). lib/artist/runArtistSocialsScrape.ts holds the access check, socials lookup, empty-socials short-circuit, credit gate, and start. Both lib/mcp/tools/artistSocials/registerScrapeArtistSocialsTool.ts (line 51) and lib/artist/postArtistSocialsScrapeHandler.ts (line 35) call it and only map the result. The duplicated sequence is gone. The MCP tool no longer calls checkCreditsAvailable directly. 2. Posts envelope (prior item 2). lib/posts/getArtistPosts.ts builds the { status, posts, pagination } envelope and the Math.ceil(totalCount / limit) || 1 rule. Both lib/posts/getArtistPostsHandler.ts (line 20) and registerGetArtistPostsTool.ts (line 62) use it. The envelope is no longer duplicated. 3. Route label (prior item 3). ARTIST_SOCIALS_SCRAPE_MODEL_ID is defined in lib/const.ts (line 104) and used in runArtistSocialsScrape.ts (line 66). The hardcoded string is gone from the tool. 4. Shared modules. Each new file exports one primary function matching its file name, including runArtistSocialsScrape.ts, startArtistSocialsScrape.ts, getArtistPosts.ts, and getScraperRunResults.ts. Nesting is shallow. Residual, non-blocking issues: - The scrape_artist_socials callback (registerScrapeArtistSocialsTool.ts lines 39-76) is about 38 lines. Roughly 10 of those lines are auth and error handling, and the rest map three result kinds to MCP responses. This is above the 20-line guideline, but it is a thin adapter with one job. - registerGetArtistPostsTool.ts (lines 15-25) still hardcodes max(100), positive(), and the defaults. It does not reuse paginationQuerySchema (lib/zod/paginationQuerySchema.ts, default limit 20, max 100). The code comment gives a reason: plain numeric fields keep the published JSON schema typed. This is a small DRY gap. Prior failure disposition: the orchestration, envelope, and route-label concerns are addressed. The optional split of startArtistSocialsScrape was not done, which the prior resolution marked optional. The schema reuse was not done in full, so it remains a minor residual above. Prior failure addressed: At HEAD a23eae9, runArtistSocialsScrape.ts holds the access check, socials lookup, credit gate, and start, and both the MCP tool and REST handler call it. getArtistPosts.ts is shared by the REST handler and registerGetArtistPostsTool.ts. ARTIST_SOCIALS_SCRAPE_MODEL_ID in lib/const.ts replaces the hardcoded route label. The prior 50-line inline handler is now about 38 lines of auth, ma...

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

usage_events.model_id is documented as METHOD /route/pattern for API
charges; use the REST route's label so MCP and REST scrapes group together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ith REST

Address the CodeRabbit Solid & Clean Code finding on #1243:

- runArtistSocialsScrape now owns the access check, socials lookup, credit
  gate and start; POST /api/artist/socials/scrape and scrape_artist_socials
  both call it and only map its result (403/402/200 vs tool result). One
  credit gate path (checkCreditsAvailable) for one rule.
- getArtistPosts builds the { status, posts, pagination } envelope for both
  GET /api/artists/{id}/posts and get_artist_posts.
- ARTIST_SOCIALS_SCRAPE_MODEL_ID replaces the hardcoded usage label.

REST wire shapes are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor Author

Preview verification (api-cl5c7asqh-recoup.vercel.app, a23eae9)

Run from a Vercel Sandbox against the PR preview. API key minted on the preview itself via POST /api/agents/signup + /api/agents/verify (email code flow), used as Authorization: Bearer on /mcp and x-api-key on REST. The key was never printed.

Check Documented / expected Actual
initialize 200, server info 200, recoup-mcp
tools/list includes scrape_artist_socials inputs artist_account_id, posts listed, inputs match
tools/list includes get_scraper_results input runId listed, input matches
tools/list includes get_artist_posts inputs artist_account_id, page, limit listed, inputs match
get_artist_posts on new artist same shape as GET /api/artists/{id}/posts {status:"success", posts:[], pagination:{...}}, identical to REST
scrape_artist_socials on artist with no socials no runs, no charge success:true, runs:[], "no linked social profiles" (REST returns [])
get_scraper_results with unknown runId error success:false, "Failed to read the scrape run."
get_artist_posts with nonexistent artist UUID error success:false, "Failed to list artist posts."
tools/list with no auth 401 401

Notes

  • The test account is in the Recoup org, so checkAccountArtistAccess passes for any artist id. The forbidden path was not exercised here; it is covered by unit tests.
  • Not covered: a scrape that actually starts Apify runs and get_scraper_results on a real runId (the test artist has no linked socials, so no credits were spent).
  • MCP scrape_artist_socials returns {success, runs} while REST returns a bare array. Intentional for the tool wrapper, flagging for review.
  • Preview env: agent+ instant signup returns 500 because the preview Privy app hit max_accounts_reached. Unrelated to this PR.

Generated by Claude Code

Every profile's start can fail (bad Apify token, unsupported platform). The
tool used to report success:true with all-null runs, so an agent could not
tell nothing happened. Return an error listing each failure instead; credits
are charged only for started runs, so the message says none were charged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 2b67e270 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants