Repository navigation
feat(mcp): add scrape_artist_socials, get_scraper_results, get_artist_posts tools - #1243
sweetmantech wants to merge 4 commits into
Conversation
…_posts tools Expose the existing socials scrape, Apify run results and artist posts REST capabilities as MCP tools so agents can pull an artist's recent social posts. - Extract startArtistSocialsScrape and getScraperRunResults so the REST handlers and the MCP tools share the same domain logic. - scrape_artist_socials: artist access check, credit gate for every linked profile, charges only runs that started (write, spends credits). - get_scraper_results: run status plus dataset once SUCCEEDED (read; credential required, not owner-scoped, matching chat#1840). - get_artist_posts: saved posts with the REST pagination defaults (read). - Classify all three in the full OAuth policy and tool inventory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: truePre-merge checks:
|
| Check name | Status | Explanation |
|---|---|---|
| Solid & Clean Code | Summary: The current head (a23eae9) addresses the main items in the standing FAIL. The scrape orchestration, posts envelope, and route label are now shared. Two minor residuals remain. Neither matches… |
Full details: Solid & Clean Code
-
Explanation
Summary: The current head (a23eae9) addresses the main items in the standing FAIL. The scrape orchestration, posts envelope, and route label are now shared. Two minor residuals remain. Neither matches a hard failure condition in the check. Evidence, checked against the current tree: 1. Scrape orchestration (prior item 1 and 2).
lib/artist/runArtistSocialsScrape.tsholds the access check, socials lookup, empty-socials short-circuit, credit gate, and start. Bothlib/mcp/tools/artistSocials/registerScrapeArtistSocialsTool.ts(line 51) andlib/artist/postArtistSocialsScrapeHandler.ts(line 35) call it and only map the result. The duplicated sequence is gone. The MCP tool no longer callscheckCreditsAvailabledirectly. 2. Posts envelope (prior item 2).lib/posts/getArtistPosts.tsbuilds the{ status, posts, pagination }envelope and theMath.ceil(totalCount / limit) || 1rule. Bothlib/posts/getArtistPostsHandler.ts(line 20) andregisterGetArtistPostsTool.ts(line 62) use it. The envelope is no longer duplicated. 3. Route label (prior item 3).ARTIST_SOCIALS_SCRAPE_MODEL_IDis defined inlib/const.ts(line 104) and used inrunArtistSocialsScrape.ts(line 66). The hardcoded string is gone from the tool. 4. Shared modules. Each new file exports one primary function matching its file name, includingrunArtistSocialsScrape.ts,startArtistSocialsScrape.ts,getArtistPosts.ts, andgetScraperRunResults.ts. Nesting is shallow. Residual, non-blocking issues: - Thescrape_artist_socialscallback (registerScrapeArtistSocialsTool.tslines 39-76) is about 38 lines. Roughly 10 of those lines are auth and error handling, and the rest map three result kinds to MCP responses. This is above the 20-line guideline, but it is a thin adapter with one job. -registerGetArtistPostsTool.ts(lines 15-25) still hardcodesmax(100),positive(), and the defaults. It does not reusepaginationQuerySchema(lib/zod/paginationQuerySchema.ts, default limit 20, max 100). The code comment gives a reason: plain numeric fields keep the published JSON schema typed. This is a small DRY gap. Prior failure disposition: the orchestration, envelope, and route-label concerns are addressed. The optional split ofstartArtistSocialsScrapewas not done, which the prior resolution marked optional. The schema reuse was not done in full, so it remains a minor residual above. Prior failure addressed: At HEAD a23eae9,runArtistSocialsScrape.tsholds the access check, socials lookup, credit gate, and start, and both the MCP tool and REST handler call it.getArtistPosts.tsis shared by the REST handler andregisterGetArtistPostsTool.ts.ARTIST_SOCIALS_SCRAPE_MODEL_IDinlib/const.tsreplaces the hardcoded route label. The prior 50-line inline handler is now about 38 lines of auth, ma...
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
usage_events.model_id is documented as METHOD /route/pattern for API charges; use the REST route's label so MCP and REST scrapes group together. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ith REST Address the CodeRabbit Solid & Clean Code finding on #1243: - runArtistSocialsScrape now owns the access check, socials lookup, credit gate and start; POST /api/artist/socials/scrape and scrape_artist_socials both call it and only map its result (403/402/200 vs tool result). One credit gate path (checkCreditsAvailable) for one rule. - getArtistPosts builds the { status, posts, pagination } envelope for both GET /api/artists/{id}/posts and get_artist_posts. - ARTIST_SOCIALS_SCRAPE_MODEL_ID replaces the hardcoded usage label. REST wire shapes are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preview verification (api-cl5c7asqh-recoup.vercel.app, a23eae9)Run from a Vercel Sandbox against the PR preview. API key minted on the preview itself via
Notes
Generated by Claude Code |
Every profile's start can fail (bad Apify token, unsupported platform). The tool used to report success:true with all-null runs, so an agent could not tell nothing happened. Return an error listing each failure instead; credits are charged only for started runs, so the message says none were charged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Before: an agent connected to the Recoup MCP server could read an artist's social profiles but had no way to start a socials scrape, check a scrape run, or read the artist's posts. Those only existed as REST routes.
After: three MCP tools expose them:
scrape_artist_socials(write, spends credits): starts a scrape of every social linked to the artist. Checks artist access, gates credits for all linked profiles ($0.05 each + $0.01 per requested post), charges only the runs that started, and registers them as one batch, exactly likePOST /api/artist/socials/scrape.get_scraper_results(read): run status, plus the dataset once SUCCEEDED. Same asGET /api/apify/runs/{runId}: requires a credential but is not owner-scoped (chat#1840).get_artist_posts(read): saved posts across the artist's socials with the REST pagination defaults (page 1, limit 20, max 100). Same asGET /api/artists/{id}/posts.Tracking and merge order
How
lib/artist/runArtistSocialsScrape.ts(access check, socials lookup, credit gate, start viastartArtistSocialsScrape),lib/apify/getScraperRunResults.ts, andlib/posts/getArtistPosts.ts(posts envelope). The REST handlers and tools only map results to their own response shapes; REST responses are unchanged. The usage label isARTIST_SOCIALS_SCRAPE_MODEL_IDinlib/const.ts.lib/mcp/tools/artistSocials/and are registered viaregisterAllArtistSocialsTools, so they are available to API-key callers and to full OAuth (mcp:tools) grants. Full OAuth also runs the existingprepareFullOAuthToolartist gate onartist_account_id.fullOAuthToolPolicy(scrape isreadOnly: falsewith a credit notice),scripts/mcp-oauth/tool-policy.json(scrape:mcp:write+mcp:spend) anddocs/mcp-oauth-tool-qualification.md. Full catalog count 89 → 92.mcp:read/mcp:writegrant (registerOAuthTools), which keeps its existing five-tool contract.Tests
runArtistSocialsScrape,startArtistSocialsScrape,getScraperRunResults,getArtistPosts.pnpm test: 1485 files / 9462 tests pass.tsc -p tsconfig.build.jsonandpnpm typecheck:mcp-oauth: clean. eslint and prettier clean on changed files.Generated by Claude Code
Summary by cubic
Adds MCP tools for starting artist social scrapes, checking scrape runs, and listing saved posts—capabilities previously only available through REST.
scrape_artist_socialschecks artist access, gates the full credit cost, charges only runs that started, and returns an error listing every failure when no run started (nothing was charged); MCP charges use the REST route label so usage groups together.get_scraper_resultsreturns run status and the dataset after success; like the REST route, it requires a credential but is not owner-scoped.get_artist_postsuses the REST pagination defaults.runArtistSocialsScrape), run-result logic, and posts envelope; REST responses are unchanged.Written for commit 2b67e27. Summary will update on new commits.
Generated by Claude Code