Feat/add new adapters (douban, sinablog, medium, substack)#185
Merged
jackwener merged 1 commit intojackwener:mainfrom Mar 21, 2026
Merged
Feat/add new adapters (douban, sinablog, medium, substack)#185jackwener merged 1 commit intojackwener:mainfrom
jackwener merged 1 commit intojackwener:mainfrom
Conversation
Owner
|
你可以 pull rebase 到最新的代码吗，并且尽量少改动除了 clis 以外的内容 |
New adapters: - douban: book-hot, movie-hot, search (browser/cookie) - sinablog: hot, search, article, user (search uses public API) - substack: feed, publication, search (search uses public API) Medium upgrade (YAML → TS): - Replace tag.yaml/user.yaml/publication.yaml with TS adapters - feed.ts (tag feed by topic), search.ts, user.ts with browser scraping - Richer data: readTime, claps, description Core pipeline improvements: - template.ts: trim template before matching (supports multiline expressions) - template.ts: evalJsExpr fallback for JS expressions in YAML templates - template.ts: add urlencode/urldecode filters - transform.ts: inline select inside map params - build-manifest.ts: TS-over-YAML dedup with warning log - build-manifest.ts: export scanTs/shouldReplaceManifestEntry for testing Co-authored-by: Yuan <leotsao.cy@gmail.com>
jackwener
force-pushed
the
feat/add-new-adapters
branch
from
d47fbda to
815aeb0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds and migrates the new adapters for:
All of these adapters are now implemented in TypeScript instead of YAML, and have been locally reviewed and verified.
Included commands
douban
book-hotmovie-hotsearchsinablog
hotsearcharticleusermedium
feedsearchusersubstack
feedpublicationsearchInternal fixes
During the migration, I also fixed a few related core issues that affected adapter reliability:
selectruns beforemapin mixed pipeline stepsselectinsidemapVerification
Build
npm run buildTests
npx vitest run src/build-manifest.test.ts src/pipeline/template.test.ts src/pipeline/transform.test.ts src/pipeline/executor.test.tsManual command verification
medium feed --topic aimedium search --keyword aimedium user --username canartucsubstack feed --category techsubstack publication --url https://ruben.substack.comsubstack search --keyword 'claude code' --type postssubstack search --keyword claude --type publicationssinablog hotsinablog search --keyword 冯矿伟sinablog article --url https://blog.sina.com.cn/s/blog_59b3f4ee0102zjjw.htmlsinablog user --uid 1504965870douban book-hotdouban movie-hotdouban search --type book --keyword 三体douban search --type movie --keyword 霸王别姬douban search --type music --keyword 周杰伦