From 875632f1979cb640400eea4619782ff0eedafe34 Mon Sep 17 00:00:00 2001 From: Gil Desmarais Date: Fri, 14 Aug 2026 00:37:16 +0200 Subject: [PATCH] docs(ruby-gem): sync docs for gem release v0.25.0 - Document --limit flag under html2rss auto CLI reference - Document channel-level author and image discovery in auto-source reference - Document article limit option (limit: 20) in auto-source reference - Add 0.25.0 migration notes for legacy :link selector removal in backward-compatibility guide --- .../guides/backward-compatibility.mdx | 31 ++++++++++++++----- .../docs/ruby-gem/reference/auto-source.mdx | 4 ++- .../docs/ruby-gem/reference/cli-reference.mdx | 2 ++ 3 files changed, 29 insertions(+), 8 deletions(-) diff --git a/src/content/docs/ruby-gem/guides/backward-compatibility.mdx b/src/content/docs/ruby-gem/guides/backward-compatibility.mdx index 885da47f..9357cf48 100644 --- a/src/content/docs/ruby-gem/guides/backward-compatibility.mdx +++ b/src/content/docs/ruby-gem/guides/backward-compatibility.mdx @@ -7,6 +7,15 @@ import { Code } from "@astrojs/starlight/components"; This page outlines recent breaking changes, purged legacy aliases, and migration steps for older feed configurations. +## 0.25.0 + +Upgrade notes for configs and integrators targeting gem **0.25.0**: + +1. **Removed `link` selector alias** — Custom selector configurations must use the canonical `url` key. The legacy `link` selector alias and exclusivity checks have been removed. +2. **`auto` article limit (`--limit N`)** — Discovery now supports a configurable article limit (`--limit N` CLI flag, default `20`) and `Html2rss.auto_source(url, limit: 20)` in Ruby. +3. **Channel metadata in `auto`** — `auto_source` automatically extracts channel-level `author` and `image` (favicon, touch-icon, or OpenGraph images) from the host page. +4. **Actionable `NoFeedItemsExtracted` error** — `auto` mode fails loud with diagnostic surface classifications (blocked surface, app-shell, unsupported extraction surface) when zero items are found. + ## 0.24.0 Upgrade notes for configs and integrators targeting gem **0.24.0**: @@ -21,21 +30,28 @@ In previous versions, `html2rss` accepted legacy selector names with a deprecati | Removed Name | Required Name | Description | | :----------- | :------------- | :-------------------- | +| `link` | `url` | Item destination URL | | `pubDate` | `published_at` | Item publication date | | `updated` | `published_at` | Item publication date | ### Migration Example -Update any occurrences of `pubDate` or `updated` in your `selectors` block to `published_at`: +Update any occurrences of `link`, `pubDate`, or `updated` in your `selectors` block to `url` or `published_at`: `; select podcast/media URLs with `enclosure`. -5. **Prefer `feed_result` for dual-format / cache**: Integrators that need RSS + JSON Feed from one scrape (or Marshal caching) should use `Html2rss.feed_result` and `status.to_h`. -6. **Validate configurations**: Run `html2rss validate config.yml` to ensure your YAML conforms to the current schema. +1. **Rename URL selectors**: Ensure article URL selectors use `url` rather than `link`. +2. **Rename date selectors**: Ensure date selectors use `published_at` rather than `updated` or `pubDate`. +3. **Move channel-level transport keys**: Ensure `strategy` and `headers` are defined at the top level of the YAML file. +4. **Pin Browserless when needed**: If you relied on `auto` falling through to Browserless, set `strategy: browserless` (or `--strategy browserless`). +5. **Use `enclosure` for RSS media**: Do not rely on `image` becoming an RSS ``; select podcast/media URLs with `enclosure`. +6. **Prefer `feed_result` for dual-format / cache**: Integrators that need RSS + JSON Feed from one scrape (or Marshal caching) should use `Html2rss.feed_result` and `status.to_h`. +7. **Validate configurations**: Run `html2rss validate config.yml` to ensure your YAML conforms to the current schema. diff --git a/src/content/docs/ruby-gem/reference/auto-source.mdx b/src/content/docs/ruby-gem/reference/auto-source.mdx index 459ab67d..605804a3 100644 --- a/src/content/docs/ruby-gem/reference/auto-source.mdx +++ b/src/content/docs/ruby-gem/reference/auto-source.mdx @@ -34,7 +34,9 @@ To enable it, add `auto_source: {}` to your configuration: such as `window.__NEXT_DATA__`, `window.__NUXT__`, or `window.STATE`. The JSON-state scraper walks those blobs, finds arrays with `title`/`url` pairs, and converts them into feed items. -`auto_source` also automatically extracts and prepends kicker/teaser labels to article titles when present. +`auto_source` also automatically extracts and prepends kicker/teaser labels to article titles when present, and discovers channel-level `author` and `image` (favicon, touch-icon, or OpenGraph images) from the host page. + +You can also restrict the maximum number of discovered articles via the `limit:` keyword argument in Ruby (`Html2rss.auto_source(url, limit: 10)`) or the `--limit` CLI flag. **`json_state` Limitations:** the scraper requires discoverable arrays of hashes containing clear `title` and `url` fields. Minified or obfuscated state objects, heavily encoded values, or blobs that require executing embedded functions are ignored. diff --git a/src/content/docs/ruby-gem/reference/cli-reference.mdx b/src/content/docs/ruby-gem/reference/cli-reference.mdx index 7d4a9d17..0089ce33 100644 --- a/src/content/docs/ruby-gem/reference/cli-reference.mdx +++ b/src/content/docs/ruby-gem/reference/cli-reference.mdx @@ -23,6 +23,7 @@ Automatically discovers items from a page and prints the generated RSS or JSONFe code={` html2rss auto https://example.com/articles ; \ html2rss auto https://example.com/articles --format jsonfeed ; \ + html2rss auto https://example.com/articles --limit 10 ; \ html2rss auto https://example.com/app --strategy browserless --max-redirects 5 --max-requests 6 ; \ BOTASAURUS_SCRAPER_URL="http://localhost:4010" html2rss auto https://example.com/protected --strategy botasaurus ; \ html2rss auto https://example.com/articles --items_selector ".post-card" ; \ @@ -38,6 +39,7 @@ Available options: - `--strategy`: Optional request strategy (`auto`, `faraday`, `browserless`, `botasaurus`, `local_file`). Defaults to `auto`, which tries `faraday` -> `botasaurus`. Pin `browserless` explicitly when you need headless Chrome (preload/interaction). - `--format`: Output format for the auto-sourced feed (`rss` or `jsonfeed`). Defaults to `rss`. +- `--limit`: Maximum number of articles to extract during discovery (defaults to `20`). - `--items_selector`: Optional CSS selector hint for item extraction. - `--max-redirects`: Maximum redirects to follow per request. - `--max-requests`: Maximum requests to allow for this feed build (defaults to `4` for auto discovery).