Repository navigation
Conversation
AI assistants and their crawlers request documentation paths that never existed or no longer do, hundreds of times a month each (about 9,000 requests in 28 days at the edge, no human sessions). Send the ones with one clear subject to the page that covers it: anomaly detection, Anomaly Advisor, performance optimization, reverse proxy, Parents and streaming, the daemon, collectors, system statistics and alert configuration. Guesses without a clear subject stay on the 404 page. The static section of netlify.toml mirrors static.toml.
✅ Deploy Preview for netdata-docusaurus ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
No actionable comments were generated in the recent review.
 |
| Layer / File(s) | Summary |
|---|---|
Add documentation redirect rules static.toml, netlify.toml |
Both files add redirects from guessed or obsolete documentation paths to current pages. The paths cover anomaly detection and its troubleshooting page, agent performance and reverse-proxy configuration, Netdata Parents, daemons, collectors, system statistics, and health-alert configuration. |
Estimated code review effort: 1 (Trivial) | ~5 minutes
Change: Bug fix
Suggested reviewers: ilyam8
Merge Risk: 
dc3c8
-
The redirect changes are ready to merge with no identified functional or operational risk.
Pre-merge checks: 
-
Check name Status Explanation Description Check Check skipped - CodeRabbit’s high-level summary is enabled. Title check The title "Redirect seventeen guessed Learn URLs that crawlers keep requesting" matches the main change. The PR adds 17 exact redirects for guessed paths that crawlers request, and the summary describ… Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Full details: Title check
-
Explanation
The title "Redirect seventeen guessed Learn URLs that crawlers keep requesting" matches the main change. The PR adds 17 exact redirects for guessed paths that crawlers request, and the summary describes redirect rules in netlify.toml and static.toml. The title is specific and short. It does not name the docs platform, but it states the primary change clearly.
-
Finishing Touches
-
Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Architecture diagram
sequenceDiagram
participant Crawler as AI Crawler / User
participant Edge as Netlify Edge
participant Config as netlify.toml / static.toml
participant Generator as Redirect Generator
participant LR as Live Redirect Rules
participant Sitemap as Live Sitemap
participant Target as Target Page (200)
participant Verify as verify-redirect-graph.js
Note over Crawler,Target: Static Site Redirect Flow
Crawler->>Edge: GET /docs/netdata-ai/machine-learning-and-anomaly-detection
Edge->>Config: Load redirect rules (netlify.toml)
Edge->>Generator: Match source against [[redirects]] rules
alt Exact match found (NEW: guessed address rule)
Generator-->>Edge: Found rule (source -> target)
Edge->>LR: Apply exact redirect (301)
LR-->>Crawler: 301 Redirect to /docs/netdata-ai/anomaly-detection
Crawler->>Target: Follow redirect
Target-->>Crawler: 200 OK (HTML page)
else No match (still 404)
Generator-->>Edge: No rule found
Edge-->>Crawler: 404 Not Found
end
Note over Config,Verify: Redirect Validation Flow
Config->>Verify: Load netlify.toml + static.toml
Verify->>Sitemap: Fetch live sitemap
Sitemap-->>Verify: Route tree (targets)
Verify->>Verify: Check all 3,931 redirects resolve
alt Target in sitemap
Verify-->>Target: Confirms 200 status
else Target missing
Verify-->>Verify: Flag dead redirect
end
Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack
View guided diff | Turn on auto-fix | Re-trigger cubic
/docs/collectors and /docs/netdata-agent/collectors name the collectors themselves, which /docs/collecting-metrics/collectors lists; the Collecting Metrics overview is one level too broad.
Summary
AI assistants and their crawlers keep requesting Learn paths that never existed or no longer do. Cloudflare's edge data for the last 28 days shows about 9,000 such requests and no human sessions in GA4. Most come from Meta's crawler; a few come from Googlebot and Claude.
This PR adds 17 exact rules. Each sends a guess with one clear subject to the page that covers it today.
/docs/netdata-ai/machine-learning-and-anomaly-detection/docs/netdata-ai/anomaly-detection/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance/docs/netdata-agent/configuration/performance-optimization/docs/netdata-ai/machine-learning-anomaly-detection/docs/netdata-ai/anomaly-detection/docs/netdata-agent/configuration/running-the-agent-behind-a-reverse-proxy.../securing-agents/running-the-agent-behind-a-reverse-proxy/docs/netdata-agent/machine-learning-and-anomaly-detection/docs/netdata-ai/anomaly-detection/docs/netdata-agent/ml-anomaly-detection/docs/netdata-ai/anomaly-detection/docs/netdata-ai/anomaly-advisor/docs/netdata-ai/troubleshooting/anomaly-advisor/docs/netdata-agent/observability-centralization-points/docs/netdata-parents/docs/netdata-anomaly-detection/docs/netdata-ai/anomaly-detection/docs/collectors/docs/collecting-metrics/collectors/docs/netdata-agent/machine-learning-anomaly-detection/docs/netdata-ai/anomaly-detection/docs/netdata-agent/ml/docs/netdata-ai/anomaly-detection/docs/alerts-&-notifications/configure-health-alerts/docs/alerts-&-notifications/alert-configuration-reference/docs/collecting-metrics/system-metrics/system-statistics.../collectors/operating-systems/system-statistics/docs/netdata-agent/daemons/docs/netdata-agent/daemon/docs/netdata-agent/collectors/docs/collecting-metrics/collectors/docs/netdata-agent/streaming/docs/netdata-parentsSome guesses have no single clear subject, for example
/docs/netdata-agent/real-time-monitoring,/docs/netdata-agent/configuration/running-the-agentand/docs/netdata-ai/alerts-and-notifications. They stay on the 404 page.Validation
netlify.tomlis byte-identical tostatic.toml.scripts/verify-redirect-graph.jsagainst a route tree built from the live sitemap andstatic/. The branch verifies 3,931 redirects, which is master's 3,914 plus these 17.Summary by cubic
Adds 17 exact redirects for Learn doc paths that never existed or no longer do, but that crawlers keep requesting — about 9,000 requests in 28 days with no human sessions. Each now goes to the page that covers its subject; guesses without one clear subject still return 404.
Validation
netlify.tomlandstatic.tomlreceive identical rules.Written for commit dc3c884. Summary will update on new commits.
Summary by CodeRabbit