Docs: make crawler-path verification legible, and explain pass-throughs - #5
Merged
Merged
Conversation
The verification snippet told people to curl `https://your-site.com/optimized-page/`, which reads like a route the plugin creates. It is not — there is no such path, so substituting only the domain produces a plain WordPress 404 and looks like a broken install. The plugin only answers URLs CiteCue holds an optimized version of, and connecting does not generate any. Replace the placeholder with `/a-page-you-optimized/`, pipe the checks through `grep -i x-citecue`, and add a table of the four outcomes so a missing header is legible as a pass-through rather than a failure. Document the pass-through causes in likelihood order, including two that are easy to mistake for a broken install: the 60 s negative cache on misses (an immediate retry makes no API call at all), and the fact that the proxy ignores logged-in users, so a wp-admin browser tab always shows the normal site. Note that `Recent AI crawler activity` records nothing when the plugin declines before calling the API — an open circuit, a negative-cached miss, serving switched off or an unrecognized UA all leave no row, so an empty table is not evidence of a cache or CDN in front. Since llms.txt keeps answering from its 5-minute local cache while the circuit is open, `x-citecue: llms-txt` does not prove the connection is live either; document flush-then-retest as the way to tell those apart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A fresh install ran the README's verification snippet verbatim:
/optimized-page/was a placeholder, but it reads like a route the plugin creates, so substituting only the domain gives a plain WordPress 404 and looks like a broken install. Nothing was wrong with the plugin.The follow-up was worse: the same site's homepage returned no
x-citecueheader either, and the docs offered no way to read that. A missing header is the normal pass-through — the plugin only answers URLs CiteCue holds an optimized version of, and connecting does not generate any.What changed
Docs only. No code touched.
README.md —
Verifying/optimized-page/→/a-page-you-optimized/, with an explicit note that no such route existsgrep -i x-citecuellms-txt,served,served+stale, and no header at allWhen nothing is servedsubsection: pass-through causes in likelihood order, and how to readRecent AI crawler activityreadme.txt — the hand-check command plus a new FAQ entry, "My llms.txt works, but pages are not being served. Why?"
Two things a reviewer should confirm
Both are claims about existing behavior that the docs now assert, and both were wrong in my first draft:
An empty activity table proves nothing.
decide()records only on the paths that reach the API —served,passthrough,error. An open circuit, a negative-cached miss,serve_enabledoff, or an unrecognized UA all returnpass()without recording (class-citecue-proxy.php:106-127). The docs previously implied no row meant WordPress never ran, which would send someone chasing a CDN that isn't there.x-citecue: llms-txtdoes not prove the API is reachable. llms.txt serves from a 5-minute local cache, and keeps serving from it while the circuit is open (class-citecue-llms-txt.php:80). So a healthy-looking llms.txt is compatible with a rejected key. The README now documents flush-then-retest as the way to separate those.If either reading is off, the wording should change with it.
Testing
None — no executable change.
composer testunaffected.🤖 Generated with Claude Code