Skip to content

Add TS admin endpoint to look up EC by id - #928

Open
prk-Jr wants to merge 28 commits into
mainfrom
feat/admin-ec-lookup-endpoint
Open

Add TS admin endpoint to look up EC by id#928
prk-Jr wants to merge 28 commits into
mainfrom
feat/admin-ec-lookup-endpoint

Conversation

@prk-Jr

@prk-Jr prk-Jr commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds GET /_ts/admin/ec/{id} (explicit EC ID) and GET /_ts/admin/ec (EC ID from the caller's ts-ec cookie) so operators can inspect EC identity graph entries without KV console access.
  • The EC response includes a derived auction view: the exact EIDs the auction would attach for the entry, plus the reason each stored partner ID was skipped (empty_uid, not_in_registry, bidstream_disabled) — making KV-to-auction EID propagation debuggable in one call. Corrupt or legacy-schema entries stay inspectable via a new raw lookup that reports entry_error + raw_body instead of failing closed.
  • Adds GET /_ts/admin/eids, the client-side half of the picture: it echoes the request's ts-eids and sharedId cookies and previews what cookie ingestion would write into the EC entry's ids map (matched partner UIDs, deduplicated exactly like the ingestion path, plus dropped sources). Pure request inspection with no KV access, so every adapter serves the real handler; always responds 200.

Changes

File Change
crates/trusted-server-core/src/ec/admin.rs New handle_admin_ec_lookup + handle_admin_eids_lookup handlers, response payload types, 14 unit tests
crates/trusted-server-core/src/ec/kv.rs New KvIdentityGraph::lookup_raw returning raw body/metadata/generation without validation
crates/trusted-server-core/src/ec/prebid_eids.rs Expose cookie-ingestion collectors pub(crate) so the EIDs preview reuses them instead of reimplementing
crates/trusted-server-core/src/ec/mod.rs Register admin module
crates/trusted-server-core/src/settings.rs Add all three routes to Settings::ADMIN_ENDPOINTS; update coverage tests
crates/trusted-server-adapter-fastly/src/app.rs Route EC + EIDs paths to the real handlers; route-table tests. The EC arm deliberately builds its own identity graph instead of reusing the bot-gated ec.kv_graph, so operator curl requests work
crates/trusted-server-adapter-axum/src/app.rs EC routes → local 501 (AdminEcNotSupported); EIDs route → real handler
crates/trusted-server-adapter-axum/tests/routes.rs Route-registration, authenticated-501 (EC) and authenticated-200 (EIDs) tests
crates/trusted-server-adapter-cloudflare/src/app.rs EC routes → local 501; EIDs route → real handler
crates/trusted-server-adapter-cloudflare/tests/routes.rs Route-registration, authenticated-501 (EC) and authenticated-200 (EIDs) tests
crates/trusted-server-adapter-spin/src/app.rs EC routes → local 501; EIDs route → real handler; extend fallback-methods table
crates/trusted-server-adapter-spin/tests/routes.rs Authenticated-501 (EC) and authenticated-200 (EIDs) tests

Closes

Closes #921

Test plan

  • cargo test-fastly && cargo test-axum (also cargo test-cloudflare && cargo test-spin)
  • cargo clippy-fastly && cargo clippy-axum (also clippy-cloudflare, clippy-cloudflare-wasm, clippy-spin-native, clippy-spin-wasm)
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run (JS untouched — left to CI)
  • JS format: cd crates/trusted-server-js/lib && npm run format (JS untouched — left to CI)
  • Docs format: cd docs && npm run format (docs untouched — left to CI)
  • WASM build: cargo check -p trusted-server-adapter-fastly --target wasm32-wasip1
  • Manual testing via fastly compute serve: verified auth gating, the EC lookup against the seeded Viceroy entry, and the EIDs echo against real browser cookies; this surfaced (and the branch fixes) an initial bug where the bot-gated KV graph made curl requests return 501
  • Other: integration parity suite (cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity)

Hardening note

All three paths were added to Settings::ADMIN_ENDPOINTS, so startup validation fails closed (a Configuration error via Report, no panic!/unwrap()/expect()) for any config whose basic-auth handler regex does not cover them; prefix-style regexes like the example config's ^/_ts/admin remain valid. Covered by the updated uncovered_admin_endpoints_* and admin_endpoints_match_fastly_router tests. Routes are registered explicitly on every adapter so they never fall through to the publisher fallback, which would forward the caller's Authorization header to the origin.

Checklist

  • Changes follow CLAUDE.md conventions
  • No unwrap() in production code — use expect("should ...")
  • Uses log macros (not println!)
  • New code has tests
  • No secrets or credentials committed

Adds GET /_ts/admin/ec/{id} (explicit EC ID) and GET /_ts/admin/ec
(EC ID from the caller's ts-ec cookie) so operators can inspect EC
identity graph entries and debug KV-to-auction EID propagation.

The core handler returns the stored KvEntry verbatim (including raw
consent strings and partner UIDs), the KV metadata mirror, the store
generation marker, and a derived auction view showing exactly which
EIDs the auction would attach and why each stored partner ID was
skipped (empty_uid, not_in_registry, bidstream_disabled). Corrupt
entries are returned with the parse error and raw body via the new
KvIdentityGraph::lookup_raw instead of failing closed.

The routes join Settings::ADMIN_ENDPOINTS so startup validation
rejects configs whose basic-auth handler regex does not cover them.
The EC identity graph is Fastly KV backed, so the Axum, Cloudflare,
and Spin adapters register the routes to local 501 responses, keeping
them off the publisher fallback that would forward the Authorization
header to the origin.

Closes #921
@prk-Jr prk-Jr self-assigned this Jul 17, 2026
prk-Jr and others added 5 commits July 17, 2026 18:46
The dispatch arm reused EcRequestState::kv_graph, which is deliberately
None for clients that fail the browser gate. Operators hit this
auth-gated endpoint with curl, so every lookup returned 501 as if no
EC store were configured. Build the identity graph directly from
settings instead, and document why the bot-gated copy must not be used.

Also point the bare-route no-cookie 404 at the explicit-id route, since
the ts-ec cookie (Domain-scoped, Secure) cannot exist on localhost.
Adds GET /_ts/admin/eids, complementing the EC lookup endpoint with the
client-side half of EID propagation: it decodes the request's ts-eids
and sharedId cookies and previews what cookie ingestion would write
into the EC entry's ids map — matched partner UIDs (deduplicated
exactly like the ingestion path) and unmatched sources that would be
dropped.

The endpoint always responds 200; missing or malformed cookies are
reported in the payload rather than as errors. It is pure request
inspection with no KV access, so every adapter serves the real handler.
The path joins Settings::ADMIN_ENDPOINTS for basic-auth coverage
validation.
Review feedback on the admin EC lookup asked for readable dates. The
echoed entry now carries derived created_iso and consent.updated_iso
fields (yyyy-MM-ddTHH:mm:ss.SSSZ) next to the stored unix-seconds
values, which stay untouched so the echo remains faithful to KV.
@aram356 aram356 added this to the 202608 milestone Aug 13, 2026
Resolve route-registration conflicts in the axum, spin, and fastly adapters.

Main's `/__ts/page-bids` legacy alias landed next to the admin EC lookup
routes, so the named-route array lengths and a shared `#[test]` attribute
collided:

- axum `named_routes` and spin `named_fallback_paths` now declare 16 entries
  (13 from main plus `/_ts/admin/ec`, `/_ts/admin/ec/{id}`, `/_ts/admin/eids`).
- fastly keeps both colliding tests: `admin_ec_lookup_routes_are_registered`
  and `page_bids_serves_canonical_path_and_deprecated_alias`.
@prk-Jr
prk-Jr requested review from ChristianPavilonis and aram356 and removed request for aram356 August 17, 2026 16:25

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Request changes. This adds sensitive admin diagnostics, but the current route/auth wiring permits an authentication bypass and can forward admin credentials to the publisher origin. The inline comments also cover Fastly-only write side effects, loss of raw KV data, and the missing operator API contract.

Comment thread crates/trusted-server-core/src/settings.rs
Comment thread crates/trusted-server-adapter-fastly/src/app.rs
Comment thread crates/trusted-server-adapter-fastly/src/app.rs Outdated
Comment thread crates/trusted-server-core/src/ec/admin.rs Outdated
Comment thread crates/trusted-server-core/src/ec/admin.rs

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Review:

Summary

Reviewed the admin EC/EID lookup routes, cross-adapter registration, KV handling, response behavior, and authorization validation. I found one high-severity authorization-coverage gap for the parameterized EC lookup route.

Findings by severity

P0 / Blockers

None.

P1 / High

One inline finding.

P2 / Medium

None.

P3 / Low

None.

CI / Existing Reviews

The displayed CI check runs for head 866bcd4c24be67ca5434fff82727375fbdfc5a45 are successful. Immediately before submission, the PR had no existing reviews or inline review comments. Local tests were not run during this review.

Comment thread crates/trusted-server-core/src/settings.rs

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Requesting changes: the Fastly admin EC GET lookup currently runs the mutating EC finalization lifecycle. I also found an incomplete dynamic-route auth-coverage validation and a documentation mismatch for authentication failures.

The currently displayed PR checks are passing. I reviewed existing inline feedback and did not repeat items resolved at the current head.

// lifecycle would attach finalization state and could ingest those cookies
// into KV after the handler returns, violating the endpoint's read-only
// contract.
if matches!(handler, NamedRouteHandler::AdminEidsLookup) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — Admin EC GET lookups are not read-only. This early return exempts only AdminEidsLookup; AdminEcLookup continues through build_ec_request_state and response finalization. A browser-shaped authenticated lookup carrying ts-eids/sharedId can therefore ingest those values into the caller's EC row, and a withdrawal signal can write a tombstone and trigger pull sync. A diagnostic GET should not mutate identity state.

Please dispatch AdminEcLookup before EC setup as well (while constructing its KV graph directly), and add a regression test that a cookie-bearing lookup response carries no EcFinalizeState and produces no KV write.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f3776a9, with the regression assertion tightened in 436e25f. execute_named now dispatches both admin diagnostics before request filters, EC setup, and finalization, and constructs the EC lookup graph directly. The browser-shaped cookie regression confirms that the handler runs, no EcFinalizeState is attached, and no Set-Cookie mutation is emitted; without finalization state, cookie ingestion, tombstone writes, and pull sync cannot run.

"/_ts/admin/eids",
];

const ADMIN_EC_ID_AUTH_PROBE: &str = concat!(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — A single fixed probe does not prove coverage of the dynamic route. For example, a handler matching ^/_ts/admin/ec/[a-f0-9]{64}[.][a-z]{6}$ accepts this .abc123 probe and lets startup succeed, but valid EC IDs whose alphanumeric suffix contains a digit or uppercase character later fail closed as a configuration 500.

Require coverage of the complete dynamic ID language (or at least validate a character-class corpus including uppercase and numeric suffixes) and add a mixed-case/digit regression case.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in de61c43. Dynamic EC auth coverage now probes both a lowercase suffix (.abc123) and a mixed-case/digit suffix (.Ab12Z9), and startup requires every probe to match. The added regression rejects a lowercase-only handler while the valid alphanumeric handler fixture covers both probes.

Comment thread docs/guide/api-reference.md Outdated

## Admin Diagnostic Endpoints

These endpoints expose sensitive identity and cookie data and require HTTP Basic Authentication. Configure a handler that covers the entire `/_ts/admin` namespace; startup rejects configurations that do not protect every admin route. All responses are JSON with `Cache-Control: no-store`.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 — The documented response contract excludes authentication failures. The shared Basic-auth middleware returns a plaintext 401 Unauthorized response without Cache-Control: no-store, so it is not JSON/no-store as this sentence says. Please either qualify this statement to successful diagnostic-handler responses or update the auth response to meet the documented contract.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in be434c2 and refined in 436e25f. The documentation now limits the JSON and no-store contract to normal diagnostic-handler responses after successful authentication, and explicitly documents the shared plaintext 401 challenge plus unexpected plaintext 5xx responses as outside that contract.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds three basic-auth-gated diagnostics (GET /_ts/admin/ec[/{id}], GET /_ts/admin/eids) plus admin-namespace hardening (fail-closed startup coverage, runtime fail-closed auth for unmatched admin paths, publisher-fallback denial). The hardening work and cross-adapter test coverage are in good shape, but the Fastly EC lookup still runs the mutating EC finalization lifecycle, the auth-coverage probe under-represents the EC ID language, and the documented response contract is inaccurate for 401s.

Blocking

🔧 wrench

  • EC lookup GET is not read-only: AdminEcLookup still flows through build_ec_request_state, request filters, and EC finalization; a browser-shaped authenticated lookup ingests the caller's ts-eids/sharedId cookies into KV and can contaminate the very record being inspected (crates/trusted-server-adapter-fastly/src/app.rs:595)
  • Auth-coverage probe misses mixed-case EC IDs: a handler regex matching only lowercase suffixes passes startup but produces ID-dependent runtime 500s (crates/trusted-server-core/src/settings.rs:2221)
  • Docs overstate the response contract: the 401 challenge is plaintext without Cache-Control: no-store (docs/guide/api-reference.md:586)

Non-blocking

🤔 thinking

  • Ingest preview drops partner-matched sources with no valid UID: they appear in neither matched nor unmatched, hiding the exact failure mode operators would debug (crates/trusted-server-core/src/ec/admin.rs:489)

♻️ refactor

  • Third copy of extract_cookie_value in core: hoist to a shared helper (crates/trusted-server-core/src/ec/admin.rs:516)

⛏ nitpick

  • Missing X-Content-Type-Options: nosniff on the shared admin JSON builder (crates/trusted-server-core/src/ec/admin.rs:537)
  • tombstone doc comment incomplete: also absent on schema-deserialization failure (crates/trusted-server-core/src/ec/admin.rs:114)
  • No unauthenticated 401 test for the new routes: every new-route adapter test sends valid credentials; the existing 401 tests cover only the keys routes. Given this PR's hardening story is auth coverage, one GET /_ts/admin/ec → 401 test per adapter would pin it.

📝 note

  • Deliberate breaking config change: configs whose admin handler covers only the keys endpoints (e.g. ^/_ts/admin/keys/) now fail startup. Correct fail-closed behavior, but worth an explicit upgrade/release note so operators are not surprised at deploy time.

CI Status

  • fmt: PASS
  • clippy: PASS
  • rust tests (fastly/axum/cloudflare/spin/parity/CLI): PASS
  • js tests: PASS
  • docs/format checks: PASS

Comment thread crates/trusted-server-adapter-fastly/src/app.rs Outdated
Comment thread crates/trusted-server-core/src/settings.rs Outdated
Comment thread docs/guide/api-reference.md Outdated
Comment thread crates/trusted-server-core/src/ec/admin.rs Outdated
Comment thread crates/trusted-server-core/src/ec/admin.rs Outdated
Comment thread crates/trusted-server-core/src/ec/admin.rs
Comment thread crates/trusted-server-core/src/ec/admin.rs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add TS admin endpoint to look up EC by id

3 participants