Conversation
…all HTML responses per spec §4.7 + §8
… Purpose 1 when gdpr_applies
…patch_auction dispatch_auction was building AuctionContext with a placeholder Request (GET https://placeholder.invalid/) that carried no headers. Prebid's request_bids copies User-Agent, x-forwarded-for, Referer, Accept-Language, and cookies from context.request before sending to Prebid Server, so SSPs received stripped requests and returned empty bids. Fix: dispatch SSP requests before req.send_async(), using the original request directly as AuctionContext.request. DispatchedAuction holds no lifetime reference to Request, so the borrow ends at return and req can be modified (restrict_accept_encoding, Host header) and sent to origin immediately after.
In collect_dispatched_auction, the select loop checked `auction_start.elapsed() >= deadline` after each SSP response and broke early if the 1500ms budget had elapsed. When origin TTFB + body download exceeded the auction budget, the check fired after collecting the first SSP response, abandoning the second SSP's already-buffered response. This left responses with only one (possibly errored) SSP, causing remaining_ms == 0 which skipped the mediator, and select_winning_bids on the partial set returned zero bids. The deadline break is wrong in this context: SSP HTTP connections are already bounded by the backend first_byte_timeout set at dispatch time (1000ms per provider). By the time collect is called at origin EOF, all SSPs have either responded or been errored by Fastly's host. The select() calls drain instantly — no WASM-level deadline enforcement is needed or safe. Also add info-level log statements at dispatch, collect, and write_bids_to_state to make the auction pipeline observable without requiring a dashboard.
In collect_dispatched_auction, the mediator was skipped when
remaining_budget_ms(auction_start, timeout_ms) == 0. In the async-dispatch
path, auction_start is set before pending_origin.wait(), so elapsed time
includes the full origin TTFB and body download. For heavy SSR pages
(autoblog), this exceeds the 1500ms SSP budget, making remaining_ms == 0
at every collection and causing the mediator to be permanently skipped.
The mediator (adserver_mock) is the primary bid source — SSPs alone
return no bids. Skipping it means window.__ts_bids == {} on every full
page load, while handle_page_bids (which uses the sequential run_auction
path) works correctly because it measures remaining time from after SSP
collection.
Fix: give the mediator its own configured timeout (mediator.timeout_ms())
instead of the exhausted SSP budget. This mirrors how run_parallel_mediation
works: the mediator's deadline is independent of SSP round-trip time.
Side effect: mediator backend name is now stable (always t1000 for adserver_mock)
rather than varying per request with remaining_ms.
Resolve conflicts in: - prebid.rs: keep both PBS stored-request tests (branch) and bid-param override rule validation tests (main) - settings.rs: keep both creative_opportunities (branch) and debug (main) fields in Settings struct - trusted-server.toml: keep [creative_opportunities] section from branch Update handle_page_bids to use compat::from_fastly_headers_ref pattern introduced by main's HTTP type migration (PR11), replacing direct use of fastly::Request with the generic Request<EdgeBody> for cookie parsing, request info extraction, and EC ID generation.
`classify_response_route` returns `BufferedProcessed` when HTML has
post-processors registered (e.g. the Next.js integration registers one
via `with_html_post_processor`). Unlike the `Stream` path, which drives
`one_behind_loop` to collect the dispatched auction at origin EOF, the
`BufferedProcessed` branch previously discarded `dispatched_auction`
entirely — so `ad_bids_state` stayed `None` and lol_html injected the
fallback `window.__ts_bids = {}` instead of real bids.
Fix: collect the in-flight dispatched auction in the `BufferedProcessed`
branch before calling `process_response_streaming`, using the same
`collect_dispatched_auction` + `write_bids_to_state` pattern that the
stream path uses. The `debug.auction_html_comment` injection is mirrored
here as well so the comment appears in both code paths when enabled.
html_escape_for_script now unicode-escapes <, >, & and U+2028/2029 in addition to \ and ". These characters allow a crafted bid value to break out of the <script> block or terminate the JS string in some parsers. AuctionOrchestrator::collect_dispatched_auction now computes remaining budget (A_deadline - elapsed) before invoking the mediator. If the budget is already exhausted the mediator is skipped and SSP bids are returned directly; otherwise the mediator timeout is capped at the tighter of its configured value and the remaining budget, preventing it from running past A_deadline.
…erride Pass the real incoming request to AuctionContext in handle_page_bids instead of a placeholder — SSPs now receive browser UA, referer, and cookies on SPA navigation bids. Guard Cache-Control in finalize_response so operator response_headers cannot overwrite the private/no-store directives set for per-user HTML and page-bids responses. Disable auction_html_comment debug flag in trusted-server.toml.
The mock mediator endpoint does not echo nurl/burl/ad_id back in its response. Build a bid index in request_bids keyed by (provider, slot_id, bidder) — where bidder is recovered from the echoed crid field — and restore the fields in parse_mediation_response from the original SSP bids. Fixes the spec requirement: both nurl and burl must travel in __ts_bids for client-side sendBeacon firing on slotRenderEnded (§4.5).
APS reads user agent from request.device — without it, real APS bids arrive with wrong or missing device targeting. Pass the incoming UA from both the page-load and page-bids auction paths.
slotRenderEnded gives a div element id via getSlotElementId(), but __ts_bids is keyed by slot id. Build a divToSlotId map during slot setup (matching the TS implementation) and use it in the event handler. Without this, nurl/burl beacons and hb_adid match checks silently fail in the server-rendered fallback whenever div_id != id.
PBS bidder credentials (mocktioneer, criteo placeholder params) were being sent directly to PBS on every auction request. Per the design spec, PBS bidder params belong in PBS stored requests keyed by slot ID, not in the edge config file. Removes PbsSlotParams struct, SlotProviders.pbs field, the to_ad_slot wiring block, and the corresponding test. Slots without inline bidder params trigger the existing storedrequest fallback path in the Prebid provider. Closes #697
- Rewrite misleading comment in apply_floor_prices: price=None bids pass through in the parallel-only path because decoding is deferred; in the mediation path the mediator decodes prices before this function runs - Add test: decoded APS bid below slot floor is dropped - Add test: decoded APS bid at or above slot floor is kept Closes #698
- Expand handle_auction doc: inline-params vs stored-request paths, config passthrough and allowed_context_keys, response headers - Document AdRequest, AdUnit, BidConfig with the stored-request contract: absent/empty bids → empty bidders map → PBS stored-request fallback - Add tests for convert_tsjs_to_auction_request: - No bids → empty bidders map (stored-request path) - Inline bids → bidders map populated - Allowed config key passes through; disallowed key dropped - Invalid 3-element banner size returns error Closes #699
- Add debug log at no-match gate in handle_publisher_request and
handle_page_bids so operators can confirm the feature is inactive
on non-article URLs without reading source code
- Add test: empty slots file (kill-switch) returns slots:[] bids:{}
- Add test: URL not matching any slot pattern returns slots:[] bids:{}
Closes #700
- Clarify in handle_auction doc that /auction is for initial render and programmatic callers; scroll/refresh/SPA navigation is slim-Prebid's domain in Phase 1 - Note Phase 2 slot-template-aware refresh API as deferred future work - Add head_inserts doc clarifying __tsAdInit handles initial render only; slotRenderEnded fires win beacons but does not trigger refresh auctions Closes #702
…ie in auction requests
Both handle_publisher_request and handle_page_bids now set device.ip and device.geo on the AuctionRequest after build_auction_request returns. Previously these were hardcoded to None, causing PBS to infer the IP from the Fastly edge IP — bidders like PubMatic filter such requests as non-human traffic. Client-side auction (formats.rs) already wired these fields. Server-side now matches that behaviour.
* Tidy small review nits across the new auction surface Addresses review findings on #680: - P2-18 / price_bucket: reject NaN/Inf cpm up front before the (x * 100.0).floor() as u64 cast (Rust's NaN-cast behaviour is only safe by convention, not contract). Add test coverage. - P2-24 / publisher.rs::write_bids_to_state: drop the per-request log line from INFO to DEBUG so production logs don't spam a slot list on every page request. - P2-25 / publisher.rs::build_bids_script / build_ad_slots_script: serde_json::to_string of a Map / Vec is infallible -- use expect("should be infallible") instead of unwrap_or_else with a silent fallback that would mask any future bug. - P2-15 / prebid: drop the dead PrebidIntegrationConfig::suppress_nurl field -- declared but never read anywhere in the codebase. The no-op test it carried goes with it. * Extract GPT bootstrap script and guard double enableServices Addresses review findings on #680: - P2-2: the inline `__tsAdInit` bootstrap injected at <head> called googletag.pubads().enableSingleRequest() and googletag.enableServices() unconditionally. The TS bundle's later-installed version guards both with a `__tsServicesEnabled` flag — the inline version did not, so the publisher's own GPT init code (or an upgrade where the bundle loads before the bids script runs) caused double-enable + duplicate refresh(), producing duplicate ad requests on every load. Now both inline and bundle converge on the same flag and only invoke `refresh(newSlots)` for the slots this pass actually defined, never the global slot list. - P2-26: the bootstrap source moves out of a concat!() literal block in head_inserts() into a syntax-highlighted gpt_bootstrap.js file pulled in via include_str!. The Rust side keeps a single named constant, GPT_BOOTSTRAP_JS, so future edits diff cleanly. Adds a regression test that asserts the guard flag is present and that unbounded refresh() is gone. * Harden auction-orchestrator state cleanup Addresses review findings on #680: - P2-6: APS provider held its per-request slot_id_map across request boundaries when the same Wasm instance was reused (the mock provider already cleared its bid_index, APS did not). parse_response now `std::mem::take`s the map so it can never carry over to a subsequent request. - P2-7: apply_floor_prices used to silently pass bids with `price=None` through the floor filter. Today both production callers decode/skip None before calling, so the pass-through was dead code that would, if revived, cause winning_bids.len() to overcount what build_bid_map ships to the client. Drop the None branch and update the existing test to pin the new contract: callers must decode prices first. * Harden /__ts/page-bids and creative-opportunities loading Addresses review findings on #680: - P2-4: /__ts/page-bids ran the full SSP auction for every request without gating crawlers or prefetches the way the publisher path does, exposing partner request quota to client-side spraying. Apply the same is_bot / is_prefetch gate handle_publisher_request uses: slots are still returned (so HTML structure is unchanged) but the auction is short-circuited. New regression tests cover both gates. - P2-5 + P2-13: the adapter previously parsed CREATIVE_OPPORTUNITIES_TOML on every request and `expect("should parse...")` on failure, so a malformed embedded TOML (CI-bypassed binary patch, future schema change, anything build.rs didn't catch) would panic every request. Parse it lazily once per Wasm instance via LazyLock; on parse failure log an error and fall back to the documented "empty slots file = feature disabled" state instead of panicking. * Consolidate HTML stream processor + auction helpers Addresses review findings on #680: - P2-3 + P2-16: create_html_stream_processor previously built HtmlProcessorConfig inline, bypassing HtmlProcessorConfig::from_settings. A future edit to from_settings (e.g. to read a new flag from Settings) would silently miss the streaming-with-auction-hold path. Now goes through from_settings, with a new with_ad_state(...) builder method that layers the ad_slots_script / ad_bids_state fields on top. The `_settings` argument on create_html_stream_processor is now actually used and is no longer underscore-prefixed. - P2-17: the auction debug-comment prepend logic was duplicated between one_behind_loop (path=stream) and the BufferedProcessed arm (path=buffered). Extracted as `prepend_auction_debug_comment(label, result, state)` so the single source of truth gets one definition and the only difference between paths is the label string. - P2-14: build_auction_request was at the project's 7-argument cap. Bundled (matched_slots, request_path, co_config) into a new MatchedSlotsContext struct — the three fields always travel together anyway. The function now takes 5 args, leaving headroom for future per-request inputs without breaking the project rule. * Document AuctionContext request contract and pin mediator placeholder Addresses review finding P2-1 on #680: collect_dispatched_auction and the publisher-side collectors built fastly::Request::get("https://placeholder.invalid/") on the fly and stuffed it into AuctionContext.request before invoking the mediator. That worked today only because the current mediator (adserver_mock) does not read request headers. A future PBS-as-mediator change would silently lose DNT, client IP, UA, etc., because the placeholder has none of them. The structural fix that surfaces the contract: - AuctionContext::request now carries a thorough doc-comment that explicitly distinguishes the dispatch path (real client request, all headers available) from the collect path (synthetic placeholder, no client headers — mediators MUST snapshot at dispatch time if they need them). - MEDIATOR_PLACEHOLDER_URL is a single named const so the three inline `"https://placeholder.invalid/"` literals across orchestrator.rs and publisher.rs converge on one source of truth. Tests / debug-asserts can compare against it. - make_collect_context now debug_asserts that the placeholder argument matches the canonical URL, so any future caller that accidentally forwards a real client request through the collect path fails loudly in debug builds instead of triggering an invisible header-loss bug. A full snapshot-headers refactor (so mediators can read real client headers across the await) is tracked as follow-up; this PR makes the existing contract impossible to violate accidentally. * Apply cargo fmt to fix-up commits * Forward ts-eids cookie through /auction endpoint Addresses pass-2 review finding P3-1 on #680: handle_auction read the request's cookie jar for consent purposes but never threaded it into the AuctionRequest, so the Extended User IDs from the `ts-eids` cookie were dropped on the floor. The publisher-page and /__ts/page-bids paths both call parse_ts_eids_cookie(cookie_jar.as_ref()); the /auction endpoint now does the same so programmatic callers (slim-Prebid, native apps, server-to-server integrations) get parity instead of silently losing identity data. * Pass-3 review fixes for #680: bot helpers, dead arg, pre-compiled globs Addresses pass-2 review findings on #680: - Dedupe the bot-UA and prefetch checks. handle_publisher_request and handle_page_bids both inlined the same 5-entry crawler list and the sec-purpose/purpose header check. Promoted to two pub(crate) helpers in publisher.rs (is_bot_user_agent, is_prefetch_request) backed by a single BOT_USER_AGENT_FRAGMENTS const, so the two paths can't drift. - Drop the dead gam_network_id argument on CreativeOpportunitySlot::to_ad_slot. The parameter was already being discarded via `let _ = gam_network_id;`. Removing it also drops the now-unused co_config field from MatchedSlotsContext. - Pre-compile slot glob patterns once per Wasm instance. Each call to matches_path previously ran Pattern::new (per pattern, per slot, per request). The slots live in the adapter's LazyLock-cached CreativeOpportunitiesFile, so adding a #[serde(skip)] compiled_patterns cache populated by CreativeOpportunitiesFile::compile() at file-load time turns matches_path into a Vec<Pattern>::iter().any() lookup on the hot path. The fallback (compile-per-call) is preserved for hand-built slots in tests. Two new regression tests pin the cache. * Address deferred perf findings from PR #680 review P2-8: Promote STREAM_CHUNK_SIZE (8192) to module scope and use it for both the brotli Decompressor and CompressorWriter internal buffers, aligning them with the read buffer in one_behind_loop. P2-9: Replace RwLock<Option<String>> with Mutex<Option<String>> for ad_bids_state across publisher.rs and html_processor.rs. Single-threaded WASM gains no parallelism from RwLock; Mutex is simpler and avoids the reader-writer bookkeeping the workload cannot use. P2-10: Rewrite html_escape_for_script as a single-pass char loop with one pre-allocated String instead of seven sequential String::replace allocations. All existing escape tests pass unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
August release candidate: current
mainplus the remaining open PRs, staged together for RC validation. Supersedes the July RC (#919).Already included through main:
In Review:
Verification
cargo fmt --all -- --check-D warningscargo test-fastly(2,075 tests via Viceroy),test-axum,test-cloudflare,test-spin./scripts/test-cli.sh(34 tests, exercising the Upgrade EdgeZero to the deploy-actions branch #940run.rschanges)