Skip to content

Enable Streaming Chunks for responses to improve TTFB on TS #320

@jevansnyc

Description

@jevansnyc

What it does: Flush to the browser immediately. Run the auction concurrently. Stream the body as origin HTML arrives, inject auction results when they're ready.

What changes in TS: The process_response_with_pipeline function in [proxy.rs:141-169] currently buffers the entire body before returning. This needs to become a true streaming pipeline, parse and forward HTML in chunks, holding only the ad-config injection point. Fastly's Rust SDK supports streaming response bodies via Body streaming APIs so we should be good.

Browser caching changes: With streaming, the response headers go out immediately, so something like this:

Cache-Control: max-age=60, stale-while-revalidate=300
Surrogate-Control: max-age=300, stale-while-revalidate=3600
Browser caches for 60s, serves stale (instantly) for 300s while TS revalidates in background. On Fastly's side, CDN holds it 300s with 1hr SWR. Real users see TTFB of edge cache latency (~5-30ms) on virtually every visit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions