feat: pluggable rate limiting for transcoded REST routes

[polaz]

Problem

structured-proxy runs embedded on each service instance and fronts public REST surfaces of gRPC services. Deployments that expose transcoded routes to untrusted clients currently have to bolt rate limiting on elsewhere (upstream service or a separate gateway), which duplicates auth-context parsing and splits edge policy across components.

Constraint that shapes the whole design: as an embedded data-plane component, the proxy must add zero blocking latency on the request path. A synchronous per-request round-trip to a shared store (Redis Lua/CAS) is therefore off the table. Every decision is made locally; any shared state is reconciled asynchronously, off the hot path.

Design

Two layers. The hot path is always local.

1. Local GCRA shaper (always on). Per-instance Generic Cell Rate Algorithm (token-bucket equivalent, single stored timestamp per key). Shapes legitimate bursts through while throttling sustained abuse to the steady rate. No boundary burst. Pure in-memory, nanosecond cost.

2. Global sliding-window gate (optional, async Redis). When a shared store is configured, a background loop pushes this instance's consumption deltas (INCRBY) and pulls the fleet aggregate (MGET) on an interval, never on the request path. The global view uses a sliding-window counter (current + previous epoch, boundary-smoothed) so the fleet-wide cap has no boundary burst either. A request is admitted iff the local GCRA admits it and the estimated global consumption plus this instance's un-pushed local delta is under the fleet budget. A lone-busy instance gets ~the full budget; a busy fleet is throttled globally. Geo-elastic.

Key extraction (who is limited), configurable per rule, with IP fallback for anonymous traffic:

• jwt_claim(name) — value of a claim from the JWT validated by the auth layer (provider-agnostic: structured-id, Okta, Entra, Keycloak via JWKS)
• header(name) — a named request header value (API-key style)
• ip — connection IP, trusted-proxy X-Forwarded-For aware

Limit resolution (what limit applies to a key), priority chain, configurable order:

• jwt — tier name from a claim (ratelimit_tier: "premium" → config profile) or direct numbers (ratelimit_rpm / ratelimit_burst)
• service — resolved from an external service, cached with TTL and refreshed asynchronously, never blocking (same shape as forward-auth); returns a tier name or explicit numbers
• profile — a static named tier from config
• default — fallback profile

Named limit profiles ({ rate, burst }) are defined in config as data; tier-name indirection lets operators retune numbers without re-issuing tokens or changing the limit service.

Response contract: RateLimit-Limit / RateLimit-Remaining / RateLimit-Reset (draft-ietf-httpapi-ratelimit-headers) on allowed and rejected requests; 429 with Retry-After on rejection.

Degradation: Redis unreachable → local-only (global ≈ N × local, documented). Limit service unreachable → last cached value / default. Neither configured → pure local GCRA against the config profile.

Acceptance criteria

• Off by default; enabling requires config only, zero code changes in consuming binaries
• Hot-path decision is always local (no synchronous store round-trip); this is an enforced invariant
• Local GCRA shaper: burst boundary, steady-rate refill, backward-clock tolerance
• Optional async global reconciliation via a shared store (INCRBY push / MGET pull on an interval, off the hot path, native commands, no Lua)
• Global gate uses a sliding-window counter (no boundary burst at the fleet level)
• Approximate global limit with a documented, tunable overshoot bound (≈ (N-1) × rate × sync_interval); degrades to local-only on store outage
• Key extraction: jwt_claim (provider-agnostic) / header / ip, with IP fallback for anonymous
• Limit resolution from JWT claim (tier name or direct numbers) and from an external service (cached + async, non-blocking), plus static config profiles and a default
• Named limit profiles { rate, burst } as config data, with tier-name indirection
• Standard rate-limit response headers on allowed and rejected requests; Retry-After on 429
• Unit tests: GCRA math, sliding-window math, limit-resolution chain, key extraction, headers; integration test with a Redis-protocol container; local-only degradation
• Docs: config reference, deployment modes, sizing guidance, overshoot model

Estimate

4d

[coderabbitai]

🔗 Related PRs

#20 - chore: crates.io badges, README accuracy, release-plz migration [merged]
#25 - feat(shield): enforce rate limiting via pluggable store [merged]
#32 - feat(transcode): propagate W3C trace-context and request deadlines to gRPC [merged]
#36 - feat(authz): external authorization via Envoy ext_authz gRPC [merged]
#42 - perf(transcode): remove per-request route allocations on the hot path [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!