diff --git a/README.md b/README.md index d1d0629..1079328 100644 --- a/README.md +++ b/README.md @@ -2,19 +2,28 @@ This repository contains independently versioned, public systems-engineering challenges for the Rough Stack platform. Each challenge owns its scenario, contestant -interface, starter, deterministic simulator, public tests, smoke benchmark, -baseline, and public metric contract. +interface, learner scaffold, deterministic simulator, public tests, smoke +benchmark, public baseline, and metric contract. The Rough Stack platform is maintained separately. Challenge packages depend on -versioned contracts; the platform consumes validated, immutable challenge bundles -and does not import challenge source code. +versioned contracts. Today, a local platform checkout validates and reads a +separate challenge checkout from disk. Published immutable bundles are a future +distribution format, not a requirement for running or contributing locally. The current manifest filename and protocol fields retain the `arena` name for compatibility with the pre-release platform. They will change only through a versioned contract migration. -Official workloads, hidden cases, exploit tests, reference implementations, and -calibration data are intentionally absent from this repository. +Official workloads, hidden cases, exploit tests, private reference +implementations, and calibration data are intentionally absent from this +repository. Public baseline implementations remain visible for reproducibility. + +Each package's manifest names two different source files. `submission.starter` +is the incomplete scaffold shown in the browser. `submission.entrypoint` is +the file replaced by submitted source inside a temporary execution workspace; +the repository copy is also the tested public baseline. Keeping these separate +lets learners start with a genuine problem without removing the baseline used +for public smoke and feasibility checks. ## Verification @@ -30,11 +39,15 @@ Changed manifests must also pass the Rough Stack platform validator. Every smoke benchmark must emit one valid result object and reproduce the same canonical result when repeated with the same seed. -## Initial build order +## Local platform integration -The first public implementation slice is `cache-pressure-easy`. The remaining -families are added one at a time after the preceding challenge passes tests, -determinism checks, and the platform manifest validator. +Clone this repository beside `roughstack/web-platform` and follow the +[web-platform local setup](https://github.com/roughstack/web-platform#running-it). +The platform's `BYTEARENA_ARENA_ROOTS` setting should point to this repository's +`challenges/` directory. Its opt-in real-run check submits the easy-cache public +baseline and learner scaffold to the local Docker runner. Public baselines are +practice feasibility witnesses; the generated arenas do not yet provide ranked +scores or multi-language submissions. ## Contributing and security diff --git a/challenges/cache-pressure-easy/arena.yaml b/challenges/cache-pressure-easy/arena.yaml index 3038d3f..7c0ce1f 100644 --- a/challenges/cache-pressure-easy/arena.yaml +++ b/challenges/cache-pressure-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: cache-pressure-easy - version: 1.0.0 + version: 1.0.1 title: Cache Pressure visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/cache.go + starter: learner/starter.go interface: bytearena.cache.Cache/v1 build: command: ["go", "test", "./challenges/cache-pressure-easy/..."] diff --git a/challenges/cache-pressure-easy/harness/harness.go b/challenges/cache-pressure-easy/harness/harness.go index 411900a..bf36e47 100644 --- a/challenges/cache-pressure-easy/harness/harness.go +++ b/challenges/cache-pressure-easy/harness/harness.go @@ -3,6 +3,7 @@ package harness import ( "bytes" + "fmt" "strconv" "github.com/roughstack/challenges/challenges/cache-pressure-easy/contract" @@ -11,7 +12,7 @@ import ( const ( ArenaID = "cache-pressure-easy" - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" WorkloadID = "public-smoke-v1" valueBytes = 96 ) @@ -62,8 +63,8 @@ func DefaultPublicConfig() PublicConfig { } // Evaluate runs a policy-independent request stream against a contestant cache. -func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result { - result := Result{ +func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) (result Result) { + result = Result{ ProtocolVersion: 1, ArenaID: ArenaID, ArenaVersion: ArenaVersion, @@ -75,8 +76,29 @@ func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result WorkloadID: WorkloadID, }, } + defer func() { + if recovered := recover(); recovered != nil { + result.Verdict = "fail" + result.Violations = append(result.Violations, fmt.Sprintf("contestant panicked: %v", recovered)) + } + }() + if factory == nil { + result.Verdict = "fail" + result.Violations = append(result.Violations, "cache factory is nil") + return result + } + if violation := validateCacheContract(seed, factory); violation != "" { + result.Verdict = "fail" + result.Violations = append(result.Violations, violation) + return result + } cache := factory(config.CapacityBytes) + if cache == nil { + result.Verdict = "fail" + result.Violations = append(result.Violations, "cache factory returned nil") + return result + } versions := make(map[uint64]uint64) operations := workload.Generate(workload.Config{ Seed: seed, @@ -140,6 +162,62 @@ func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result return result } +// validateCacheContract uses a fresh, small instance to check exact LRU +// semantics independently of the larger performance workload. Otherwise a +// cache that always misses could pass while implementing none of the contract. +func validateCacheContract(seed uint64, factory contract.Factory) string { + unit := contract.AccountedBytes([]byte{1}) + cache := factory(2 * unit) + if cache == nil { + return "cache factory returned nil during contract check" + } + a, b, c := seed+1, seed+2, seed+3 + first := []byte{0x11} + cache.Put(a, first, contract.RequestMeta{}) + first[0] = 0xff + cache.Put(b, []byte{0x22}, contract.RequestMeta{}) + if cache.UsedBytes() != 2*unit { + return "cache must admit and exactly account for two fitting entries" + } + value, ok := cache.Get(a, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{0x11}) { + return "cache must return the stored value without aliasing the input" + } + value[0] = 0xee + value, ok = cache.Get(a, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{0x11}) { + return "cache Get must return an owned copy" + } + cache.Put(c, []byte{0x33}, contract.RequestMeta{}) + if _, ok := cache.Get(b, contract.RequestMeta{}); ok { + return "cache must evict the least recently used entry" + } + for key, want := range map[uint64]byte{a: 0x11, c: 0x33} { + value, ok = cache.Get(key, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{want}) { + return "cache evicted or changed a more recent entry" + } + } + cache.Delete(a) + if _, ok := cache.Get(a, contract.RequestMeta{}); ok || cache.UsedBytes() != unit { + return "cache Delete must remove the key and release its accounted bytes" + } + cache.Put(c, make([]byte, 2*unit), contract.RequestMeta{}) + value, ok = cache.Get(c, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{0x33}) || cache.UsedBytes() != unit { + return "oversized replacement must leave the resident value intact" + } + zero := factory(0) + if zero == nil { + return "cache factory returned nil for zero capacity" + } + zero.Put(a, []byte{0x11}, contract.RequestMeta{}) + if _, ok := zero.Get(a, contract.RequestMeta{}); ok || zero.UsedBytes() != 0 { + return "zero-capacity cache must not admit entries" + } + return "" +} + func backingValue(seed, key, version uint64) []byte { value := make([]byte, valueBytes) state := seed ^ rotateLeft(key, 17) ^ rotateLeft(version, 41) diff --git a/challenges/cache-pressure-easy/harness/harness_test.go b/challenges/cache-pressure-easy/harness/harness_test.go index 8f72a3e..4a6746e 100644 --- a/challenges/cache-pressure-easy/harness/harness_test.go +++ b/challenges/cache-pressure-easy/harness/harness_test.go @@ -3,11 +3,48 @@ package harness import ( "encoding/json" "reflect" + "strings" "testing" + "github.com/roughstack/challenges/challenges/cache-pressure-easy/contract" "github.com/roughstack/challenges/challenges/cache-pressure-easy/starter" ) +type neverAdmitCache struct{} + +func (neverAdmitCache) Get(uint64, contract.RequestMeta) ([]byte, bool) { return nil, false } +func (neverAdmitCache) Put(uint64, []byte, contract.RequestMeta) {} +func (neverAdmitCache) Delete(uint64) {} +func (neverAdmitCache) UsedBytes() uint64 { return 0 } + +type ignoreDeleteCache struct{ contract.Cache } + +func (ignoreDeleteCache) Delete(uint64) {} + +func TestEvaluateRejectsCachesThatDoNotImplementTheContract(t *testing.T) { + tests := []struct { + name string + factory contract.Factory + want string + }{ + {"never admits", func(uint64) contract.Cache { return neverAdmitCache{} }, "admit"}, + {"ignores delete", func(capacity uint64) contract.Cache { + return ignoreDeleteCache{starter.NewCache(capacity)} + }, "Delete"}, + {"nil cache", func(uint64) contract.Cache { return nil }, "nil"}, + {"nil factory", nil, "nil"}, + {"panicking factory", func(uint64) contract.Cache { panic("broken") }, "panicked"}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + result := Evaluate(17, DefaultPublicConfig(), test.factory) + if result.Verdict != "fail" || len(result.Violations) == 0 || !strings.Contains(result.Violations[0], test.want) { + t.Fatalf("expected %q failure, got %+v", test.want, result) + } + }) + } +} + func TestEvaluateIsDeterministicAndWithinCapacity(t *testing.T) { config := DefaultPublicConfig() first := Evaluate(^uint64(0), config, starter.NewCache) diff --git a/challenges/cache-pressure-easy/learner/starter.go b/challenges/cache-pressure-easy/learner/starter.go new file mode 100644 index 0000000..d78b917 --- /dev/null +++ b/challenges/cache-pressure-easy/learner/starter.go @@ -0,0 +1,30 @@ +// Package starter is the learner-facing scaffold. The verified public +// implementation lives at the manifest entrypoint and is not prefilled here. +package starter + +import "github.com/roughstack/challenges/challenges/cache-pressure-easy/contract" + +type cache struct { + capacity uint64 + // TODO: track stored values, their accounted sizes, and recency. +} + +func NewCache(capacityBytes uint64) contract.Cache { + return &cache{capacity: capacityBytes} +} + +func (c *cache) Get(key uint64, meta contract.RequestMeta) ([]byte, bool) { + panic("TODO: return a private copy and update recency on a hit") +} + +func (c *cache) Put(key uint64, value []byte, meta contract.RequestMeta) { + panic("TODO: enforce the exact byte budget and LRU eviction") +} + +func (c *cache) Delete(key uint64) { + panic("TODO: remove the entry and its accounting") +} + +func (c *cache) UsedBytes() uint64 { + panic("TODO: report the charged bytes, including metadata") +} diff --git a/challenges/cache-pressure-easy/statement.json b/challenges/cache-pressure-easy/statement.json index 500c05f..fafeb82 100644 --- a/challenges/cache-pressure-easy/statement.json +++ b/challenges/cache-pressure-easy/statement.json @@ -9,19 +9,17 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.snapshot", "props": { - "label": "A cache read path", - "nodes": [ - { "label": "Request", "detail": "Get key 7" }, - { "label": "Cache", "detail": "Look up and refresh recency", "tone": "accent" }, - { "label": "Backing store", "detail": "Read only on a miss", "tone": "warning" }, - { "label": "Caller", "detail": "Receive an owned copy", "tone": "signal" } - ], - "edges": ["lookup", "miss", "fill + return"] + "title": "Oldest on the left · newest on the right", + "rows": [ + { "label": "After two puts", "cells": [{ "label": "A", "detail": "oldest", "tone": "warning" }, { "label": "B", "detail": "newest", "tone": "accent" }] }, + { "label": "Get A", "cells": [{ "label": "B", "detail": "oldest", "tone": "warning" }, { "label": "A", "detail": "newest", "tone": "signal" }], "note": "A hit changes order without fetching from the backing store." }, + { "label": "Put C", "cells": [{ "label": "A", "detail": "oldest", "tone": "accent" }, { "label": "C", "detail": "newest", "tone": "signal" }], "note": "B leaves because it was least recently used." } + ] } }, - "caption": "A hit skips the backing store. A miss fetches once, admits the value if it fits, and returns a copy." + "caption": "One hit changes the eviction decision. This two-entry example illustrates order; real capacity counts each value's bytes plus metadata." }, { "kind": "callout", @@ -37,9 +35,9 @@ }, { "kind": "example", - "input": "capacity = 2 entries\nPut(A), Put(B), Get(A), Put(C)", - "output": "recency: C → A\nevicted: B", - "explain": "`Get(A)` makes A newest. When C arrives, B is the oldest entry, so B leaves. Use a map for lookup plus an ordered recency list for constant-time movement." + "input": "capacity = 66 bytes (two 1-byte values, each charged 32 metadata bytes)\nPut(A, 1B), Put(B, 1B), Get(A), Put(C, 1B)", + "output": "newest → oldest: C, A\nevicted: B", + "explain": "`Get(A)` makes A newest. When C arrives, B is the oldest entry, so B leaves. A map from keys to nodes in a doubly linked list gives constant-time lookup, recency updates, and eviction." }, { "kind": "constraints", @@ -55,14 +53,14 @@ "items": [ { "title": "Make accounting exact", "md": "Centralize each entry's charged size and update it on insert, replacement, and eviction." }, { "title": "Separate lookup from order", "md": "Use one structure to find a key and another to move that entry to the newest position." }, - { "title": "Optimize after correctness", "md": "A linear list can pass semantics; constant-time recency maintenance improves `metadata_work`." } + { "title": "Optimize after correctness", "md": "A linear list can pass semantics; a map plus linked list improves actual lookup and update time. The public `metadata_work` counter does not measure that internal time." } ] }, { "kind": "callout", "tone": "note", - "title": "What the score is saying", - "md": "Fewer `backing_reads` means the policy retained useful data. Lower `metadata_work` means the same decisions required less bookkeeping. Correctness is always the gate." + "title": "What the metrics say", + "md": "`backing_reads` counts misses on the public trace. `metadata_work` counts harness-visible calls and accounting checks; it does not measure the time your internal recency structure takes. Correctness is the gate; this practice challenge is not ranked yet." } ] } diff --git a/challenges/cache-pressure-medium/README.md b/challenges/cache-pressure-medium/README.md index 431ddac..b01265c 100644 --- a/challenges/cache-pressure-medium/README.md +++ b/challenges/cache-pressure-medium/README.md @@ -60,7 +60,10 @@ protection against scan pollution. budget at peak occupancy. Lower means the cache exploited its budget more fully; it is a soft signal alongside the hit-rate and cost metrics. -Correctness gates scoring. Public smoke output is reproducible but unofficial; +The default public workload also requires a request hit rate of at least 4000 +basis points (40%). This rejects a never-admit policy without requiring one +particular eviction or admission algorithm. Correctness gates scoring. Public +smoke output is reproducible but unofficial; normalization and calibration belong to the platform and private judge. ## Verify diff --git a/challenges/cache-pressure-medium/arena.yaml b/challenges/cache-pressure-medium/arena.yaml index 54d37c1..b74f6d0 100644 --- a/challenges/cache-pressure-medium/arena.yaml +++ b/challenges/cache-pressure-medium/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: cache-pressure-medium - version: 1.0.0 + version: 1.0.1 title: Cache Pressure visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/cache.go + starter: learner/starter.go interface: bytearena.cache.Cache/v1 build: command: ["go", "test", "./challenges/cache-pressure-medium/..."] diff --git a/challenges/cache-pressure-medium/harness/harness.go b/challenges/cache-pressure-medium/harness/harness.go index c12a61f..04e838d 100644 --- a/challenges/cache-pressure-medium/harness/harness.go +++ b/challenges/cache-pressure-medium/harness/harness.go @@ -11,7 +11,7 @@ import ( const ( ArenaID = "cache-pressure-medium" - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" WorkloadID = "public-smoke-v1" ) @@ -56,6 +56,8 @@ type PublicConfig struct { Operations int HotKeys uint64 ScanKeys uint64 + // Zero disables the performance floor for small contract-focused tests. + MinRequestHitRateBasisPoints uint64 } // DefaultPublicConfig returns the fast public smoke workload. The scan is @@ -74,10 +76,11 @@ func DefaultPublicConfig() PublicConfig { scanKeys = 1 } return PublicConfig{ - CapacityBytes: capacity, - Operations: 4096, - HotKeys: 16, - ScanKeys: scanKeys, + CapacityBytes: capacity, + Operations: 4096, + HotKeys: 16, + ScanKeys: scanKeys, + MinRequestHitRateBasisPoints: ScanResistanceFloorBasisPoints, } } @@ -216,6 +219,9 @@ func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result if reads > 0 { result.Metrics.RequestHitRate = readHits * 10000 / reads } + if config.MinRequestHitRateBasisPoints > 0 && result.Metrics.RequestHitRate < config.MinRequestHitRateBasisPoints { + result.Violations = append(result.Violations, "request hit rate is below the public minimum") + } if peakUsed < config.CapacityBytes { result.Metrics.PeakMemoryHeadroom = config.CapacityBytes - peakUsed } diff --git a/challenges/cache-pressure-medium/harness/harness_test.go b/challenges/cache-pressure-medium/harness/harness_test.go index fd92573..024aa41 100644 --- a/challenges/cache-pressure-medium/harness/harness_test.go +++ b/challenges/cache-pressure-medium/harness/harness_test.go @@ -74,6 +74,10 @@ func TestScanResistanceWorkloadAndFloor(t *testing.T) { if noCacheRate != 0 { t.Fatalf("no-cache policy reported a non-zero hit rate: %d", noCacheRate) } + noCacheResult := Evaluate(^uint64(0), config, newNoCache) + if noCacheResult.Verdict != "fail" || !containsViolation(noCacheResult.Violations, "request hit rate is below the public minimum") { + t.Fatalf("never-admit cache escaped the public performance floor: %+v", noCacheResult) + } if starterRate < ScanResistanceFloorBasisPoints { t.Fatalf("starter request hit rate %d is below the published floor %d", starterRate, ScanResistanceFloorBasisPoints) } diff --git a/challenges/cache-pressure-medium/learner/starter.go b/challenges/cache-pressure-medium/learner/starter.go new file mode 100644 index 0000000..ef16b32 --- /dev/null +++ b/challenges/cache-pressure-medium/learner/starter.go @@ -0,0 +1,29 @@ +// Package starter is the learner-facing scaffold for the TTL cache. +package starter + +import "github.com/roughstack/challenges/challenges/cache-pressure-medium/contract" + +type cache struct { + capacity uint64 + // TODO: track values, expiry ticks, recency, and byte accounting. +} + +func NewCache(capacityBytes uint64) contract.Cache { + return &cache{capacity: capacityBytes} +} + +func (c *cache) Get(key uint64, meta contract.RequestMeta) ([]byte, bool) { + panic("TODO: expire at the exact tick and update recency") +} + +func (c *cache) Put(key uint64, value []byte, meta contract.RequestMeta) { + panic("TODO: handle expiry, replacement, admission, and eviction") +} + +func (c *cache) Delete(key uint64) { + panic("TODO: remove an entry without leaking its byte charge") +} + +func (c *cache) UsedBytes() uint64 { + panic("TODO: report current accounted bytes") +} diff --git a/challenges/cache-pressure-medium/statement.json b/challenges/cache-pressure-medium/statement.json index b5336f4..19b6df7 100644 --- a/challenges/cache-pressure-medium/statement.json +++ b/challenges/cache-pressure-medium/statement.json @@ -9,16 +9,14 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.snapshot", "props": { - "label": "Scan pollution", - "nodes": [ - { "label": "Hot set", "detail": "Repeated, valuable hits", "tone": "signal" }, - { "label": "Cold scan", "detail": "One cache-sized pass", "tone": "warning" }, - { "label": "Single LRU", "detail": "Cold keys become newest", "tone": "danger" }, - { "label": "Hot set again", "detail": "Useful keys now miss", "tone": "warning" } - ], - "edges": ["warm", "displaces", "pays twice"] + "title": "What a one-time scan does to a small cache", + "rows": [ + { "label": "Warm", "cells": [{ "label": "H1", "tone": "signal" }, { "label": "H2", "tone": "signal" }, { "label": "H3", "tone": "signal" }], "note": "The hot set is reused often." }, + { "label": "Cold scan", "cells": [{ "label": "S1", "tone": "warning" }, { "label": "S2", "tone": "warning" }, { "label": "S3", "tone": "warning" }], "note": "Pure LRU fills the budget with one-hit keys." }, + { "label": "Return", "cells": [{ "label": "H1 miss", "tone": "danger" }, { "label": "H2 miss", "tone": "danger" }, { "label": "H3 miss", "tone": "danger" }], "note": "A protected hot segment would keep these requests cheap." } + ] } }, "caption": "Pure LRU mistakes recent for reusable. A one-time scan looks important long enough to evict the data that will actually return." @@ -57,7 +55,8 @@ "Charge `len(value) + contract.PerEntryMetadataBytes` for every resident entry.", "Reject oversized inserts and replacements without evicting existing data.", "`TTL = 0` never expires; `Get` never extends expiry.", - "Copy caller buffers and remove expired entries from both lookup and accounting." + "Copy caller buffers and remove expired entries from both lookup and accounting.", + "The public workload requires a request hit rate of at least 40%; a cache that admits nothing does not pass." ] }, { diff --git a/challenges/chunk-store-easy/arena.yaml b/challenges/chunk-store-easy/arena.yaml index 18c9fe9..ab10ba5 100644 --- a/challenges/chunk-store-easy/arena.yaml +++ b/challenges/chunk-store-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: chunk-store-easy - version: 1.0.0 + version: 1.0.1 title: Chunk Store visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.chunkstore.Store/v1 build: command: ["go", "test", "./challenges/chunk-store-easy/..."] diff --git a/challenges/chunk-store-easy/harness/harness.go b/challenges/chunk-store-easy/harness/harness.go index fabf264..b2c8197 100644 --- a/challenges/chunk-store-easy/harness/harness.go +++ b/challenges/chunk-store-easy/harness/harness.go @@ -20,7 +20,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "chunk-store-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" diff --git a/challenges/chunk-store-easy/learner/starter.go b/challenges/chunk-store-easy/learner/starter.go new file mode 100644 index 0000000..fc14cf8 --- /dev/null +++ b/challenges/chunk-store-easy/learner/starter.go @@ -0,0 +1,42 @@ +// Package starter is the learner-facing scaffold for a framed blob store. +package starter + +import ( + "io" + + "github.com/roughstack/challenges/challenges/chunk-store-easy/contract" +) + +type store struct { + device contract.Device + memoryLimit uint64 + // TODO: keep only a bounded in-memory index. +} + +func OpenStore(_ string, device contract.Device, memoryLimit uint64) (contract.Store, error) { + return &store{device: device, memoryLimit: memoryLimit}, nil +} + +func (s *store) Put(id, version uint64, data io.Reader, size uint64) error { + panic("TODO: encode and commit a verifiable frame") +} + +func (s *store) ReadAt(id, version uint64, p []byte, off int64) (int, error) { + panic("TODO: find, validate, and slice the requested blob") +} + +func (s *store) Delete(id, version uint64) error { + panic("TODO: mark one version deleted") +} + +func (s *store) Compact() error { + panic("TODO: reclaim space without changing visible bytes") +} + +func (s *store) Close() error { + panic("TODO: release the store") +} + +func (s *store) IndexBytes() uint64 { + panic("TODO: report bounded index memory") +} diff --git a/challenges/chunk-store-easy/statement.json b/challenges/chunk-store-easy/statement.json index b0f9b65..fafc1f9 100644 --- a/challenges/chunk-store-easy/statement.json +++ b/challenges/chunk-store-easy/statement.json @@ -9,17 +9,14 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.frames", "props": { - "label": "Blob ingest pipeline", - "nodes": [ - { "label": "Blob", "detail": "Immutable bytes" }, - { "label": "Chunk", "detail": "Split at fixed block size", "tone": "accent" }, - { "label": "Encode", "detail": "Compress only when smaller", "tone": "warning" }, - { "label": "Frame", "detail": "Lengths + CRC-32", "tone": "signal" }, - { "label": "Device", "detail": "Append block bytes" } - ], - "edges": ["split", "per block", "protect", "write"] + "title": "One blob · independently readable frames", + "note": "ReadAt uses the index to select a frame", + "frames": [ + { "label": "Block 0", "status": "not read", "tone": "neutral", "fields": [{ "label": "header", "detail": "lengths", "tone": "neutral" }, { "label": "AAAA", "detail": "payload", "tone": "neutral" }, { "label": "CRC", "detail": "integrity", "tone": "neutral" }] }, + { "label": "Block 1", "status": "read + verified", "tone": "signal", "fields": [{ "label": "header", "detail": "lengths", "tone": "accent" }, { "label": "BCDE", "detail": "requested CD", "tone": "signal" }, { "label": "CRC", "detail": "checked", "tone": "warning" }], "note": "Offset 5, length 2 touches only this block in the example below." } + ] } }, "caption": "Compression and integrity are block-local. That locality is what makes exact random reads cheap and bounded." diff --git a/challenges/deadline-scheduler-easy/arena.yaml b/challenges/deadline-scheduler-easy/arena.yaml index d5d2be5..03a9223 100644 --- a/challenges/deadline-scheduler-easy/arena.yaml +++ b/challenges/deadline-scheduler-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: deadline-scheduler-easy - version: 1.0.0 + version: 1.0.1 title: Deadline Scheduler visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.deadlinescheduler.Scheduler/v1 build: command: ["go", "test", "./challenges/deadline-scheduler-easy/..."] diff --git a/challenges/deadline-scheduler-easy/harness/harness.go b/challenges/deadline-scheduler-easy/harness/harness.go index 6889e06..8e480c7 100644 --- a/challenges/deadline-scheduler-easy/harness/harness.go +++ b/challenges/deadline-scheduler-easy/harness/harness.go @@ -19,7 +19,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "deadline-scheduler-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" // FixtureWorkloadID identifies explicit job fixtures used by public tests. diff --git a/challenges/deadline-scheduler-easy/learner/starter.go b/challenges/deadline-scheduler-easy/learner/starter.go new file mode 100644 index 0000000..b95c019 --- /dev/null +++ b/challenges/deadline-scheduler-easy/learner/starter.go @@ -0,0 +1,28 @@ +// Package starter is the learner-facing scaffold for a one-worker scheduler. +package starter + +import "github.com/roughstack/challenges/challenges/deadline-scheduler-easy/contract" + +type scheduler struct { + // TODO: track waiting and running jobs. +} + +func NewScheduler(cluster contract.Cluster) contract.Scheduler { + return &scheduler{} +} + +func (s *scheduler) OnArrival(job contract.Job, nowTick uint64) []contract.Action { + panic("TODO: record the arrival and dispatch only when idle") +} + +func (s *scheduler) OnProgress(progress contract.Progress, nowTick uint64) []contract.Action { + panic("TODO: reflect the worker's progress") +} + +func (s *scheduler) OnComplete(completion contract.Completion, nowTick uint64) []contract.Action { + panic("TODO: complete the job and decide what runs next") +} + +func (s *scheduler) StateBytes() uint64 { + panic("TODO: account for scheduler state") +} diff --git a/challenges/deadline-scheduler-easy/statement.json b/challenges/deadline-scheduler-easy/statement.json index ad8ff78..6ab5172 100644 --- a/challenges/deadline-scheduler-easy/statement.json +++ b/challenges/deadline-scheduler-easy/statement.json @@ -9,15 +9,17 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.timeline", + "id": "systems.ranges", "props": { - "label": "Non-preemptive worker schedule", - "windows": [ - { "label": "Job A", "range": "[0, 6)", "events": [{ "label": "6 ticks", "tone": "warning" }], "finalized": true }, - { "label": "Job B", "range": "[6, 8)", "events": [{ "label": "2 ticks", "tone": "signal" }], "finalized": true }, - { "label": "Job C", "range": "[8, 11)", "events": [{ "label": "3 ticks", "tone": "accent" }] } + "title": "One worker · three non-preemptive jobs", + "min": 0, + "max": 11, + "rows": [ + { "label": "A", "start": 0, "end": 6, "tone": "warning", "note": "Starts at 0; cannot be interrupted when B and C arrive." }, + { "label": "B", "start": 6, "end": 8, "tone": "signal", "note": "Waits until the worker becomes idle." }, + { "label": "C", "start": 8, "end": 11, "tone": "accent", "note": "Ordering is chosen only at an idle boundary." } ], - "watermark": "virtual clock" + "markers": [{ "at": 1, "label": "C arrives", "tone": "danger" }, { "at": 6, "label": "A done", "tone": "signal" }] } }, "caption": "Once A starts it owns the worker until tick 6. Better ordering happens only at idle boundaries." @@ -61,7 +63,7 @@ "kind": "callout", "tone": "note", "title": "Read the metrics together", - "md": "Mean flow rewards short-job ordering; p95 flow and starvation penalty expose policies that sacrifice old jobs. Peak state and scheduler work show the bookkeeping cost." + "md": "Mean flow rewards short-job ordering; p95 flow and starvation penalty expose policies that sacrifice old jobs. Peak state tracks reported memory. `scheduler_work` counts callback and action traffic, not the CPU time of your scheduling algorithm." } ] } diff --git a/challenges/durable-queue-easy/arena.yaml b/challenges/durable-queue-easy/arena.yaml index 2abfdc1..6aafdc1 100644 --- a/challenges/durable-queue-easy/arena.yaml +++ b/challenges/durable-queue-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: durable-queue-easy - version: 1.0.0 + version: 1.0.1 title: Durable Work Queue visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/queue.go + starter: learner/starter.go interface: bytearena.durablequeue.Queue/v1 build: command: ["go", "test", "./challenges/durable-queue-easy/..."] diff --git a/challenges/durable-queue-easy/harness/harness.go b/challenges/durable-queue-easy/harness/harness.go index b1e597c..8a3eda6 100644 --- a/challenges/durable-queue-easy/harness/harness.go +++ b/challenges/durable-queue-easy/harness/harness.go @@ -20,7 +20,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "durable-queue-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" ) diff --git a/challenges/durable-queue-easy/learner/starter.go b/challenges/durable-queue-easy/learner/starter.go new file mode 100644 index 0000000..0fe96d4 --- /dev/null +++ b/challenges/durable-queue-easy/learner/starter.go @@ -0,0 +1,37 @@ +// Package starter is the learner-facing scaffold for a lease-based queue. +package starter + +import ( + "context" + + "github.com/roughstack/challenges/challenges/durable-queue-easy/contract" +) + +type queue struct { + capacity uint64 + // TODO: protect queued, leased, and closed state under concurrency. +} + +func OpenQueue(_ string, capacityBytes uint64) (contract.Queue, error) { + return &queue{capacity: capacityBytes}, nil +} + +func (q *queue) Enqueue(ctx context.Context, job contract.Job) error { + panic("TODO: copy the payload and apply capacity backpressure") +} + +func (q *queue) Lease(ctx context.Context, nowTick, visibility uint64) (contract.Lease, error) { + panic("TODO: issue a lease or wait for work") +} + +func (q *queue) Ack(ctx context.Context, token uint64) error { + panic("TODO: permanently remove the leased job") +} + +func (q *queue) Nack(ctx context.Context, token uint64) error { + panic("TODO: make the job visible again") +} + +func (q *queue) Close() error { + panic("TODO: wake waiters and close idempotently") +} diff --git a/challenges/durable-queue-easy/statement.json b/challenges/durable-queue-easy/statement.json index bb03e80..a9674ac 100644 --- a/challenges/durable-queue-easy/statement.json +++ b/challenges/durable-queue-easy/statement.json @@ -9,15 +9,14 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.state", + "id": "systems.snapshot", "props": { - "label": "Queue job lifecycle", - "states": [ - { "label": "Available", "detail": "FIFO among ready jobs", "tone": "accent" }, - { "label": "Leased", "detail": "Unique token + deadline", "tone": "warning" }, - { "label": "Acknowledged", "detail": "Payload bytes released", "tone": "signal" } - ], - "transitions": ["Lease", "Ack"] + "title": "FIFO is defined by when a job becomes available", + "rows": [ + { "label": "Enqueue", "cells": [{ "label": "A", "detail": "ready", "tone": "accent" }, { "label": "B", "detail": "ready", "tone": "accent" }] }, + { "label": "Lease A", "cells": [{ "label": "B", "detail": "ready", "tone": "accent" }, { "label": "A", "detail": "leased until 10", "tone": "warning" }], "note": "A still occupies capacity while leased." }, + { "label": "Nack A", "cells": [{ "label": "B", "detail": "next", "tone": "signal" }, { "label": "A", "detail": "back of queue", "tone": "accent" }], "note": "A's old token is now stale." } + ] } }, "caption": "Nack or visibility expiry returns a leased job to the back of the available queue. It receives a fresh token when leased again." @@ -61,7 +60,7 @@ "kind": "callout", "tone": "note", "title": "Why virtual time matters", - "md": "Lease expiry is driven by the injected `nowTick`, never wall time. The same seed therefore produces the same redeliveries, waits, and score." + "md": "Lease expiry is driven by the injected `nowTick`, never wall time. The same seed therefore produces the same redeliveries and logical wait measurements. This practice challenge is not ranked yet." } ] } diff --git a/challenges/fair-gate-easy/arena.yaml b/challenges/fair-gate-easy/arena.yaml index 87e01bd..57ca7c8 100644 --- a/challenges/fair-gate-easy/arena.yaml +++ b/challenges/fair-gate-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: fair-gate-easy - version: 1.0.0 + version: 1.0.1 title: Fair Gate visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.fairgate.Limiter/v1 build: command: ["go", "test", "./challenges/fair-gate-easy/..."] diff --git a/challenges/fair-gate-easy/harness/harness.go b/challenges/fair-gate-easy/harness/harness.go index e8a340f..4b65031 100644 --- a/challenges/fair-gate-easy/harness/harness.go +++ b/challenges/fair-gate-easy/harness/harness.go @@ -17,7 +17,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "fair-gate-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" // FixtureWorkloadID identifies explicit request fixtures used by tests. diff --git a/challenges/fair-gate-easy/learner/starter.go b/challenges/fair-gate-easy/learner/starter.go new file mode 100644 index 0000000..c41859d --- /dev/null +++ b/challenges/fair-gate-easy/learner/starter.go @@ -0,0 +1,17 @@ +// Package starter is the learner-facing scaffold for a token-bucket gate. +package starter + +import "github.com/roughstack/challenges/challenges/fair-gate-easy/contract" + +type limiter struct { + cfg contract.Config + // TODO: track fractional tokens and the most recent tick. +} + +func NewLimiter(cfg contract.Config) contract.Limiter { + return &limiter{cfg: cfg} +} + +func (l *limiter) Decide(req contract.Request) contract.Decision { + panic("TODO: refill, charge once, and return the exact retry tick") +} diff --git a/challenges/fair-gate-easy/scoring.yaml b/challenges/fair-gate-easy/scoring.yaml index 7c500e2..e42706c 100644 --- a/challenges/fair-gate-easy/scoring.yaml +++ b/challenges/fair-gate-easy/scoring.yaml @@ -1,3 +1,7 @@ +version: 1 +score_range: [0, 10000] +correctness_gate: true +public_score: unofficial metrics: decisions_per_second: weight: 0.60 diff --git a/challenges/fair-gate-easy/statement.json b/challenges/fair-gate-easy/statement.json index 0f3ce16..849aeb7 100644 --- a/challenges/fair-gate-easy/statement.json +++ b/challenges/fair-gate-easy/statement.json @@ -9,16 +9,16 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.budget", "props": { - "label": "Token bucket decision", - "nodes": [ - { "label": "Elapsed ticks", "detail": "Compute lazy refill" }, - { "label": "Bucket", "detail": "Clamp tokens to Burst", "tone": "accent" }, - { "label": "Request cost", "detail": "Compare whole tokens", "tone": "warning" }, - { "label": "Decision", "detail": "Allow or exact RetryAt", "tone": "signal" } - ], - "edges": ["refill", "then", "emit"] + "title": "A burst of four tokens · refill two per three ticks", + "capacity": 4, + "snapshots": [ + { "label": "tick 0 · full", "segments": [{ "label": "available", "value": 4, "tone": "signal" }] }, + { "label": "tick 0 · after cost 3", "segments": [{ "label": "available", "value": 1, "tone": "warning" }], "note": "a second cost-3 request is rejected" }, + { "label": "tick 3 · before retry", "segments": [{ "label": "available", "value": 3, "tone": "signal" }], "note": "two tokens accrued lazily" }, + { "label": "tick 3 · after retry", "segments": [{ "label": "available", "value": 0, "tone": "accent" }], "note": "the retry is now admitted" } + ] } }, "caption": "No timer updates the bucket in the background. Each request first accounts for elapsed virtual time, then makes one deterministic decision." @@ -62,7 +62,7 @@ "kind": "callout", "tone": "note", "title": "Optimization target", - "md": "Once decisions and `RetryAt` are exact, reduce repeated arithmetic and allocation. Throughput rewards the fast path; `arithmetic_work` makes avoidable recomputation visible." + "md": "Once decisions and `RetryAt` are exact, allocations may distinguish implementations. The public `decisions_per_second` and `arithmetic_work` counters depend on the fixed trace, not on how fast your code runs or how many arithmetic operations it performs." } ] } diff --git a/challenges/journal-recovery-easy/arena.yaml b/challenges/journal-recovery-easy/arena.yaml index 842cb72..0437e8a 100644 --- a/challenges/journal-recovery-easy/arena.yaml +++ b/challenges/journal-recovery-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: journal-recovery-easy - version: 1.0.0 + version: 1.0.1 title: Journal Recovery visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/journal.go + starter: learner/starter.go interface: bytearena.journal.Journal/v1 build: command: ["go", "test", "./challenges/journal-recovery-easy/..."] diff --git a/challenges/journal-recovery-easy/harness/harness.go b/challenges/journal-recovery-easy/harness/harness.go index cbfcf94..801c3e8 100644 --- a/challenges/journal-recovery-easy/harness/harness.go +++ b/challenges/journal-recovery-easy/harness/harness.go @@ -19,7 +19,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "journal-recovery-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" ) diff --git a/challenges/journal-recovery-easy/learner/starter.go b/challenges/journal-recovery-easy/learner/starter.go new file mode 100644 index 0000000..2d18028 --- /dev/null +++ b/challenges/journal-recovery-easy/learner/starter.go @@ -0,0 +1,29 @@ +// Package starter is the learner-facing scaffold for a crash-safe journal. +package starter + +import "github.com/roughstack/challenges/challenges/journal-recovery-easy/contract" + +type journal struct { + device contract.Device + // TODO: track the next LSN, durable prefix, and closed state. +} + +func OpenJournal(_ string, device contract.Device) (contract.Journal, error) { + return &journal{device: device}, nil +} + +func (j *journal) Append(record contract.Record) (uint64, error) { + panic("TODO: encode a bounded, checksummed frame") +} + +func (j *journal) Sync(upto uint64) error { + panic("TODO: persist a known prefix") +} + +func (j *journal) Recover(apply func(contract.Record) error) (contract.RecoveryInfo, error) { + panic("TODO: replay the valid durable prefix") +} + +func (j *journal) Close() error { + panic("TODO: close idempotently") +} diff --git a/challenges/journal-recovery-easy/statement.json b/challenges/journal-recovery-easy/statement.json index 2ccd9a2..d85a2cf 100644 --- a/challenges/journal-recovery-easy/statement.json +++ b/challenges/journal-recovery-easy/statement.json @@ -9,16 +9,15 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.frames", "props": { - "label": "Journal append and durability", - "nodes": [ - { "label": "Record", "detail": "Copy caller payload" }, - { "label": "Frame", "detail": "Magic · length · LSN · CRC", "tone": "accent" }, - { "label": "Write", "detail": "Append bytes to Device", "tone": "warning" }, - { "label": "Sync", "detail": "Advance durability boundary", "tone": "signal" } - ], - "edges": ["encode", "append", "make durable"] + "title": "The durable prefix ends after LSN 2", + "note": "recovery scans left to right", + "frames": [ + { "label": "LSN 1", "status": "synced", "tone": "signal", "fields": [{ "label": "magic", "tone": "accent" }, { "label": "len + seq", "tone": "accent" }, { "label": "payload", "tone": "signal" }, { "label": "CRC", "tone": "warning" }] }, + { "label": "LSN 2", "status": "synced", "tone": "signal", "fields": [{ "label": "magic", "tone": "accent" }, { "label": "len + seq", "tone": "accent" }, { "label": "payload", "tone": "signal" }, { "label": "CRC", "tone": "warning" }] }, + { "label": "LSN 3", "status": "torn tail", "tone": "danger", "fields": [{ "label": "magic", "tone": "accent" }, { "label": "len + seq", "tone": "accent" }, { "label": "partial", "detail": "payload", "tone": "danger" }, { "label": "missing", "detail": "bytes + CRC", "tone": "neutral" }], "note": "A declared frame that ends early is not applied." } + ] } }, "caption": "Append gives a record an LSN, but only Sync makes the requested prefix durable. Recovery trusts structure and checksum, not intent." diff --git a/challenges/lossy-link-easy/arena.yaml b/challenges/lossy-link-easy/arena.yaml index aa7d0d1..78cdc70 100644 --- a/challenges/lossy-link-easy/arena.yaml +++ b/challenges/lossy-link-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: lossy-link-easy - version: 1.0.0 + version: 1.0.1 title: Lossy Link visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.lossylink.Endpoint/v1 build: command: ["go", "test", "./challenges/lossy-link-easy/..."] diff --git a/challenges/lossy-link-easy/harness/harness.go b/challenges/lossy-link-easy/harness/harness.go index cb24c76..1634f6e 100644 --- a/challenges/lossy-link-easy/harness/harness.go +++ b/challenges/lossy-link-easy/harness/harness.go @@ -14,7 +14,7 @@ import ( const ( ArenaID = "lossy-link-easy" - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" WorkloadID = "public-smoke-v1" // maxTicks bounds the virtual event loop. Reaching it without completing diff --git a/challenges/lossy-link-easy/learner/starter.go b/challenges/lossy-link-easy/learner/starter.go new file mode 100644 index 0000000..a2d059b --- /dev/null +++ b/challenges/lossy-link-easy/learner/starter.go @@ -0,0 +1,47 @@ +// Package starter is the learner-facing scaffold for a reliable byte stream. +package starter + +import "github.com/roughstack/challenges/challenges/lossy-link-easy/contract" + +type sender struct { + cfg contract.Config + // TODO: track bytes in flight, acknowledgements, and timers. +} + +type receiver struct { + cfg contract.Config + // TODO: track the next expected sequence and delivered bytes. +} + +func NewSender(cfg contract.Config) contract.Endpoint { return &sender{cfg: cfg} } +func NewReceiver(cfg contract.Config) contract.Endpoint { return &receiver{cfg: cfg} } + +func (s *sender) OnAppData(data []byte, eof bool) []contract.Packet { + panic("TODO: packetize and send") +} +func (s *sender) OnPacket(pkt contract.Packet, nowTick uint64) []contract.Packet { + panic("TODO: process valid acknowledgements") +} +func (s *sender) OnTimer(timerID, nowTick uint64) []contract.Packet { + panic("TODO: retransmit the outstanding packet") +} +func (s *sender) ReadDelivered(max int) []byte { panic("TODO: sender delivers no application bytes") } +func (s *sender) NextTimers() []contract.Timer { panic("TODO: expose active timers") } +func (s *sender) StateBytes() uint64 { panic("TODO: report buffered state") } +func (s *sender) Done() bool { panic("TODO: report completion only after acknowledgement") } + +func (r *receiver) OnAppData(data []byte, eof bool) []contract.Packet { + panic("TODO: receiver accepts no outgoing app data") +} +func (r *receiver) OnPacket(pkt contract.Packet, nowTick uint64) []contract.Packet { + panic("TODO: validate, deliver in order, and acknowledge") +} +func (r *receiver) OnTimer(timerID, nowTick uint64) []contract.Packet { + panic("TODO: receiver timer behavior") +} +func (r *receiver) ReadDelivered(max int) []byte { + panic("TODO: drain only the requested delivered bytes") +} +func (r *receiver) NextTimers() []contract.Timer { panic("TODO: receiver timer behavior") } +func (r *receiver) StateBytes() uint64 { panic("TODO: report buffered state") } +func (r *receiver) Done() bool { panic("TODO: report end-of-stream delivery") } diff --git a/challenges/lossy-link-easy/statement.json b/challenges/lossy-link-easy/statement.json index 65869d6..5c7a648 100644 --- a/challenges/lossy-link-easy/statement.json +++ b/challenges/lossy-link-easy/statement.json @@ -3,21 +3,24 @@ "blocks": [ { "kind": "prose", - "md": "Build reliability above an unreliable link. With only one unacknowledged packet at a time, the protocol needs a sequence bit, a checksum, an acknowledgement, and a retransmission timer." + "md": "Build reliability above an unreliable link. With only one unacknowledged packet at a time, the protocol needs a small wrapping sequence number, a checksum, an acknowledgement, and a retransmission timer." }, { "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.state", + "id": "systems.exchange", "props": { - "label": "Stop-and-wait sender", - "states": [ - { "label": "Ready", "detail": "Accept the next app chunk", "tone": "signal" }, - { "label": "Waiting", "detail": "One packet is in flight", "tone": "warning" }, - { "label": "Advance", "detail": "Matching ACK flips sequence", "tone": "accent" } - ], - "transitions": ["send DATA", "receive ACK"] + "title": "A lost ACK is why duplicates must be safe", + "left": "Sender", + "right": "Receiver", + "events": [ + { "from": "left", "label": "DATA 0 · abc", "detail": "deliver abc", "tone": "signal" }, + { "from": "right", "label": "ACK 0", "detail": "waiting", "dropped": true, "tone": "danger" }, + { "from": "local", "label": "timer fires", "tone": "warning" }, + { "from": "left", "label": "DATA 0 · retry", "detail": "duplicate, no delivery", "tone": "warning" }, + { "from": "right", "label": "ACK 0 again", "detail": "advance to 1", "tone": "accent" } + ] } }, "caption": "Loss or corruption leaves the sender waiting. When its timer fires, it retransmits the same packet; the receiver discards duplicates but repeats the ACK." @@ -60,7 +63,7 @@ { "kind": "callout", "tone": "note", - "title": "Read the score as overhead", + "title": "Read the metrics as overhead", "md": "Completion ticks measure waiting; transmitted bytes, packet count, and retransmissions measure how much extra work reliability required; state bytes price the protocol memory." } ] diff --git a/challenges/lsm-compactor-easy/arena.yaml b/challenges/lsm-compactor-easy/arena.yaml index 11adc47..f492d58 100644 --- a/challenges/lsm-compactor-easy/arena.yaml +++ b/challenges/lsm-compactor-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: lsm-compactor-easy - version: 1.0.0 + version: 1.0.1 title: LSM Compactor visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.lsmcompactor.Policy/v1 build: command: ["go", "test", "./challenges/lsm-compactor-easy/..."] diff --git a/challenges/lsm-compactor-easy/harness/harness.go b/challenges/lsm-compactor-easy/harness/harness.go index c83e359..64f0a4b 100644 --- a/challenges/lsm-compactor-easy/harness/harness.go +++ b/challenges/lsm-compactor-easy/harness/harness.go @@ -16,7 +16,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "lsm-compactor-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" diff --git a/challenges/lsm-compactor-easy/learner/starter.go b/challenges/lsm-compactor-easy/learner/starter.go new file mode 100644 index 0000000..c885b14 --- /dev/null +++ b/challenges/lsm-compactor-easy/learner/starter.go @@ -0,0 +1,16 @@ +// Package starter is the learner-facing scaffold for an LSM merge policy. +package starter + +import "github.com/roughstack/challenges/challenges/lsm-compactor-easy/contract" + +type policy struct { + cfg contract.Config +} + +func NewPolicy(cfg contract.Config) contract.Policy { + return &policy{cfg: cfg} +} + +func (p *policy) Pick(state contract.State) (contract.Plan, bool) { + panic("TODO: choose a legal overlap-closed compaction plan") +} diff --git a/challenges/lsm-compactor-easy/statement.json b/challenges/lsm-compactor-easy/statement.json index e50815a..a1cd3a7 100644 --- a/challenges/lsm-compactor-easy/statement.json +++ b/challenges/lsm-compactor-easy/statement.json @@ -9,19 +9,21 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.ranges", "props": { - "label": "Two-level LSM lifecycle", - "nodes": [ - { "label": "Memtable", "detail": "Mutable recent entries", "tone": "signal" }, - { "label": "Level 0", "detail": "New immutable, overlapping runs", "tone": "warning" }, - { "label": "Compaction", "detail": "Merge newest value per key", "tone": "accent" }, - { "label": "Level 1", "detail": "Sorted non-overlapping runs" } + "title": "Selecting A + B closes over both Level-1 runs", + "min": 0, + "max": 60, + "rows": [ + { "label": "L0 A", "start": 10, "end": 30, "tone": "warning", "note": "Selected run" }, + { "label": "L0 B", "start": 25, "end": 50, "tone": "warning", "note": "Selected run, overlaps A" }, + { "label": "L1 X", "start": 0, "end": 20, "tone": "accent", "note": "Must join: overlaps A" }, + { "label": "L1 Y", "start": 40, "end": 60, "tone": "accent", "note": "Must join: overlaps B" } ], - "edges": ["flush", "pick inputs", "rewrite"] + "markers": [{ "at": 10, "label": "union starts", "tone": "signal" }, { "at": 50, "label": "union ends", "tone": "signal" }] } }, - "caption": "Waiting creates more Level-0 overlap and read work. Compacting too eagerly rewrites data repeatedly and stalls the write path." + "caption": "The selected Level-0 range spans keys 10 to 50. Both Level-1 runs intersect it, so a legal merge must include X and Y." }, { "kind": "callout", diff --git a/challenges/stream-windows-easy/arena.yaml b/challenges/stream-windows-easy/arena.yaml index 7b2f99e..26d9b1c 100644 --- a/challenges/stream-windows-easy/arena.yaml +++ b/challenges/stream-windows-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: stream-windows-easy - version: 1.0.0 + version: 1.0.1 title: Stream Windows visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.streamwindows.Operator/v1 build: command: ["go", "test", "./challenges/stream-windows-easy/..."] diff --git a/challenges/stream-windows-easy/harness/harness.go b/challenges/stream-windows-easy/harness/harness.go index 0f8b4d5..6bce19b 100644 --- a/challenges/stream-windows-easy/harness/harness.go +++ b/challenges/stream-windows-easy/harness/harness.go @@ -18,7 +18,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "stream-windows-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" // ticksPerLogicalSecond converts the injected virtual tick into a logical diff --git a/challenges/stream-windows-easy/learner/starter.go b/challenges/stream-windows-easy/learner/starter.go new file mode 100644 index 0000000..41f162f --- /dev/null +++ b/challenges/stream-windows-easy/learner/starter.go @@ -0,0 +1,29 @@ +// Package starter is the learner-facing scaffold for event-time windows. +package starter + +import "github.com/roughstack/challenges/challenges/stream-windows-easy/contract" + +type operator struct { + cfg contract.Config + // TODO: maintain per-key, per-window aggregates and watermark state. +} + +func NewOperator(cfg contract.Config) contract.Operator { + return &operator{cfg: cfg} +} + +func (o *operator) OnEvent(event contract.Event) []contract.Output { + panic("TODO: assign this event to its half-open window") +} + +func (o *operator) OnWatermark(tick uint64) []contract.Output { + panic("TODO: finalize exactly the windows behind the watermark") +} + +func (o *operator) Close() []contract.Output { + panic("TODO: finalize every remaining window once") +} + +func (o *operator) StateBytes() uint64 { + panic("TODO: report live aggregate state") +} diff --git a/challenges/stream-windows-easy/statement.json b/challenges/stream-windows-easy/statement.json index e0310fa..e43c834 100644 --- a/challenges/stream-windows-easy/statement.json +++ b/challenges/stream-windows-easy/statement.json @@ -9,26 +9,16 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.timeline", + "id": "systems.ranges", "props": { - "label": "Fixed event-time windows", - "windows": [ - { - "label": "Window 0", - "range": "[0, 10)", - "events": [ - { "label": "api +3", "tone": "accent" }, - { "label": "api +5", "tone": "accent" } - ], - "finalized": true - }, - { - "label": "Window 1", - "range": "[10, 20)", - "events": [{ "label": "db +2", "tone": "warning" }] - } + "title": "Half-open windows on the event-time axis", + "min": 0, + "max": 20, + "rows": [ + { "label": "[0, 10)", "start": 0, "end": 10, "tone": "signal", "note": "api +3 and api +5 · emit count 2, sum 8" }, + { "label": "[10, 20)", "start": 10, "end": 20, "tone": "warning", "note": "db +2 · still live at watermark 10" } ], - "watermark": "10" + "markers": [{ "at": 10, "label": "watermark", "tone": "accent" }] } }, "caption": "At watermark 10, Window 0 is final: emit api(count=2, sum=8) and evict it. Window 1 remains live." @@ -72,7 +62,7 @@ "kind": "callout", "tone": "note", "title": "Why the metrics fit the model", - "md": "Throughput rewards a small hot path; peak state checks timely eviction; output delay checks prompt finalization; allocation bytes reveal per-event churn." + "md": "Peak state checks timely eviction; allocation bytes reveal per-event churn. In the public harness, event throughput follows the fixed virtual-time trace and output delay is fixed for any correct implementation that emits at the watermark. These counters provide context, not evidence that one correct operator is faster." } ] } diff --git a/docs/challenge-audit-2026-10-15.md b/docs/challenge-audit-2026-10-15.md new file mode 100644 index 0000000..4bb2837 --- /dev/null +++ b/docs/challenge-audit-2026-10-15.md @@ -0,0 +1,68 @@ +# Challenge audit, 2026-10-15 + +This is the current evidence, not a claim that finite tests prove every possible +implementation correct. The companion [quality matrix](challenge-quality.md) +lists the adversarial and boundary tests each challenge should ultimately own. + +## Evidence checked + +- All ten public Go baselines passed their smoke workloads at seeds `12345` and + `987654321`. The full Go suite passed with the race detector. +- All ten baselines were submitted through the web platform's production build + and completed with passing verdicts in an isolated local database. This caught + a platform result-schema mismatch: Journal Recovery and Lossy Link include + the optional `run.fault_profile` field, which the platform now accepts. +- The original SSD execution runtime's Go and conformance suites passed. +- All thirteen challenge pages rendered an accessible educational diagram in + the browser. The web unit suite, a phone-sized problem/editor interaction + check, and the existing 40-viewport layout audit passed. +- The easy cache now has a standalone contract probe that rejects an incorrect + LRU, accounting, aliasing, deletion, or oversized-replacement implementation. + The medium cache now enforces its advertised 40% request-hit floor on the + default public workload, so a never-admit cache fails. +- A production node needs both the published arena registry and access to the + matching challenge packages. Registry-only mode can display previously + materialized database content, but cannot refresh statements or execute a + package that is unavailable locally. +- Follow-up visual review found that the browser was prefilled with the passing + baseline. The manifests now declare a separate `submission.starter`, and + all ten packages provide incomplete learner scaffolds while retaining the + public baseline at `submission.entrypoint`. The scaffold path is validated + separately from the file overwritten during a run. +- Version `1.0.1` of every harness now matches its manifest after the scaffold + release. A real submission of the easy-cache public baseline at this version + passed through the platform execution API; submitting the untouched learner + scaffold failed its public tests as expected. The platform's arena validator + checks Go harness and manifest versions for future drift. Browser checks + confirmed that all ten practice editors display unfinished code, and the + expanded visual and responsive suites passed. + +## Challenge-by-challenge conclusion + +| Challenge | Does the problem have a coherent solution? | Public evidence and next gap | +| --- | --- | --- | +| Cache Pressure, easy | Yes. Exact LRU behavior is prescribed; implementations need not share a data structure. | Baseline and contract probe pass. Add a small independent model across randomized operation sequences and a candidate that lies about `UsedBytes` while retaining data. | +| Cache Pressure, medium | Yes. Many admission/eviction policies are legal; the public workload now requires at least 40% request hits. | Baseline passes the floor across the checked seeds; never-admit fails. Add a model-based TTL/accounting probe independent of the generated workload, especially replacement at an exact expiry tick. | +| Chunk Store | Yes. Bytes returned and frame-local integrity are definite even though internal indexing can differ. | Corrupt length, checksum, over-allocation, unrelated reads, and reclaim tests exist. Expand every-tail-truncation and cross-block read cases under a tiny independent frame oracle. | +| Deadline Scheduler | Yes. Many fair, non-preemptive schedules are valid; no single order is mandated. | Arrival, ties, aging, idle gaps, failure gating, and starter tests pass. Add exhaustive short-job schedule comparison to validate flow-time accounting and starvation bounds. | +| Durable Work Queue | Yes. The visible lease/ack/nack behavior is definite; lock and storage choices are not. | Correctness, panic, nil, and watchdog tests pass. Add a concurrent history checker for no loss or duplication and more exact visibility-deadline ties. | +| Fair Gate | Yes. Given a token bucket config and request trace, allow/deny and retry tick are mathematically determined. | Tests reject over-admission, over-charging, and wrong retry ticks. Add an independent rational-arithmetic oracle for randomly generated short traces and overflow boundaries. | +| Journal Recovery | Yes. Recovery must return the valid durable prefix, regardless of internal data structure. | Every byte-tail truncation of a short encoded log, plus LSN gaps/duplicates, callback errors, corruption, panic, and nil cases pass. Add more interior-versus-tail fault pairs and an independent prefix oracle. | +| Lossy Link | Yes. Delivered bytes are fixed; retransmission schedule and packet count can vary. | Loss, corruption, duplicate, stale ACK, timer, wrap, and boundary tests pass. Add a bounded fault-sequence enumerator to check safety and liveness against many combined faults. | +| LSM Compactor | Yes. Several legal plans exist; reads after merging must preserve newest-version and tombstone semantics. | Invalid overlap closure, stale/duplicate plans, tombstones, empty runs, and tie-breaking pass. Add an independent key-version map oracle over enumerated small run graphs. | +| Stream Windows | Yes. Half-open window assignment and finalized sums are unique for a legal event trace. | Boundary, regression, duplicate/premature output, negative sum, close, and max-tick tests pass. Add randomized per-key map oracle and repeated watermark/event interleavings. | +| Compaction, original SSD | Yes. A minimum move count exists, though more than one move sequence can attain it. | Runtime tests demonstrate a legal optimal solution and reject illegal moves. Retain cross-language submitted-source conformance as a release gate. | +| Victim Selection, original SSD | Yes. Any legal policy preserves data; the best online choice depends on future writes. | Runtime simulation and conformance pass. Add property tests for page-map conservation and migration accounting under varied seeds. | +| Wear Levelling, original SSD | Yes. Legal policies trade migration cost against erase-count spread; there is no single mandated policy. | Runtime simulation and conformance pass. Add adversarial hot/cold workloads and explicit Pareto trade-off checks. | + +## Publication limits + +All ten generated public smoke results currently report `score: 0`, including +correct baselines. Their weighted scoring files are design intent, not an active +ranking system. Some displayed measurements, notably fair-gate throughput and +arithmetic work and stream-window event throughput/output delay, are fixed by +the public trace once the answer is correct; they should be labeled as trace +context until a contestant-sensitive measurement replaces them. Generated +challenges currently ship Go only, whereas the original SSD ladder has the +multi-language runtime. Do not advertise the generated set as ranked or +multi-language until those separate gates are implemented and tested. diff --git a/docs/challenge-quality.md b/docs/challenge-quality.md new file mode 100644 index 0000000..8fc7178 --- /dev/null +++ b/docs/challenge-quality.md @@ -0,0 +1,61 @@ +# Challenge quality checklist + +Every challenge needs three independent forms of evidence: a valid implementation +passes, a deliberately invalid implementation fails for the right reason, and +the same seed produces the same logical result. Unit coverage alone cannot prove +these properties. The public baseline is a feasibility witness, not an optimum +or a substitute for adversarial testing. The learner-facing starter must be an +incomplete, compilable scaffold, never that passing baseline. + +## Shared acceptance checks + +1. The statement, interface, starter, harness, and result metrics describe the + same behavior. Every illustrated example must be hand-checkable. +2. Test empty inputs, one-item inputs, exact boundaries, maximum legal values, + overflow paths, aliasing, repeated calls, and invalid calls where the + contract defines their behavior. +3. Compare each small generated case with an independent, simple oracle. Vary + seeds and sizes; avoid reusing the contestant implementation as the oracle. +4. Run a known-correct baseline and at least one targeted broken implementation + through the public harness. Assert the verdict and the relevant violation. +5. Repeat a seed and compare logical metrics and output. Measure runtime-derived + allocation metrics separately; they are not deterministic logical counters. +6. Check that every advertised metric is affected by a contestant choice, or + label it as workload context with zero scoring weight. Verify that a passing + result has a meaningful score before enabling ranking. +7. Run the language entrypoint in the actual execution image. A local Go unit + test does not establish that the browser's submitted source builds or runs. + +## Per-challenge test design + +| Challenge | Independent oracle and adversarial cases | Boundary and failure cases | What the learner should see | +| --- | --- | --- | --- | +| Cache Pressure, easy | Tiny map plus recency list; verify exact LRU order after hits and replacements. A never-admit cache and an eviction policy that removes the newest key must fail the contract probe. | Zero and exact capacity, oversized replacement, delete then read, repeated write, input/output aliasing, exact byte accounting. | A two-slot cache showing a hit moving a key to the newest position, followed by eviction of the other key. | +| Cache Pressure, medium | Independent byte-budget and expiry model. Compare pure LRU with a scan-resistant policy on the same hot-scan-hot trace; both must be correct while their miss costs differ. | Entry sizes at the budget, oversized replacement, TTL at and one tick before expiry, zero TTL, maximum-tick saturation, delete, aliasing. | Hot keys before and after a cold scan, with capacity and expiry shown separately. | +| Chunk Store | Byte-for-byte block/frame oracle; verify only touched blocks are read. Mutate a checksum, header length, and payload independently. | Empty and one-byte blobs, exact block boundaries, cross-block reads, duplicate key, delete, close, memory limit, truncated frame, decompression bomb. | A blob split into separately framed blocks and a short read touching only one block. | +| Deadline Scheduler | Enumerate short job sets and simulate a single non-preemptive worker. Compare FIFO with short-job-first where both are legal; check flow-time and starvation calculations. | Same-tick arrivals, idle gaps, zero/large work, completed/unknown job IDs, double placement, action while busy, clock overflow, no-progress policy. | A worker lane with arrival markers, waiting time, and a non-preemptive running interval. | +| Durable Work Queue | Sequential queue model for available and leased jobs, then concurrent producer/consumer traces checked for no loss or duplication. | Full/zero capacity, cancellation before admission, Nack, exact visibility deadline, stale tokens, expiry tie order, Close with blocked waiters, payload aliasing. | Available and leased lanes, with the visibility deadline returning a job to the queue. | +| Fair Gate | Rational arithmetic oracle for the token count and earliest retry tick. A limiter that over-admits or reports a late retry must fail. | Empty/full bucket, zero rate or burst, zero/oversized cost, same-tick requests, fractional refill, large tick jump, overflow, non-monotonic clock. | A token bucket at three ticks showing refill, an admitted cost, and a rejected retry. | +| Journal Recovery | Independent frame encoder/decoder and durable-prefix model. Crash after each write or sync position; recover exactly the valid prefix. | Empty log, every possible tail truncation, garbage tail, interior corruption, bad CRC, bad length, LSN gap/duplicate, callback error, caller mutation. | Physical frames, a sync frontier, and the difference between a torn tail and interior corruption. | +| Lossy Link | Compare the delivered byte stream with the original, independent of packet event order. Inject one fault at each send/ACK/timer point. | Empty and boundary payloads, lost/corrupt/duplicate/delayed data and ACK, stale timer, wraparound, packet limit, checksum, liveness timeout. | Sender and receiver lanes showing a lost packet, timer, retransmission, duplicate suppression, and ACK. | +| LSM Compactor | Small key-version oracle including tombstones; enumerate legal overlap-closed plans and compare read results before/after each merge. | Equal/max keys, empty and tombstone-only runs, overlapping L1 closure, stale/duplicate/distant IDs, oversized plan, no-op and always-compact policies. | Overlapping L0 runs, the exact L1 overlap set, and a merged run with one newest version per key. | +| Stream Windows | Simple per-key map grouped by `floor(tick/windowSize)`; compare canonical outputs after each watermark. | Tick zero and maximum legal tick, boundary events, repeated/regressing watermarks, empty windows, negative sums, interleaved keys, Close, duplicates, eviction. | Two adjacent event-time windows and a watermark that finalizes only the first. | + +## Original SSD ladder + +The three original challenges are defined in `web-platform/prisma/seed/challenges.ts` +and evaluated by `execution-runtime`: + +| Challenge | Key proof and adversarial tests | Visual teaching goal | +| --- | --- | --- | +| Compaction | Prove the lower bound by counting live values outside the destination prefix. Verify legal moves preserve the multiset, reach the prefix, and score exactly at the bound; reject self moves, occupied destinations, missing moves, and duplicate data. Run the same solution through each language binding. | Show the prefix boundary, one legal move, and the resulting contiguous free suffix. | +| Victim Selection | Simulate seeded writes with an independent page map. Check valid block indices, migration accounting, no lost page, no stall, and correct write amplification. Compare at least greedy and a distinct legal policy; there is no unique best decision for unknown future traffic. | Show valid/dead/free pages and the migration cost of two possible victims. | +| Wear Levelling | Reuse victim-selection checks, then test hot-page thrash, capacity pressure, and interrupted migration with a fresh policy instance for each case. Measure both write amplification and worst-block wear. | Show a wear histogram beside the extra migration cost of avoiding the most-worn block. | + +## Publication gate + +Before a variant is advertised as ranked, require an independent reference or +calibration run, a versioned scoring rule, a nonzero score for a known-correct +solution, mutation tests for its correctness gates, and an end-to-end submitted +source run. Passing public smoke tests alone establish a working practice +exercise, not a competitive score.