From 36731563055dd7055d087184aafbbdcae97eab3f Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:33:12 +0530 Subject: [PATCH 01/22] docs(challenges): define quality and visual review matrix --- docs/challenge-quality.md | 60 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 docs/challenge-quality.md diff --git a/docs/challenge-quality.md b/docs/challenge-quality.md new file mode 100644 index 0000000..16d996f --- /dev/null +++ b/docs/challenge-quality.md @@ -0,0 +1,60 @@ +# Challenge quality checklist + +Every challenge needs three independent forms of evidence: a valid implementation +passes, a deliberately invalid implementation fails for the right reason, and +the same seed produces the same logical result. Unit coverage alone cannot prove +these properties. The public starter is a feasibility witness, not an optimum +or a substitute for adversarial testing. + +## Shared acceptance checks + +1. The statement, interface, starter, harness, and result metrics describe the + same behavior. Every illustrated example must be hand-checkable. +2. Test empty inputs, one-item inputs, exact boundaries, maximum legal values, + overflow paths, aliasing, repeated calls, and invalid calls where the + contract defines their behavior. +3. Compare each small generated case with an independent, simple oracle. Vary + seeds and sizes; avoid reusing the contestant implementation as the oracle. +4. Run a known-correct starter and at least one targeted broken implementation + through the public harness. Assert the verdict and the relevant violation. +5. Repeat a seed and compare logical metrics and output. Measure runtime-derived + allocation metrics separately; they are not deterministic logical counters. +6. Check that every advertised metric is affected by a contestant choice, or + label it as workload context with zero scoring weight. Verify that a passing + result has a meaningful score before enabling ranking. +7. Run the language entrypoint in the actual execution image. A local Go unit + test does not establish that the browser's submitted source builds or runs. + +## Per-challenge test design + +| Challenge | Independent oracle and adversarial cases | Boundary and failure cases | What the learner should see | +| --- | --- | --- | --- | +| Cache Pressure, easy | Tiny map plus recency list; verify exact LRU order after hits and replacements. A never-admit cache and an eviction policy that removes the newest key must fail the contract probe. | Zero and exact capacity, oversized replacement, delete then read, repeated write, input/output aliasing, exact byte accounting. | A two-slot cache showing a hit moving a key to the newest position, followed by eviction of the other key. | +| Cache Pressure, medium | Independent byte-budget and expiry model. Compare pure LRU with a scan-resistant policy on the same hot-scan-hot trace; both must be correct while their miss costs differ. | Entry sizes at the budget, oversized replacement, TTL at and one tick before expiry, zero TTL, maximum-tick saturation, delete, aliasing. | Hot keys before and after a cold scan, with capacity and expiry shown separately. | +| Chunk Store | Byte-for-byte block/frame oracle; verify only touched blocks are read. Mutate a checksum, header length, and payload independently. | Empty and one-byte blobs, exact block boundaries, cross-block reads, duplicate key, delete, close, memory limit, truncated frame, decompression bomb. | A blob split into separately framed blocks and a short read touching only one block. | +| Deadline Scheduler | Enumerate short job sets and simulate a single non-preemptive worker. Compare FIFO with short-job-first where both are legal; check flow-time and starvation calculations. | Same-tick arrivals, idle gaps, zero/large work, completed/unknown job IDs, double placement, action while busy, clock overflow, no-progress policy. | A worker lane with arrival markers, waiting time, and a non-preemptive running interval. | +| Durable Work Queue | Sequential queue model for available and leased jobs, then concurrent producer/consumer traces checked for no loss or duplication. | Full/zero capacity, cancellation before admission, Nack, exact visibility deadline, stale tokens, expiry tie order, Close with blocked waiters, payload aliasing. | Available and leased lanes, with the visibility deadline returning a job to the queue. | +| Fair Gate | Rational arithmetic oracle for the token count and earliest retry tick. A limiter that over-admits or reports a late retry must fail. | Empty/full bucket, zero rate or burst, zero/oversized cost, same-tick requests, fractional refill, large tick jump, overflow, non-monotonic clock. | A token bucket at three ticks showing refill, an admitted cost, and a rejected retry. | +| Journal Recovery | Independent frame encoder/decoder and durable-prefix model. Crash after each write or sync position; recover exactly the valid prefix. | Empty log, every possible tail truncation, garbage tail, interior corruption, bad CRC, bad length, LSN gap/duplicate, callback error, caller mutation. | Physical frames, a sync frontier, and the difference between a torn tail and interior corruption. | +| Lossy Link | Compare the delivered byte stream with the original, independent of packet event order. Inject one fault at each send/ACK/timer point. | Empty and boundary payloads, lost/corrupt/duplicate/delayed data and ACK, stale timer, wraparound, packet limit, checksum, liveness timeout. | Sender and receiver lanes showing a lost packet, timer, retransmission, duplicate suppression, and ACK. | +| LSM Compactor | Small key-version oracle including tombstones; enumerate legal overlap-closed plans and compare read results before/after each merge. | Equal/max keys, empty and tombstone-only runs, overlapping L1 closure, stale/duplicate/distant IDs, oversized plan, no-op and always-compact policies. | Overlapping L0 runs, the exact L1 overlap set, and a merged run with one newest version per key. | +| Stream Windows | Simple per-key map grouped by `floor(tick/windowSize)`; compare canonical outputs after each watermark. | Tick zero and maximum legal tick, boundary events, repeated/regressing watermarks, empty windows, negative sums, interleaved keys, Close, duplicates, eviction. | Two adjacent event-time windows and a watermark that finalizes only the first. | + +## Original SSD ladder + +The three original challenges are defined in `web-platform/prisma/seed/challenges.ts` +and evaluated by `execution-runtime`: + +| Challenge | Key proof and adversarial tests | Visual teaching goal | +| --- | --- | --- | +| Compaction | Prove the lower bound by counting live values outside the destination prefix. Verify legal moves preserve the multiset, reach the prefix, and score exactly at the bound; reject self moves, occupied destinations, missing moves, and duplicate data. Run the same solution through each language binding. | Show the prefix boundary, one legal move, and the resulting contiguous free suffix. | +| Victim Selection | Simulate seeded writes with an independent page map. Check valid block indices, migration accounting, no lost page, no stall, and correct write amplification. Compare at least greedy and a distinct legal policy; there is no unique best decision for unknown future traffic. | Show valid/dead/free pages and the migration cost of two possible victims. | +| Wear Levelling | Reuse victim-selection checks, then test hot-page thrash, capacity pressure, and interrupted migration with a fresh policy instance for each case. Measure both write amplification and worst-block wear. | Show a wear histogram beside the extra migration cost of avoiding the most-worn block. | + +## Publication gate + +Before a variant is advertised as ranked, require an independent reference or +calibration run, a versioned scoring rule, a nonzero score for a known-correct +solution, mutation tests for its correctness gates, and an end-to-end submitted +source run. Passing public smoke tests alone establish a working practice +exercise, not a competitive score. From bda5b1d631944db6ed85f3d1ea17d59c757cd992 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:34:06 +0530 Subject: [PATCH 02/22] fix(cache): enforce exact LRU contract in public harness --- .../cache-pressure-easy/harness/harness.go | 82 ++++++++++++++++++- .../harness/harness_test.go | 37 +++++++++ 2 files changed, 117 insertions(+), 2 deletions(-) diff --git a/challenges/cache-pressure-easy/harness/harness.go b/challenges/cache-pressure-easy/harness/harness.go index 411900a..539d93e 100644 --- a/challenges/cache-pressure-easy/harness/harness.go +++ b/challenges/cache-pressure-easy/harness/harness.go @@ -3,6 +3,7 @@ package harness import ( "bytes" + "fmt" "strconv" "github.com/roughstack/challenges/challenges/cache-pressure-easy/contract" @@ -62,8 +63,8 @@ func DefaultPublicConfig() PublicConfig { } // Evaluate runs a policy-independent request stream against a contestant cache. -func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result { - result := Result{ +func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) (result Result) { + result = Result{ ProtocolVersion: 1, ArenaID: ArenaID, ArenaVersion: ArenaVersion, @@ -75,8 +76,29 @@ func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result WorkloadID: WorkloadID, }, } + defer func() { + if recovered := recover(); recovered != nil { + result.Verdict = "fail" + result.Violations = append(result.Violations, fmt.Sprintf("contestant panicked: %v", recovered)) + } + }() + if factory == nil { + result.Verdict = "fail" + result.Violations = append(result.Violations, "cache factory is nil") + return result + } + if violation := validateCacheContract(seed, factory); violation != "" { + result.Verdict = "fail" + result.Violations = append(result.Violations, violation) + return result + } cache := factory(config.CapacityBytes) + if cache == nil { + result.Verdict = "fail" + result.Violations = append(result.Violations, "cache factory returned nil") + return result + } versions := make(map[uint64]uint64) operations := workload.Generate(workload.Config{ Seed: seed, @@ -140,6 +162,62 @@ func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result return result } +// validateCacheContract uses a fresh, small instance to check exact LRU +// semantics independently of the larger performance workload. Otherwise a +// cache that always misses could pass while implementing none of the contract. +func validateCacheContract(seed uint64, factory contract.Factory) string { + unit := contract.AccountedBytes([]byte{1}) + cache := factory(2 * unit) + if cache == nil { + return "cache factory returned nil during contract check" + } + a, b, c := seed+1, seed+2, seed+3 + first := []byte{0x11} + cache.Put(a, first, contract.RequestMeta{}) + first[0] = 0xff + cache.Put(b, []byte{0x22}, contract.RequestMeta{}) + if cache.UsedBytes() != 2*unit { + return "cache must admit and exactly account for two fitting entries" + } + value, ok := cache.Get(a, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{0x11}) { + return "cache must return the stored value without aliasing the input" + } + value[0] = 0xee + value, ok = cache.Get(a, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{0x11}) { + return "cache Get must return an owned copy" + } + cache.Put(c, []byte{0x33}, contract.RequestMeta{}) + if _, ok := cache.Get(b, contract.RequestMeta{}); ok { + return "cache must evict the least recently used entry" + } + for key, want := range map[uint64]byte{a: 0x11, c: 0x33} { + value, ok = cache.Get(key, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{want}) { + return "cache evicted or changed a more recent entry" + } + } + cache.Delete(a) + if _, ok := cache.Get(a, contract.RequestMeta{}); ok || cache.UsedBytes() != unit { + return "cache Delete must remove the key and release its accounted bytes" + } + cache.Put(c, make([]byte, 2*unit), contract.RequestMeta{}) + value, ok = cache.Get(c, contract.RequestMeta{}) + if !ok || !bytes.Equal(value, []byte{0x33}) || cache.UsedBytes() != unit { + return "oversized replacement must leave the resident value intact" + } + zero := factory(0) + if zero == nil { + return "cache factory returned nil for zero capacity" + } + zero.Put(a, []byte{0x11}, contract.RequestMeta{}) + if _, ok := zero.Get(a, contract.RequestMeta{}); ok || zero.UsedBytes() != 0 { + return "zero-capacity cache must not admit entries" + } + return "" +} + func backingValue(seed, key, version uint64) []byte { value := make([]byte, valueBytes) state := seed ^ rotateLeft(key, 17) ^ rotateLeft(version, 41) diff --git a/challenges/cache-pressure-easy/harness/harness_test.go b/challenges/cache-pressure-easy/harness/harness_test.go index 8f72a3e..4a6746e 100644 --- a/challenges/cache-pressure-easy/harness/harness_test.go +++ b/challenges/cache-pressure-easy/harness/harness_test.go @@ -3,11 +3,48 @@ package harness import ( "encoding/json" "reflect" + "strings" "testing" + "github.com/roughstack/challenges/challenges/cache-pressure-easy/contract" "github.com/roughstack/challenges/challenges/cache-pressure-easy/starter" ) +type neverAdmitCache struct{} + +func (neverAdmitCache) Get(uint64, contract.RequestMeta) ([]byte, bool) { return nil, false } +func (neverAdmitCache) Put(uint64, []byte, contract.RequestMeta) {} +func (neverAdmitCache) Delete(uint64) {} +func (neverAdmitCache) UsedBytes() uint64 { return 0 } + +type ignoreDeleteCache struct{ contract.Cache } + +func (ignoreDeleteCache) Delete(uint64) {} + +func TestEvaluateRejectsCachesThatDoNotImplementTheContract(t *testing.T) { + tests := []struct { + name string + factory contract.Factory + want string + }{ + {"never admits", func(uint64) contract.Cache { return neverAdmitCache{} }, "admit"}, + {"ignores delete", func(capacity uint64) contract.Cache { + return ignoreDeleteCache{starter.NewCache(capacity)} + }, "Delete"}, + {"nil cache", func(uint64) contract.Cache { return nil }, "nil"}, + {"nil factory", nil, "nil"}, + {"panicking factory", func(uint64) contract.Cache { panic("broken") }, "panicked"}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + result := Evaluate(17, DefaultPublicConfig(), test.factory) + if result.Verdict != "fail" || len(result.Violations) == 0 || !strings.Contains(result.Violations[0], test.want) { + t.Fatalf("expected %q failure, got %+v", test.want, result) + } + }) + } +} + func TestEvaluateIsDeterministicAndWithinCapacity(t *testing.T) { config := DefaultPublicConfig() first := Evaluate(^uint64(0), config, starter.NewCache) From 8908edb576eaceeeee7345042cfefb11fbec2f20 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:34:17 +0530 Subject: [PATCH 03/22] docs(cache): clarify constant-time recency structure --- challenges/cache-pressure-easy/statement.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/challenges/cache-pressure-easy/statement.json b/challenges/cache-pressure-easy/statement.json index 500c05f..d3a51b4 100644 --- a/challenges/cache-pressure-easy/statement.json +++ b/challenges/cache-pressure-easy/statement.json @@ -39,7 +39,7 @@ "kind": "example", "input": "capacity = 2 entries\nPut(A), Put(B), Get(A), Put(C)", "output": "recency: C → A\nevicted: B", - "explain": "`Get(A)` makes A newest. When C arrives, B is the oldest entry, so B leaves. Use a map for lookup plus an ordered recency list for constant-time movement." + "explain": "`Get(A)` makes A newest. When C arrives, B is the oldest entry, so B leaves. A map from keys to nodes in a doubly linked list gives constant-time lookup, recency updates, and eviction." }, { "kind": "constraints", From 2192095cf5ee15f3237660b9b61c3a2cba4d61ac Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:40 +0530 Subject: [PATCH 04/22] fix(cache): enforce medium public hit-rate floor --- challenges/cache-pressure-medium/README.md | 5 ++++- .../cache-pressure-medium/harness/harness.go | 14 ++++++++++---- .../harness/harness_test.go | 4 ++++ .../cache-pressure-medium/statement.json | 19 +++++++++---------- 4 files changed, 27 insertions(+), 15 deletions(-) diff --git a/challenges/cache-pressure-medium/README.md b/challenges/cache-pressure-medium/README.md index 431ddac..b01265c 100644 --- a/challenges/cache-pressure-medium/README.md +++ b/challenges/cache-pressure-medium/README.md @@ -60,7 +60,10 @@ protection against scan pollution. budget at peak occupancy. Lower means the cache exploited its budget more fully; it is a soft signal alongside the hit-rate and cost metrics. -Correctness gates scoring. Public smoke output is reproducible but unofficial; +The default public workload also requires a request hit rate of at least 4000 +basis points (40%). This rejects a never-admit policy without requiring one +particular eviction or admission algorithm. Correctness gates scoring. Public +smoke output is reproducible but unofficial; normalization and calibration belong to the platform and private judge. ## Verify diff --git a/challenges/cache-pressure-medium/harness/harness.go b/challenges/cache-pressure-medium/harness/harness.go index c12a61f..e5d5def 100644 --- a/challenges/cache-pressure-medium/harness/harness.go +++ b/challenges/cache-pressure-medium/harness/harness.go @@ -56,6 +56,8 @@ type PublicConfig struct { Operations int HotKeys uint64 ScanKeys uint64 + // Zero disables the performance floor for small contract-focused tests. + MinRequestHitRateBasisPoints uint64 } // DefaultPublicConfig returns the fast public smoke workload. The scan is @@ -74,10 +76,11 @@ func DefaultPublicConfig() PublicConfig { scanKeys = 1 } return PublicConfig{ - CapacityBytes: capacity, - Operations: 4096, - HotKeys: 16, - ScanKeys: scanKeys, + CapacityBytes: capacity, + Operations: 4096, + HotKeys: 16, + ScanKeys: scanKeys, + MinRequestHitRateBasisPoints: ScanResistanceFloorBasisPoints, } } @@ -216,6 +219,9 @@ func Evaluate(seed uint64, config PublicConfig, factory contract.Factory) Result if reads > 0 { result.Metrics.RequestHitRate = readHits * 10000 / reads } + if config.MinRequestHitRateBasisPoints > 0 && result.Metrics.RequestHitRate < config.MinRequestHitRateBasisPoints { + result.Violations = append(result.Violations, "request hit rate is below the public minimum") + } if peakUsed < config.CapacityBytes { result.Metrics.PeakMemoryHeadroom = config.CapacityBytes - peakUsed } diff --git a/challenges/cache-pressure-medium/harness/harness_test.go b/challenges/cache-pressure-medium/harness/harness_test.go index fd92573..024aa41 100644 --- a/challenges/cache-pressure-medium/harness/harness_test.go +++ b/challenges/cache-pressure-medium/harness/harness_test.go @@ -74,6 +74,10 @@ func TestScanResistanceWorkloadAndFloor(t *testing.T) { if noCacheRate != 0 { t.Fatalf("no-cache policy reported a non-zero hit rate: %d", noCacheRate) } + noCacheResult := Evaluate(^uint64(0), config, newNoCache) + if noCacheResult.Verdict != "fail" || !containsViolation(noCacheResult.Violations, "request hit rate is below the public minimum") { + t.Fatalf("never-admit cache escaped the public performance floor: %+v", noCacheResult) + } if starterRate < ScanResistanceFloorBasisPoints { t.Fatalf("starter request hit rate %d is below the published floor %d", starterRate, ScanResistanceFloorBasisPoints) } diff --git a/challenges/cache-pressure-medium/statement.json b/challenges/cache-pressure-medium/statement.json index b5336f4..19b6df7 100644 --- a/challenges/cache-pressure-medium/statement.json +++ b/challenges/cache-pressure-medium/statement.json @@ -9,16 +9,14 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.snapshot", "props": { - "label": "Scan pollution", - "nodes": [ - { "label": "Hot set", "detail": "Repeated, valuable hits", "tone": "signal" }, - { "label": "Cold scan", "detail": "One cache-sized pass", "tone": "warning" }, - { "label": "Single LRU", "detail": "Cold keys become newest", "tone": "danger" }, - { "label": "Hot set again", "detail": "Useful keys now miss", "tone": "warning" } - ], - "edges": ["warm", "displaces", "pays twice"] + "title": "What a one-time scan does to a small cache", + "rows": [ + { "label": "Warm", "cells": [{ "label": "H1", "tone": "signal" }, { "label": "H2", "tone": "signal" }, { "label": "H3", "tone": "signal" }], "note": "The hot set is reused often." }, + { "label": "Cold scan", "cells": [{ "label": "S1", "tone": "warning" }, { "label": "S2", "tone": "warning" }, { "label": "S3", "tone": "warning" }], "note": "Pure LRU fills the budget with one-hit keys." }, + { "label": "Return", "cells": [{ "label": "H1 miss", "tone": "danger" }, { "label": "H2 miss", "tone": "danger" }, { "label": "H3 miss", "tone": "danger" }], "note": "A protected hot segment would keep these requests cheap." } + ] } }, "caption": "Pure LRU mistakes recent for reusable. A one-time scan looks important long enough to evict the data that will actually return." @@ -57,7 +55,8 @@ "Charge `len(value) + contract.PerEntryMetadataBytes` for every resident entry.", "Reject oversized inserts and replacements without evicting existing data.", "`TTL = 0` never expires; `Get` never extends expiry.", - "Copy caller buffers and remove expired entries from both lookup and accounting." + "Copy caller buffers and remove expired entries from both lookup and accounting.", + "The public workload requires a request hit rate of at least 40%; a cache that admits nothing does not pass." ] }, { From e52aa395d9c89764f79e408869b8e14b8c9e93fa Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:48 +0530 Subject: [PATCH 05/22] docs(cache): show LRU state transitions --- challenges/cache-pressure-easy/statement.json | 20 +++++++++---------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/challenges/cache-pressure-easy/statement.json b/challenges/cache-pressure-easy/statement.json index d3a51b4..88a34f3 100644 --- a/challenges/cache-pressure-easy/statement.json +++ b/challenges/cache-pressure-easy/statement.json @@ -9,19 +9,17 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.snapshot", "props": { - "label": "A cache read path", - "nodes": [ - { "label": "Request", "detail": "Get key 7" }, - { "label": "Cache", "detail": "Look up and refresh recency", "tone": "accent" }, - { "label": "Backing store", "detail": "Read only on a miss", "tone": "warning" }, - { "label": "Caller", "detail": "Receive an owned copy", "tone": "signal" } - ], - "edges": ["lookup", "miss", "fill + return"] + "title": "Oldest on the left · newest on the right", + "rows": [ + { "label": "After two puts", "cells": [{ "label": "A", "detail": "oldest", "tone": "warning" }, { "label": "B", "detail": "newest", "tone": "accent" }] }, + { "label": "Get A", "cells": [{ "label": "B", "detail": "oldest", "tone": "warning" }, { "label": "A", "detail": "newest", "tone": "signal" }], "note": "A hit changes order without fetching from the backing store." }, + { "label": "Put C", "cells": [{ "label": "A", "detail": "oldest", "tone": "accent" }, { "label": "C", "detail": "newest", "tone": "signal" }], "note": "B leaves because it was least recently used." } + ] } }, - "caption": "A hit skips the backing store. A miss fetches once, admits the value if it fits, and returns a copy." + "caption": "One hit changes the eviction decision. This two-entry example illustrates order; real capacity counts each value's bytes plus metadata." }, { "kind": "callout", @@ -62,7 +60,7 @@ "kind": "callout", "tone": "note", "title": "What the score is saying", - "md": "Fewer `backing_reads` means the policy retained useful data. Lower `metadata_work` means the same decisions required less bookkeeping. Correctness is always the gate." + "md": "`backing_reads` counts misses on the public trace. `metadata_work` counts harness-visible calls and accounting checks; it does not measure the time your internal recency structure takes. Correctness is the gate, and the public score is currently unofficial." } ] } From 1a61e1b9226c754ff9498071659256ee61fedbef Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:48 +0530 Subject: [PATCH 06/22] docs(chunk): illustrate frame-local reads --- challenges/chunk-store-easy/statement.json | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/challenges/chunk-store-easy/statement.json b/challenges/chunk-store-easy/statement.json index b0f9b65..fafc1f9 100644 --- a/challenges/chunk-store-easy/statement.json +++ b/challenges/chunk-store-easy/statement.json @@ -9,17 +9,14 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.frames", "props": { - "label": "Blob ingest pipeline", - "nodes": [ - { "label": "Blob", "detail": "Immutable bytes" }, - { "label": "Chunk", "detail": "Split at fixed block size", "tone": "accent" }, - { "label": "Encode", "detail": "Compress only when smaller", "tone": "warning" }, - { "label": "Frame", "detail": "Lengths + CRC-32", "tone": "signal" }, - { "label": "Device", "detail": "Append block bytes" } - ], - "edges": ["split", "per block", "protect", "write"] + "title": "One blob · independently readable frames", + "note": "ReadAt uses the index to select a frame", + "frames": [ + { "label": "Block 0", "status": "not read", "tone": "neutral", "fields": [{ "label": "header", "detail": "lengths", "tone": "neutral" }, { "label": "AAAA", "detail": "payload", "tone": "neutral" }, { "label": "CRC", "detail": "integrity", "tone": "neutral" }] }, + { "label": "Block 1", "status": "read + verified", "tone": "signal", "fields": [{ "label": "header", "detail": "lengths", "tone": "accent" }, { "label": "BCDE", "detail": "requested CD", "tone": "signal" }, { "label": "CRC", "detail": "checked", "tone": "warning" }], "note": "Offset 5, length 2 touches only this block in the example below." } + ] } }, "caption": "Compression and integrity are block-local. That locality is what makes exact random reads cheap and bounded." From 03b64530362f54dddd40fae3f62deb5c06cca7e4 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:48 +0530 Subject: [PATCH 07/22] docs(scheduler): show non-preemptive job timeline --- .../deadline-scheduler-easy/statement.json | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/challenges/deadline-scheduler-easy/statement.json b/challenges/deadline-scheduler-easy/statement.json index ad8ff78..6ab5172 100644 --- a/challenges/deadline-scheduler-easy/statement.json +++ b/challenges/deadline-scheduler-easy/statement.json @@ -9,15 +9,17 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.timeline", + "id": "systems.ranges", "props": { - "label": "Non-preemptive worker schedule", - "windows": [ - { "label": "Job A", "range": "[0, 6)", "events": [{ "label": "6 ticks", "tone": "warning" }], "finalized": true }, - { "label": "Job B", "range": "[6, 8)", "events": [{ "label": "2 ticks", "tone": "signal" }], "finalized": true }, - { "label": "Job C", "range": "[8, 11)", "events": [{ "label": "3 ticks", "tone": "accent" }] } + "title": "One worker · three non-preemptive jobs", + "min": 0, + "max": 11, + "rows": [ + { "label": "A", "start": 0, "end": 6, "tone": "warning", "note": "Starts at 0; cannot be interrupted when B and C arrive." }, + { "label": "B", "start": 6, "end": 8, "tone": "signal", "note": "Waits until the worker becomes idle." }, + { "label": "C", "start": 8, "end": 11, "tone": "accent", "note": "Ordering is chosen only at an idle boundary." } ], - "watermark": "virtual clock" + "markers": [{ "at": 1, "label": "C arrives", "tone": "danger" }, { "at": 6, "label": "A done", "tone": "signal" }] } }, "caption": "Once A starts it owns the worker until tick 6. Better ordering happens only at idle boundaries." @@ -61,7 +63,7 @@ "kind": "callout", "tone": "note", "title": "Read the metrics together", - "md": "Mean flow rewards short-job ordering; p95 flow and starvation penalty expose policies that sacrifice old jobs. Peak state and scheduler work show the bookkeeping cost." + "md": "Mean flow rewards short-job ordering; p95 flow and starvation penalty expose policies that sacrifice old jobs. Peak state tracks reported memory. `scheduler_work` counts callback and action traffic, not the CPU time of your scheduling algorithm." } ] } From 767f0b191a92974c87c4de32d7e4fa770272c6f5 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:48 +0530 Subject: [PATCH 08/22] docs(queue): show lease and requeue transitions --- challenges/durable-queue-easy/statement.json | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/challenges/durable-queue-easy/statement.json b/challenges/durable-queue-easy/statement.json index bb03e80..c933da2 100644 --- a/challenges/durable-queue-easy/statement.json +++ b/challenges/durable-queue-easy/statement.json @@ -9,15 +9,14 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.state", + "id": "systems.snapshot", "props": { - "label": "Queue job lifecycle", - "states": [ - { "label": "Available", "detail": "FIFO among ready jobs", "tone": "accent" }, - { "label": "Leased", "detail": "Unique token + deadline", "tone": "warning" }, - { "label": "Acknowledged", "detail": "Payload bytes released", "tone": "signal" } - ], - "transitions": ["Lease", "Ack"] + "title": "FIFO is defined by when a job becomes available", + "rows": [ + { "label": "Enqueue", "cells": [{ "label": "A", "detail": "ready", "tone": "accent" }, { "label": "B", "detail": "ready", "tone": "accent" }] }, + { "label": "Lease A", "cells": [{ "label": "B", "detail": "ready", "tone": "accent" }, { "label": "A", "detail": "leased until 10", "tone": "warning" }], "note": "A still occupies capacity while leased." }, + { "label": "Nack A", "cells": [{ "label": "B", "detail": "next", "tone": "signal" }, { "label": "A", "detail": "back of queue", "tone": "accent" }], "note": "A's old token is now stale." } + ] } }, "caption": "Nack or visibility expiry returns a leased job to the back of the available queue. It receives a fresh token when leased again." @@ -61,7 +60,7 @@ "kind": "callout", "tone": "note", "title": "Why virtual time matters", - "md": "Lease expiry is driven by the injected `nowTick`, never wall time. The same seed therefore produces the same redeliveries, waits, and score." + "md": "Lease expiry is driven by the injected `nowTick`, never wall time. The same seed therefore produces the same redeliveries and logical wait measurements. The public score is currently unofficial." } ] } From 79644c2bd91b045d3cc3592b5f7318e3d313769f Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:49 +0530 Subject: [PATCH 09/22] docs(gate): visualize token-bucket budget --- challenges/fair-gate-easy/statement.json | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/challenges/fair-gate-easy/statement.json b/challenges/fair-gate-easy/statement.json index 0f3ce16..849aeb7 100644 --- a/challenges/fair-gate-easy/statement.json +++ b/challenges/fair-gate-easy/statement.json @@ -9,16 +9,16 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.budget", "props": { - "label": "Token bucket decision", - "nodes": [ - { "label": "Elapsed ticks", "detail": "Compute lazy refill" }, - { "label": "Bucket", "detail": "Clamp tokens to Burst", "tone": "accent" }, - { "label": "Request cost", "detail": "Compare whole tokens", "tone": "warning" }, - { "label": "Decision", "detail": "Allow or exact RetryAt", "tone": "signal" } - ], - "edges": ["refill", "then", "emit"] + "title": "A burst of four tokens · refill two per three ticks", + "capacity": 4, + "snapshots": [ + { "label": "tick 0 · full", "segments": [{ "label": "available", "value": 4, "tone": "signal" }] }, + { "label": "tick 0 · after cost 3", "segments": [{ "label": "available", "value": 1, "tone": "warning" }], "note": "a second cost-3 request is rejected" }, + { "label": "tick 3 · before retry", "segments": [{ "label": "available", "value": 3, "tone": "signal" }], "note": "two tokens accrued lazily" }, + { "label": "tick 3 · after retry", "segments": [{ "label": "available", "value": 0, "tone": "accent" }], "note": "the retry is now admitted" } + ] } }, "caption": "No timer updates the bucket in the background. Each request first accounts for elapsed virtual time, then makes one deterministic decision." @@ -62,7 +62,7 @@ "kind": "callout", "tone": "note", "title": "Optimization target", - "md": "Once decisions and `RetryAt` are exact, reduce repeated arithmetic and allocation. Throughput rewards the fast path; `arithmetic_work` makes avoidable recomputation visible." + "md": "Once decisions and `RetryAt` are exact, allocations may distinguish implementations. The public `decisions_per_second` and `arithmetic_work` counters depend on the fixed trace, not on how fast your code runs or how many arithmetic operations it performs." } ] } From fee994a1ad609be79002df3d9ebfe15e0abb2c6d Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:49 +0530 Subject: [PATCH 10/22] docs(journal): show durable frame frontier --- challenges/journal-recovery-easy/statement.json | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/challenges/journal-recovery-easy/statement.json b/challenges/journal-recovery-easy/statement.json index 2ccd9a2..d85a2cf 100644 --- a/challenges/journal-recovery-easy/statement.json +++ b/challenges/journal-recovery-easy/statement.json @@ -9,16 +9,15 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.frames", "props": { - "label": "Journal append and durability", - "nodes": [ - { "label": "Record", "detail": "Copy caller payload" }, - { "label": "Frame", "detail": "Magic · length · LSN · CRC", "tone": "accent" }, - { "label": "Write", "detail": "Append bytes to Device", "tone": "warning" }, - { "label": "Sync", "detail": "Advance durability boundary", "tone": "signal" } - ], - "edges": ["encode", "append", "make durable"] + "title": "The durable prefix ends after LSN 2", + "note": "recovery scans left to right", + "frames": [ + { "label": "LSN 1", "status": "synced", "tone": "signal", "fields": [{ "label": "magic", "tone": "accent" }, { "label": "len + seq", "tone": "accent" }, { "label": "payload", "tone": "signal" }, { "label": "CRC", "tone": "warning" }] }, + { "label": "LSN 2", "status": "synced", "tone": "signal", "fields": [{ "label": "magic", "tone": "accent" }, { "label": "len + seq", "tone": "accent" }, { "label": "payload", "tone": "signal" }, { "label": "CRC", "tone": "warning" }] }, + { "label": "LSN 3", "status": "torn tail", "tone": "danger", "fields": [{ "label": "magic", "tone": "accent" }, { "label": "len + seq", "tone": "accent" }, { "label": "partial", "detail": "payload", "tone": "danger" }, { "label": "missing", "detail": "bytes + CRC", "tone": "neutral" }], "note": "A declared frame that ends early is not applied." } + ] } }, "caption": "Append gives a record an LSN, but only Sync makes the requested prefix durable. Recovery trusts structure and checksum, not intent." From 2b84ebc62e6a24740dedba57d888c623a6e84b8e Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:49 +0530 Subject: [PATCH 11/22] docs(link): trace lost ACK recovery --- challenges/lossy-link-easy/statement.json | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/challenges/lossy-link-easy/statement.json b/challenges/lossy-link-easy/statement.json index 65869d6..de01c3a 100644 --- a/challenges/lossy-link-easy/statement.json +++ b/challenges/lossy-link-easy/statement.json @@ -3,21 +3,24 @@ "blocks": [ { "kind": "prose", - "md": "Build reliability above an unreliable link. With only one unacknowledged packet at a time, the protocol needs a sequence bit, a checksum, an acknowledgement, and a retransmission timer." + "md": "Build reliability above an unreliable link. With only one unacknowledged packet at a time, the protocol needs a small wrapping sequence number, a checksum, an acknowledgement, and a retransmission timer." }, { "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.state", + "id": "systems.exchange", "props": { - "label": "Stop-and-wait sender", - "states": [ - { "label": "Ready", "detail": "Accept the next app chunk", "tone": "signal" }, - { "label": "Waiting", "detail": "One packet is in flight", "tone": "warning" }, - { "label": "Advance", "detail": "Matching ACK flips sequence", "tone": "accent" } - ], - "transitions": ["send DATA", "receive ACK"] + "title": "A lost ACK is why duplicates must be safe", + "left": "Sender", + "right": "Receiver", + "events": [ + { "from": "left", "label": "DATA 0 · abc", "detail": "deliver abc", "tone": "signal" }, + { "from": "right", "label": "ACK 0", "detail": "waiting", "dropped": true, "tone": "danger" }, + { "from": "local", "label": "timer fires", "tone": "warning" }, + { "from": "left", "label": "DATA 0 · retry", "detail": "duplicate, no delivery", "tone": "warning" }, + { "from": "right", "label": "ACK 0 again", "detail": "advance to 1", "tone": "accent" } + ] } }, "caption": "Loss or corruption leaves the sender waiting. When its timer fires, it retransmits the same packet; the receiver discards duplicates but repeats the ACK." From de2d9e591c4344f0edab3ef347c8b6167ff46c85 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:49 +0530 Subject: [PATCH 12/22] docs(lsm): illustrate overlap closure --- challenges/lsm-compactor-easy/statement.json | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/challenges/lsm-compactor-easy/statement.json b/challenges/lsm-compactor-easy/statement.json index e50815a..a1cd3a7 100644 --- a/challenges/lsm-compactor-easy/statement.json +++ b/challenges/lsm-compactor-easy/statement.json @@ -9,19 +9,21 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.flow", + "id": "systems.ranges", "props": { - "label": "Two-level LSM lifecycle", - "nodes": [ - { "label": "Memtable", "detail": "Mutable recent entries", "tone": "signal" }, - { "label": "Level 0", "detail": "New immutable, overlapping runs", "tone": "warning" }, - { "label": "Compaction", "detail": "Merge newest value per key", "tone": "accent" }, - { "label": "Level 1", "detail": "Sorted non-overlapping runs" } + "title": "Selecting A + B closes over both Level-1 runs", + "min": 0, + "max": 60, + "rows": [ + { "label": "L0 A", "start": 10, "end": 30, "tone": "warning", "note": "Selected run" }, + { "label": "L0 B", "start": 25, "end": 50, "tone": "warning", "note": "Selected run, overlaps A" }, + { "label": "L1 X", "start": 0, "end": 20, "tone": "accent", "note": "Must join: overlaps A" }, + { "label": "L1 Y", "start": 40, "end": 60, "tone": "accent", "note": "Must join: overlaps B" } ], - "edges": ["flush", "pick inputs", "rewrite"] + "markers": [{ "at": 10, "label": "union starts", "tone": "signal" }, { "at": 50, "label": "union ends", "tone": "signal" }] } }, - "caption": "Waiting creates more Level-0 overlap and read work. Compacting too eagerly rewrites data repeatedly and stalls the write path." + "caption": "The selected Level-0 range spans keys 10 to 50. Both Level-1 runs intersect it, so a legal merge must include X and Y." }, { "kind": "callout", From 4c692111b6a568adc2157212e513639ed0623a52 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:49 +0530 Subject: [PATCH 13/22] docs(stream): show watermark boundary --- challenges/stream-windows-easy/statement.json | 28 ++++++------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/challenges/stream-windows-easy/statement.json b/challenges/stream-windows-easy/statement.json index e0310fa..e43c834 100644 --- a/challenges/stream-windows-easy/statement.json +++ b/challenges/stream-windows-easy/statement.json @@ -9,26 +9,16 @@ "kind": "figure", "label": "FIG 1", "illustration": { - "id": "systems.timeline", + "id": "systems.ranges", "props": { - "label": "Fixed event-time windows", - "windows": [ - { - "label": "Window 0", - "range": "[0, 10)", - "events": [ - { "label": "api +3", "tone": "accent" }, - { "label": "api +5", "tone": "accent" } - ], - "finalized": true - }, - { - "label": "Window 1", - "range": "[10, 20)", - "events": [{ "label": "db +2", "tone": "warning" }] - } + "title": "Half-open windows on the event-time axis", + "min": 0, + "max": 20, + "rows": [ + { "label": "[0, 10)", "start": 0, "end": 10, "tone": "signal", "note": "api +3 and api +5 · emit count 2, sum 8" }, + { "label": "[10, 20)", "start": 10, "end": 20, "tone": "warning", "note": "db +2 · still live at watermark 10" } ], - "watermark": "10" + "markers": [{ "at": 10, "label": "watermark", "tone": "accent" }] } }, "caption": "At watermark 10, Window 0 is final: emit api(count=2, sum=8) and evict it. Window 1 remains live." @@ -72,7 +62,7 @@ "kind": "callout", "tone": "note", "title": "Why the metrics fit the model", - "md": "Throughput rewards a small hot path; peak state checks timely eviction; output delay checks prompt finalization; allocation bytes reveal per-event churn." + "md": "Peak state checks timely eviction; allocation bytes reveal per-event churn. In the public harness, event throughput follows the fixed virtual-time trace and output delay is fixed for any correct implementation that emits at the watermark. These counters provide context, not evidence that one correct operator is faster." } ] } From b7b8aa9a96df9407cbf6291dc329f9363f40c66a Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:52 +0530 Subject: [PATCH 14/22] docs(gate): mark public scoring metadata consistently --- challenges/fair-gate-easy/scoring.yaml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/challenges/fair-gate-easy/scoring.yaml b/challenges/fair-gate-easy/scoring.yaml index 7c500e2..e42706c 100644 --- a/challenges/fair-gate-easy/scoring.yaml +++ b/challenges/fair-gate-easy/scoring.yaml @@ -1,3 +1,7 @@ +version: 1 +score_range: [0, 10000] +correctness_gate: true +public_score: unofficial metrics: decisions_per_second: weight: 0.60 From c2e54ebf6a21c397fb01ecf4a2485e72e76e7ae3 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:48:54 +0530 Subject: [PATCH 15/22] docs(challenges): record tested behavior and release gaps --- docs/challenge-audit-2026-10-15.md | 48 ++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 docs/challenge-audit-2026-10-15.md diff --git a/docs/challenge-audit-2026-10-15.md b/docs/challenge-audit-2026-10-15.md new file mode 100644 index 0000000..c90e72b --- /dev/null +++ b/docs/challenge-audit-2026-10-15.md @@ -0,0 +1,48 @@ +# Challenge audit, 2026-10-15 + +This is the current evidence, not a claim that finite tests prove every possible +implementation correct. The companion [quality matrix](challenge-quality.md) +lists the adversarial and boundary tests each challenge should ultimately own. + +## Evidence checked + +- All ten Go starters passed their public smoke workloads at seeds `12345` and + `987654321`. The full Go suite passed with the race detector. +- The original SSD execution runtime's Go and conformance suites passed. +- All thirteen challenge pages rendered an accessible educational diagram in + the browser. The web unit suite, a phone-sized problem/editor interaction + check, and the existing 40-viewport layout audit passed. +- The easy cache now has a standalone contract probe that rejects an incorrect + LRU, accounting, aliasing, deletion, or oversized-replacement implementation. + The medium cache now enforces its advertised 40% request-hit floor on the + default public workload, so a never-admit cache fails. + +## Challenge-by-challenge conclusion + +| Challenge | Does the problem have a coherent solution? | Public evidence and next gap | +| --- | --- | --- | +| Cache Pressure, easy | Yes. Exact LRU behavior is prescribed; implementations need not share a data structure. | Starter and contract probe pass. Add a small independent model across randomized operation sequences and a candidate that lies about `UsedBytes` while retaining data. | +| Cache Pressure, medium | Yes. Many admission/eviction policies are legal; the public workload now requires at least 40% request hits. | Starter passes the floor across the checked seeds; never-admit fails. Add a model-based TTL/accounting probe independent of the generated workload, especially replacement at an exact expiry tick. | +| Chunk Store | Yes. Bytes returned and frame-local integrity are definite even though internal indexing can differ. | Corrupt length, checksum, over-allocation, unrelated reads, and reclaim tests exist. Expand every-tail-truncation and cross-block read cases under a tiny independent frame oracle. | +| Deadline Scheduler | Yes. Many fair, non-preemptive schedules are valid; no single order is mandated. | Arrival, ties, aging, idle gaps, failure gating, and starter tests pass. Add exhaustive short-job schedule comparison to validate flow-time accounting and starvation bounds. | +| Durable Work Queue | Yes. The visible lease/ack/nack behavior is definite; lock and storage choices are not. | Correctness, panic, nil, and watchdog tests pass. Add a concurrent history checker for no loss or duplication and more exact visibility-deadline ties. | +| Fair Gate | Yes. Given a token bucket config and request trace, allow/deny and retry tick are mathematically determined. | Tests reject over-admission, over-charging, and wrong retry ticks. Add an independent rational-arithmetic oracle for randomly generated short traces and overflow boundaries. | +| Journal Recovery | Yes. Recovery must return the valid durable prefix, regardless of internal data structure. | Torn tails, LSN gaps/duplicates, callback errors, corruption, panic, and nil cases pass. Add every possible truncation position for short encoded logs and interior-versus-tail fault pairs. | +| Lossy Link | Yes. Delivered bytes are fixed; retransmission schedule and packet count can vary. | Loss, corruption, duplicate, stale ACK, timer, wrap, and boundary tests pass. Add a bounded fault-sequence enumerator to check safety and liveness against many combined faults. | +| LSM Compactor | Yes. Several legal plans exist; reads after merging must preserve newest-version and tombstone semantics. | Invalid overlap closure, stale/duplicate plans, tombstones, empty runs, and tie-breaking pass. Add an independent key-version map oracle over enumerated small run graphs. | +| Stream Windows | Yes. Half-open window assignment and finalized sums are unique for a legal event trace. | Boundary, regression, duplicate/premature output, negative sum, close, and max-tick tests pass. Add randomized per-key map oracle and repeated watermark/event interleavings. | +| Compaction, original SSD | Yes. A minimum move count exists, though more than one move sequence can attain it. | Runtime tests demonstrate a legal optimal solution and reject illegal moves. Retain cross-language submitted-source conformance as a release gate. | +| Victim Selection, original SSD | Yes. Any legal policy preserves data; the best online choice depends on future writes. | Runtime simulation and conformance pass. Add property tests for page-map conservation and migration accounting under varied seeds. | +| Wear Levelling, original SSD | Yes. Legal policies trade migration cost against erase-count spread; there is no single mandated policy. | Runtime simulation and conformance pass. Add adversarial hot/cold workloads and explicit Pareto trade-off checks. | + +## Publication limits + +All ten generated public smoke results currently report `score: 0`, including +correct starters. Their weighted scoring files are design intent, not an active +ranking system. Some displayed measurements, notably fair-gate throughput and +arithmetic work and stream-window event throughput/output delay, are fixed by +the public trace once the answer is correct; they should be labeled as trace +context until a contestant-sensitive measurement replaces them. Generated +challenges currently ship Go only, whereas the original SSD ladder has the +multi-language runtime. Do not advertise the generated set as ranked or +multi-language until those separate gates are implemented and tested. From 90128471e4070f6979a4a5d2bb7a85b3a71403d7 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:52:32 +0530 Subject: [PATCH 16/22] docs(challenges): align examples and ranking language --- challenges/cache-pressure-easy/statement.json | 10 +++++----- challenges/durable-queue-easy/statement.json | 2 +- challenges/lossy-link-easy/statement.json | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/challenges/cache-pressure-easy/statement.json b/challenges/cache-pressure-easy/statement.json index 88a34f3..fafeb82 100644 --- a/challenges/cache-pressure-easy/statement.json +++ b/challenges/cache-pressure-easy/statement.json @@ -35,8 +35,8 @@ }, { "kind": "example", - "input": "capacity = 2 entries\nPut(A), Put(B), Get(A), Put(C)", - "output": "recency: C → A\nevicted: B", + "input": "capacity = 66 bytes (two 1-byte values, each charged 32 metadata bytes)\nPut(A, 1B), Put(B, 1B), Get(A), Put(C, 1B)", + "output": "newest → oldest: C, A\nevicted: B", "explain": "`Get(A)` makes A newest. When C arrives, B is the oldest entry, so B leaves. A map from keys to nodes in a doubly linked list gives constant-time lookup, recency updates, and eviction." }, { @@ -53,14 +53,14 @@ "items": [ { "title": "Make accounting exact", "md": "Centralize each entry's charged size and update it on insert, replacement, and eviction." }, { "title": "Separate lookup from order", "md": "Use one structure to find a key and another to move that entry to the newest position." }, - { "title": "Optimize after correctness", "md": "A linear list can pass semantics; constant-time recency maintenance improves `metadata_work`." } + { "title": "Optimize after correctness", "md": "A linear list can pass semantics; a map plus linked list improves actual lookup and update time. The public `metadata_work` counter does not measure that internal time." } ] }, { "kind": "callout", "tone": "note", - "title": "What the score is saying", - "md": "`backing_reads` counts misses on the public trace. `metadata_work` counts harness-visible calls and accounting checks; it does not measure the time your internal recency structure takes. Correctness is the gate, and the public score is currently unofficial." + "title": "What the metrics say", + "md": "`backing_reads` counts misses on the public trace. `metadata_work` counts harness-visible calls and accounting checks; it does not measure the time your internal recency structure takes. Correctness is the gate; this practice challenge is not ranked yet." } ] } diff --git a/challenges/durable-queue-easy/statement.json b/challenges/durable-queue-easy/statement.json index c933da2..a9674ac 100644 --- a/challenges/durable-queue-easy/statement.json +++ b/challenges/durable-queue-easy/statement.json @@ -60,7 +60,7 @@ "kind": "callout", "tone": "note", "title": "Why virtual time matters", - "md": "Lease expiry is driven by the injected `nowTick`, never wall time. The same seed therefore produces the same redeliveries and logical wait measurements. The public score is currently unofficial." + "md": "Lease expiry is driven by the injected `nowTick`, never wall time. The same seed therefore produces the same redeliveries and logical wait measurements. This practice challenge is not ranked yet." } ] } diff --git a/challenges/lossy-link-easy/statement.json b/challenges/lossy-link-easy/statement.json index de01c3a..5c7a648 100644 --- a/challenges/lossy-link-easy/statement.json +++ b/challenges/lossy-link-easy/statement.json @@ -63,7 +63,7 @@ { "kind": "callout", "tone": "note", - "title": "Read the score as overhead", + "title": "Read the metrics as overhead", "md": "Completion ticks measure waiting; transmitted bytes, packet count, and retransmissions measure how much extra work reliability required; state bytes price the protocol memory." } ] From 0664b1bc25df0abcb8c54521ef3d8f457590a981 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 01:58:12 +0530 Subject: [PATCH 17/22] docs(challenges): record production submission verification --- docs/challenge-audit-2026-10-15.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/challenge-audit-2026-10-15.md b/docs/challenge-audit-2026-10-15.md index c90e72b..a3dcdad 100644 --- a/docs/challenge-audit-2026-10-15.md +++ b/docs/challenge-audit-2026-10-15.md @@ -8,6 +8,10 @@ lists the adversarial and boundary tests each challenge should ultimately own. - All ten Go starters passed their public smoke workloads at seeds `12345` and `987654321`. The full Go suite passed with the race detector. +- All ten starters were submitted through the web platform's production build + and completed with passing verdicts in an isolated local database. This caught + a platform result-schema mismatch: Journal Recovery and Lossy Link include + the optional `run.fault_profile` field, which the platform now accepts. - The original SSD execution runtime's Go and conformance suites passed. - All thirteen challenge pages rendered an accessible educational diagram in the browser. The web unit suite, a phone-sized problem/editor interaction @@ -16,6 +20,10 @@ lists the adversarial and boundary tests each challenge should ultimately own. LRU, accounting, aliasing, deletion, or oversized-replacement implementation. The medium cache now enforces its advertised 40% request-hit floor on the default public workload, so a never-admit cache fails. +- A production node needs both the published arena registry and access to the + matching challenge packages. Registry-only mode can display previously + materialized database content, but cannot refresh statements or execute a + package that is unavailable locally. ## Challenge-by-challenge conclusion From 90d2afb25314505338d7b7fae4d51961cec05439 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:08:46 +0530 Subject: [PATCH 18/22] feat: separate learner scaffolds from public baselines --- challenges/cache-pressure-easy/arena.yaml | 3 +- .../cache-pressure-easy/learner/starter.go | 30 ++++++++++++ challenges/cache-pressure-medium/arena.yaml | 3 +- .../cache-pressure-medium/learner/starter.go | 29 ++++++++++++ challenges/chunk-store-easy/arena.yaml | 3 +- .../chunk-store-easy/learner/starter.go | 42 +++++++++++++++++ challenges/deadline-scheduler-easy/arena.yaml | 3 +- .../learner/starter.go | 28 +++++++++++ challenges/durable-queue-easy/arena.yaml | 3 +- .../durable-queue-easy/learner/starter.go | 37 +++++++++++++++ challenges/fair-gate-easy/arena.yaml | 3 +- challenges/fair-gate-easy/learner/starter.go | 17 +++++++ challenges/journal-recovery-easy/arena.yaml | 3 +- .../journal-recovery-easy/learner/starter.go | 29 ++++++++++++ challenges/lossy-link-easy/arena.yaml | 3 +- challenges/lossy-link-easy/learner/starter.go | 47 +++++++++++++++++++ challenges/lsm-compactor-easy/arena.yaml | 3 +- .../lsm-compactor-easy/learner/starter.go | 16 +++++++ challenges/stream-windows-easy/arena.yaml | 3 +- .../stream-windows-easy/learner/starter.go | 29 ++++++++++++ 20 files changed, 324 insertions(+), 10 deletions(-) create mode 100644 challenges/cache-pressure-easy/learner/starter.go create mode 100644 challenges/cache-pressure-medium/learner/starter.go create mode 100644 challenges/chunk-store-easy/learner/starter.go create mode 100644 challenges/deadline-scheduler-easy/learner/starter.go create mode 100644 challenges/durable-queue-easy/learner/starter.go create mode 100644 challenges/fair-gate-easy/learner/starter.go create mode 100644 challenges/journal-recovery-easy/learner/starter.go create mode 100644 challenges/lossy-link-easy/learner/starter.go create mode 100644 challenges/lsm-compactor-easy/learner/starter.go create mode 100644 challenges/stream-windows-easy/learner/starter.go diff --git a/challenges/cache-pressure-easy/arena.yaml b/challenges/cache-pressure-easy/arena.yaml index 3038d3f..7c0ce1f 100644 --- a/challenges/cache-pressure-easy/arena.yaml +++ b/challenges/cache-pressure-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: cache-pressure-easy - version: 1.0.0 + version: 1.0.1 title: Cache Pressure visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/cache.go + starter: learner/starter.go interface: bytearena.cache.Cache/v1 build: command: ["go", "test", "./challenges/cache-pressure-easy/..."] diff --git a/challenges/cache-pressure-easy/learner/starter.go b/challenges/cache-pressure-easy/learner/starter.go new file mode 100644 index 0000000..d78b917 --- /dev/null +++ b/challenges/cache-pressure-easy/learner/starter.go @@ -0,0 +1,30 @@ +// Package starter is the learner-facing scaffold. The verified public +// implementation lives at the manifest entrypoint and is not prefilled here. +package starter + +import "github.com/roughstack/challenges/challenges/cache-pressure-easy/contract" + +type cache struct { + capacity uint64 + // TODO: track stored values, their accounted sizes, and recency. +} + +func NewCache(capacityBytes uint64) contract.Cache { + return &cache{capacity: capacityBytes} +} + +func (c *cache) Get(key uint64, meta contract.RequestMeta) ([]byte, bool) { + panic("TODO: return a private copy and update recency on a hit") +} + +func (c *cache) Put(key uint64, value []byte, meta contract.RequestMeta) { + panic("TODO: enforce the exact byte budget and LRU eviction") +} + +func (c *cache) Delete(key uint64) { + panic("TODO: remove the entry and its accounting") +} + +func (c *cache) UsedBytes() uint64 { + panic("TODO: report the charged bytes, including metadata") +} diff --git a/challenges/cache-pressure-medium/arena.yaml b/challenges/cache-pressure-medium/arena.yaml index 54d37c1..b74f6d0 100644 --- a/challenges/cache-pressure-medium/arena.yaml +++ b/challenges/cache-pressure-medium/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: cache-pressure-medium - version: 1.0.0 + version: 1.0.1 title: Cache Pressure visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/cache.go + starter: learner/starter.go interface: bytearena.cache.Cache/v1 build: command: ["go", "test", "./challenges/cache-pressure-medium/..."] diff --git a/challenges/cache-pressure-medium/learner/starter.go b/challenges/cache-pressure-medium/learner/starter.go new file mode 100644 index 0000000..ef16b32 --- /dev/null +++ b/challenges/cache-pressure-medium/learner/starter.go @@ -0,0 +1,29 @@ +// Package starter is the learner-facing scaffold for the TTL cache. +package starter + +import "github.com/roughstack/challenges/challenges/cache-pressure-medium/contract" + +type cache struct { + capacity uint64 + // TODO: track values, expiry ticks, recency, and byte accounting. +} + +func NewCache(capacityBytes uint64) contract.Cache { + return &cache{capacity: capacityBytes} +} + +func (c *cache) Get(key uint64, meta contract.RequestMeta) ([]byte, bool) { + panic("TODO: expire at the exact tick and update recency") +} + +func (c *cache) Put(key uint64, value []byte, meta contract.RequestMeta) { + panic("TODO: handle expiry, replacement, admission, and eviction") +} + +func (c *cache) Delete(key uint64) { + panic("TODO: remove an entry without leaking its byte charge") +} + +func (c *cache) UsedBytes() uint64 { + panic("TODO: report current accounted bytes") +} diff --git a/challenges/chunk-store-easy/arena.yaml b/challenges/chunk-store-easy/arena.yaml index 18c9fe9..ab10ba5 100644 --- a/challenges/chunk-store-easy/arena.yaml +++ b/challenges/chunk-store-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: chunk-store-easy - version: 1.0.0 + version: 1.0.1 title: Chunk Store visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.chunkstore.Store/v1 build: command: ["go", "test", "./challenges/chunk-store-easy/..."] diff --git a/challenges/chunk-store-easy/learner/starter.go b/challenges/chunk-store-easy/learner/starter.go new file mode 100644 index 0000000..fc14cf8 --- /dev/null +++ b/challenges/chunk-store-easy/learner/starter.go @@ -0,0 +1,42 @@ +// Package starter is the learner-facing scaffold for a framed blob store. +package starter + +import ( + "io" + + "github.com/roughstack/challenges/challenges/chunk-store-easy/contract" +) + +type store struct { + device contract.Device + memoryLimit uint64 + // TODO: keep only a bounded in-memory index. +} + +func OpenStore(_ string, device contract.Device, memoryLimit uint64) (contract.Store, error) { + return &store{device: device, memoryLimit: memoryLimit}, nil +} + +func (s *store) Put(id, version uint64, data io.Reader, size uint64) error { + panic("TODO: encode and commit a verifiable frame") +} + +func (s *store) ReadAt(id, version uint64, p []byte, off int64) (int, error) { + panic("TODO: find, validate, and slice the requested blob") +} + +func (s *store) Delete(id, version uint64) error { + panic("TODO: mark one version deleted") +} + +func (s *store) Compact() error { + panic("TODO: reclaim space without changing visible bytes") +} + +func (s *store) Close() error { + panic("TODO: release the store") +} + +func (s *store) IndexBytes() uint64 { + panic("TODO: report bounded index memory") +} diff --git a/challenges/deadline-scheduler-easy/arena.yaml b/challenges/deadline-scheduler-easy/arena.yaml index d5d2be5..03a9223 100644 --- a/challenges/deadline-scheduler-easy/arena.yaml +++ b/challenges/deadline-scheduler-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: deadline-scheduler-easy - version: 1.0.0 + version: 1.0.1 title: Deadline Scheduler visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.deadlinescheduler.Scheduler/v1 build: command: ["go", "test", "./challenges/deadline-scheduler-easy/..."] diff --git a/challenges/deadline-scheduler-easy/learner/starter.go b/challenges/deadline-scheduler-easy/learner/starter.go new file mode 100644 index 0000000..b95c019 --- /dev/null +++ b/challenges/deadline-scheduler-easy/learner/starter.go @@ -0,0 +1,28 @@ +// Package starter is the learner-facing scaffold for a one-worker scheduler. +package starter + +import "github.com/roughstack/challenges/challenges/deadline-scheduler-easy/contract" + +type scheduler struct { + // TODO: track waiting and running jobs. +} + +func NewScheduler(cluster contract.Cluster) contract.Scheduler { + return &scheduler{} +} + +func (s *scheduler) OnArrival(job contract.Job, nowTick uint64) []contract.Action { + panic("TODO: record the arrival and dispatch only when idle") +} + +func (s *scheduler) OnProgress(progress contract.Progress, nowTick uint64) []contract.Action { + panic("TODO: reflect the worker's progress") +} + +func (s *scheduler) OnComplete(completion contract.Completion, nowTick uint64) []contract.Action { + panic("TODO: complete the job and decide what runs next") +} + +func (s *scheduler) StateBytes() uint64 { + panic("TODO: account for scheduler state") +} diff --git a/challenges/durable-queue-easy/arena.yaml b/challenges/durable-queue-easy/arena.yaml index 2abfdc1..6aafdc1 100644 --- a/challenges/durable-queue-easy/arena.yaml +++ b/challenges/durable-queue-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: durable-queue-easy - version: 1.0.0 + version: 1.0.1 title: Durable Work Queue visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/queue.go + starter: learner/starter.go interface: bytearena.durablequeue.Queue/v1 build: command: ["go", "test", "./challenges/durable-queue-easy/..."] diff --git a/challenges/durable-queue-easy/learner/starter.go b/challenges/durable-queue-easy/learner/starter.go new file mode 100644 index 0000000..0fe96d4 --- /dev/null +++ b/challenges/durable-queue-easy/learner/starter.go @@ -0,0 +1,37 @@ +// Package starter is the learner-facing scaffold for a lease-based queue. +package starter + +import ( + "context" + + "github.com/roughstack/challenges/challenges/durable-queue-easy/contract" +) + +type queue struct { + capacity uint64 + // TODO: protect queued, leased, and closed state under concurrency. +} + +func OpenQueue(_ string, capacityBytes uint64) (contract.Queue, error) { + return &queue{capacity: capacityBytes}, nil +} + +func (q *queue) Enqueue(ctx context.Context, job contract.Job) error { + panic("TODO: copy the payload and apply capacity backpressure") +} + +func (q *queue) Lease(ctx context.Context, nowTick, visibility uint64) (contract.Lease, error) { + panic("TODO: issue a lease or wait for work") +} + +func (q *queue) Ack(ctx context.Context, token uint64) error { + panic("TODO: permanently remove the leased job") +} + +func (q *queue) Nack(ctx context.Context, token uint64) error { + panic("TODO: make the job visible again") +} + +func (q *queue) Close() error { + panic("TODO: wake waiters and close idempotently") +} diff --git a/challenges/fair-gate-easy/arena.yaml b/challenges/fair-gate-easy/arena.yaml index 87e01bd..57ca7c8 100644 --- a/challenges/fair-gate-easy/arena.yaml +++ b/challenges/fair-gate-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: fair-gate-easy - version: 1.0.0 + version: 1.0.1 title: Fair Gate visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.fairgate.Limiter/v1 build: command: ["go", "test", "./challenges/fair-gate-easy/..."] diff --git a/challenges/fair-gate-easy/learner/starter.go b/challenges/fair-gate-easy/learner/starter.go new file mode 100644 index 0000000..c41859d --- /dev/null +++ b/challenges/fair-gate-easy/learner/starter.go @@ -0,0 +1,17 @@ +// Package starter is the learner-facing scaffold for a token-bucket gate. +package starter + +import "github.com/roughstack/challenges/challenges/fair-gate-easy/contract" + +type limiter struct { + cfg contract.Config + // TODO: track fractional tokens and the most recent tick. +} + +func NewLimiter(cfg contract.Config) contract.Limiter { + return &limiter{cfg: cfg} +} + +func (l *limiter) Decide(req contract.Request) contract.Decision { + panic("TODO: refill, charge once, and return the exact retry tick") +} diff --git a/challenges/journal-recovery-easy/arena.yaml b/challenges/journal-recovery-easy/arena.yaml index 842cb72..0437e8a 100644 --- a/challenges/journal-recovery-easy/arena.yaml +++ b/challenges/journal-recovery-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: journal-recovery-easy - version: 1.0.0 + version: 1.0.1 title: Journal Recovery visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/journal.go + starter: learner/starter.go interface: bytearena.journal.Journal/v1 build: command: ["go", "test", "./challenges/journal-recovery-easy/..."] diff --git a/challenges/journal-recovery-easy/learner/starter.go b/challenges/journal-recovery-easy/learner/starter.go new file mode 100644 index 0000000..2d18028 --- /dev/null +++ b/challenges/journal-recovery-easy/learner/starter.go @@ -0,0 +1,29 @@ +// Package starter is the learner-facing scaffold for a crash-safe journal. +package starter + +import "github.com/roughstack/challenges/challenges/journal-recovery-easy/contract" + +type journal struct { + device contract.Device + // TODO: track the next LSN, durable prefix, and closed state. +} + +func OpenJournal(_ string, device contract.Device) (contract.Journal, error) { + return &journal{device: device}, nil +} + +func (j *journal) Append(record contract.Record) (uint64, error) { + panic("TODO: encode a bounded, checksummed frame") +} + +func (j *journal) Sync(upto uint64) error { + panic("TODO: persist a known prefix") +} + +func (j *journal) Recover(apply func(contract.Record) error) (contract.RecoveryInfo, error) { + panic("TODO: replay the valid durable prefix") +} + +func (j *journal) Close() error { + panic("TODO: close idempotently") +} diff --git a/challenges/lossy-link-easy/arena.yaml b/challenges/lossy-link-easy/arena.yaml index aa7d0d1..78cdc70 100644 --- a/challenges/lossy-link-easy/arena.yaml +++ b/challenges/lossy-link-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: lossy-link-easy - version: 1.0.0 + version: 1.0.1 title: Lossy Link visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.lossylink.Endpoint/v1 build: command: ["go", "test", "./challenges/lossy-link-easy/..."] diff --git a/challenges/lossy-link-easy/learner/starter.go b/challenges/lossy-link-easy/learner/starter.go new file mode 100644 index 0000000..a2d059b --- /dev/null +++ b/challenges/lossy-link-easy/learner/starter.go @@ -0,0 +1,47 @@ +// Package starter is the learner-facing scaffold for a reliable byte stream. +package starter + +import "github.com/roughstack/challenges/challenges/lossy-link-easy/contract" + +type sender struct { + cfg contract.Config + // TODO: track bytes in flight, acknowledgements, and timers. +} + +type receiver struct { + cfg contract.Config + // TODO: track the next expected sequence and delivered bytes. +} + +func NewSender(cfg contract.Config) contract.Endpoint { return &sender{cfg: cfg} } +func NewReceiver(cfg contract.Config) contract.Endpoint { return &receiver{cfg: cfg} } + +func (s *sender) OnAppData(data []byte, eof bool) []contract.Packet { + panic("TODO: packetize and send") +} +func (s *sender) OnPacket(pkt contract.Packet, nowTick uint64) []contract.Packet { + panic("TODO: process valid acknowledgements") +} +func (s *sender) OnTimer(timerID, nowTick uint64) []contract.Packet { + panic("TODO: retransmit the outstanding packet") +} +func (s *sender) ReadDelivered(max int) []byte { panic("TODO: sender delivers no application bytes") } +func (s *sender) NextTimers() []contract.Timer { panic("TODO: expose active timers") } +func (s *sender) StateBytes() uint64 { panic("TODO: report buffered state") } +func (s *sender) Done() bool { panic("TODO: report completion only after acknowledgement") } + +func (r *receiver) OnAppData(data []byte, eof bool) []contract.Packet { + panic("TODO: receiver accepts no outgoing app data") +} +func (r *receiver) OnPacket(pkt contract.Packet, nowTick uint64) []contract.Packet { + panic("TODO: validate, deliver in order, and acknowledge") +} +func (r *receiver) OnTimer(timerID, nowTick uint64) []contract.Packet { + panic("TODO: receiver timer behavior") +} +func (r *receiver) ReadDelivered(max int) []byte { + panic("TODO: drain only the requested delivered bytes") +} +func (r *receiver) NextTimers() []contract.Timer { panic("TODO: receiver timer behavior") } +func (r *receiver) StateBytes() uint64 { panic("TODO: report buffered state") } +func (r *receiver) Done() bool { panic("TODO: report end-of-stream delivery") } diff --git a/challenges/lsm-compactor-easy/arena.yaml b/challenges/lsm-compactor-easy/arena.yaml index 11adc47..f492d58 100644 --- a/challenges/lsm-compactor-easy/arena.yaml +++ b/challenges/lsm-compactor-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: lsm-compactor-easy - version: 1.0.0 + version: 1.0.1 title: LSM Compactor visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.lsmcompactor.Policy/v1 build: command: ["go", "test", "./challenges/lsm-compactor-easy/..."] diff --git a/challenges/lsm-compactor-easy/learner/starter.go b/challenges/lsm-compactor-easy/learner/starter.go new file mode 100644 index 0000000..c885b14 --- /dev/null +++ b/challenges/lsm-compactor-easy/learner/starter.go @@ -0,0 +1,16 @@ +// Package starter is the learner-facing scaffold for an LSM merge policy. +package starter + +import "github.com/roughstack/challenges/challenges/lsm-compactor-easy/contract" + +type policy struct { + cfg contract.Config +} + +func NewPolicy(cfg contract.Config) contract.Policy { + return &policy{cfg: cfg} +} + +func (p *policy) Pick(state contract.State) (contract.Plan, bool) { + panic("TODO: choose a legal overlap-closed compaction plan") +} diff --git a/challenges/stream-windows-easy/arena.yaml b/challenges/stream-windows-easy/arena.yaml index 7b2f99e..26d9b1c 100644 --- a/challenges/stream-windows-easy/arena.yaml +++ b/challenges/stream-windows-easy/arena.yaml @@ -2,7 +2,7 @@ apiVersion: bytearena.dev/v1 kind: Arena metadata: id: stream-windows-easy - version: 1.0.0 + version: 1.0.1 title: Stream Windows visibility: public mode: practice @@ -11,6 +11,7 @@ metadata: submission: language: go entrypoint: starter/starter.go + starter: learner/starter.go interface: bytearena.streamwindows.Operator/v1 build: command: ["go", "test", "./challenges/stream-windows-easy/..."] diff --git a/challenges/stream-windows-easy/learner/starter.go b/challenges/stream-windows-easy/learner/starter.go new file mode 100644 index 0000000..41f162f --- /dev/null +++ b/challenges/stream-windows-easy/learner/starter.go @@ -0,0 +1,29 @@ +// Package starter is the learner-facing scaffold for event-time windows. +package starter + +import "github.com/roughstack/challenges/challenges/stream-windows-easy/contract" + +type operator struct { + cfg contract.Config + // TODO: maintain per-key, per-window aggregates and watermark state. +} + +func NewOperator(cfg contract.Config) contract.Operator { + return &operator{cfg: cfg} +} + +func (o *operator) OnEvent(event contract.Event) []contract.Output { + panic("TODO: assign this event to its half-open window") +} + +func (o *operator) OnWatermark(tick uint64) []contract.Output { + panic("TODO: finalize exactly the windows behind the watermark") +} + +func (o *operator) Close() []contract.Output { + panic("TODO: finalize every remaining window once") +} + +func (o *operator) StateBytes() uint64 { + panic("TODO: report live aggregate state") +} From dfb8fae0497ba0c4db7c066e4f3bc424799bf56d Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:08:46 +0530 Subject: [PATCH 19/22] docs: clarify scaffold and baseline quality gates --- README.md | 16 ++++++++++++---- docs/challenge-audit-2026-10-15.md | 17 +++++++++++------ docs/challenge-quality.md | 7 ++++--- 3 files changed, 27 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index d1d0629..a236dd1 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,8 @@ This repository contains independently versioned, public systems-engineering challenges for the Rough Stack platform. Each challenge owns its scenario, contestant -interface, starter, deterministic simulator, public tests, smoke benchmark, -baseline, and public metric contract. +interface, learner scaffold, deterministic simulator, public tests, smoke +benchmark, public baseline, and metric contract. The Rough Stack platform is maintained separately. Challenge packages depend on versioned contracts; the platform consumes validated, immutable challenge bundles @@ -13,8 +13,16 @@ The current manifest filename and protocol fields retain the `arena` name for compatibility with the pre-release platform. They will change only through a versioned contract migration. -Official workloads, hidden cases, exploit tests, reference implementations, and -calibration data are intentionally absent from this repository. +Official workloads, hidden cases, exploit tests, private reference +implementations, and calibration data are intentionally absent from this +repository. Public baseline implementations remain visible for reproducibility. + +Each package's manifest names two different source files. `submission.starter` +is the incomplete scaffold shown in the browser. `submission.entrypoint` is +the file replaced by submitted source inside a temporary execution workspace; +the repository copy is also the tested public baseline. Keeping these separate +lets learners start with a genuine problem without removing the baseline used +for public smoke and feasibility checks. ## Verification diff --git a/docs/challenge-audit-2026-10-15.md b/docs/challenge-audit-2026-10-15.md index a3dcdad..26ea9f1 100644 --- a/docs/challenge-audit-2026-10-15.md +++ b/docs/challenge-audit-2026-10-15.md @@ -6,9 +6,9 @@ lists the adversarial and boundary tests each challenge should ultimately own. ## Evidence checked -- All ten Go starters passed their public smoke workloads at seeds `12345` and +- All ten public Go baselines passed their smoke workloads at seeds `12345` and `987654321`. The full Go suite passed with the race detector. -- All ten starters were submitted through the web platform's production build +- All ten baselines were submitted through the web platform's production build and completed with passing verdicts in an isolated local database. This caught a platform result-schema mismatch: Journal Recovery and Lossy Link include the optional `run.fault_profile` field, which the platform now accepts. @@ -24,18 +24,23 @@ lists the adversarial and boundary tests each challenge should ultimately own. matching challenge packages. Registry-only mode can display previously materialized database content, but cannot refresh statements or execute a package that is unavailable locally. +- Follow-up visual review found that the browser was prefilled with the passing + baseline. The manifests now declare a separate `submission.starter`, and + all ten packages provide incomplete learner scaffolds while retaining the + public baseline at `submission.entrypoint`. The scaffold path is validated + separately from the file overwritten during a run. ## Challenge-by-challenge conclusion | Challenge | Does the problem have a coherent solution? | Public evidence and next gap | | --- | --- | --- | -| Cache Pressure, easy | Yes. Exact LRU behavior is prescribed; implementations need not share a data structure. | Starter and contract probe pass. Add a small independent model across randomized operation sequences and a candidate that lies about `UsedBytes` while retaining data. | -| Cache Pressure, medium | Yes. Many admission/eviction policies are legal; the public workload now requires at least 40% request hits. | Starter passes the floor across the checked seeds; never-admit fails. Add a model-based TTL/accounting probe independent of the generated workload, especially replacement at an exact expiry tick. | +| Cache Pressure, easy | Yes. Exact LRU behavior is prescribed; implementations need not share a data structure. | Baseline and contract probe pass. Add a small independent model across randomized operation sequences and a candidate that lies about `UsedBytes` while retaining data. | +| Cache Pressure, medium | Yes. Many admission/eviction policies are legal; the public workload now requires at least 40% request hits. | Baseline passes the floor across the checked seeds; never-admit fails. Add a model-based TTL/accounting probe independent of the generated workload, especially replacement at an exact expiry tick. | | Chunk Store | Yes. Bytes returned and frame-local integrity are definite even though internal indexing can differ. | Corrupt length, checksum, over-allocation, unrelated reads, and reclaim tests exist. Expand every-tail-truncation and cross-block read cases under a tiny independent frame oracle. | | Deadline Scheduler | Yes. Many fair, non-preemptive schedules are valid; no single order is mandated. | Arrival, ties, aging, idle gaps, failure gating, and starter tests pass. Add exhaustive short-job schedule comparison to validate flow-time accounting and starvation bounds. | | Durable Work Queue | Yes. The visible lease/ack/nack behavior is definite; lock and storage choices are not. | Correctness, panic, nil, and watchdog tests pass. Add a concurrent history checker for no loss or duplication and more exact visibility-deadline ties. | | Fair Gate | Yes. Given a token bucket config and request trace, allow/deny and retry tick are mathematically determined. | Tests reject over-admission, over-charging, and wrong retry ticks. Add an independent rational-arithmetic oracle for randomly generated short traces and overflow boundaries. | -| Journal Recovery | Yes. Recovery must return the valid durable prefix, regardless of internal data structure. | Torn tails, LSN gaps/duplicates, callback errors, corruption, panic, and nil cases pass. Add every possible truncation position for short encoded logs and interior-versus-tail fault pairs. | +| Journal Recovery | Yes. Recovery must return the valid durable prefix, regardless of internal data structure. | Every byte-tail truncation of a short encoded log, plus LSN gaps/duplicates, callback errors, corruption, panic, and nil cases pass. Add more interior-versus-tail fault pairs and an independent prefix oracle. | | Lossy Link | Yes. Delivered bytes are fixed; retransmission schedule and packet count can vary. | Loss, corruption, duplicate, stale ACK, timer, wrap, and boundary tests pass. Add a bounded fault-sequence enumerator to check safety and liveness against many combined faults. | | LSM Compactor | Yes. Several legal plans exist; reads after merging must preserve newest-version and tombstone semantics. | Invalid overlap closure, stale/duplicate plans, tombstones, empty runs, and tie-breaking pass. Add an independent key-version map oracle over enumerated small run graphs. | | Stream Windows | Yes. Half-open window assignment and finalized sums are unique for a legal event trace. | Boundary, regression, duplicate/premature output, negative sum, close, and max-tick tests pass. Add randomized per-key map oracle and repeated watermark/event interleavings. | @@ -46,7 +51,7 @@ lists the adversarial and boundary tests each challenge should ultimately own. ## Publication limits All ten generated public smoke results currently report `score: 0`, including -correct starters. Their weighted scoring files are design intent, not an active +correct baselines. Their weighted scoring files are design intent, not an active ranking system. Some displayed measurements, notably fair-gate throughput and arithmetic work and stream-window event throughput/output delay, are fixed by the public trace once the answer is correct; they should be labeled as trace diff --git a/docs/challenge-quality.md b/docs/challenge-quality.md index 16d996f..8fc7178 100644 --- a/docs/challenge-quality.md +++ b/docs/challenge-quality.md @@ -3,8 +3,9 @@ Every challenge needs three independent forms of evidence: a valid implementation passes, a deliberately invalid implementation fails for the right reason, and the same seed produces the same logical result. Unit coverage alone cannot prove -these properties. The public starter is a feasibility witness, not an optimum -or a substitute for adversarial testing. +these properties. The public baseline is a feasibility witness, not an optimum +or a substitute for adversarial testing. The learner-facing starter must be an +incomplete, compilable scaffold, never that passing baseline. ## Shared acceptance checks @@ -15,7 +16,7 @@ or a substitute for adversarial testing. contract defines their behavior. 3. Compare each small generated case with an independent, simple oracle. Vary seeds and sizes; avoid reusing the contestant implementation as the oracle. -4. Run a known-correct starter and at least one targeted broken implementation +4. Run a known-correct baseline and at least one targeted broken implementation through the public harness. Assert the verdict and the relevant violation. 5. Repeat a seed and compare logical metrics and output. Measure runtime-derived allocation metrics separately; they are not deterministic logical counters. From 61dfb9d57f2a60686bac9bfdcac0e9248b8995be Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:13:19 +0530 Subject: [PATCH 20/22] fix: align public harness protocol versions with manifests --- challenges/cache-pressure-easy/harness/harness.go | 2 +- challenges/cache-pressure-medium/harness/harness.go | 2 +- challenges/chunk-store-easy/harness/harness.go | 2 +- challenges/deadline-scheduler-easy/harness/harness.go | 2 +- challenges/durable-queue-easy/harness/harness.go | 2 +- challenges/fair-gate-easy/harness/harness.go | 2 +- challenges/journal-recovery-easy/harness/harness.go | 2 +- challenges/lossy-link-easy/harness/harness.go | 2 +- challenges/lsm-compactor-easy/harness/harness.go | 2 +- challenges/stream-windows-easy/harness/harness.go | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) diff --git a/challenges/cache-pressure-easy/harness/harness.go b/challenges/cache-pressure-easy/harness/harness.go index 539d93e..bf36e47 100644 --- a/challenges/cache-pressure-easy/harness/harness.go +++ b/challenges/cache-pressure-easy/harness/harness.go @@ -12,7 +12,7 @@ import ( const ( ArenaID = "cache-pressure-easy" - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" WorkloadID = "public-smoke-v1" valueBytes = 96 ) diff --git a/challenges/cache-pressure-medium/harness/harness.go b/challenges/cache-pressure-medium/harness/harness.go index e5d5def..04e838d 100644 --- a/challenges/cache-pressure-medium/harness/harness.go +++ b/challenges/cache-pressure-medium/harness/harness.go @@ -11,7 +11,7 @@ import ( const ( ArenaID = "cache-pressure-medium" - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" WorkloadID = "public-smoke-v1" ) diff --git a/challenges/chunk-store-easy/harness/harness.go b/challenges/chunk-store-easy/harness/harness.go index fabf264..b2c8197 100644 --- a/challenges/chunk-store-easy/harness/harness.go +++ b/challenges/chunk-store-easy/harness/harness.go @@ -20,7 +20,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "chunk-store-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" diff --git a/challenges/deadline-scheduler-easy/harness/harness.go b/challenges/deadline-scheduler-easy/harness/harness.go index 6889e06..8e480c7 100644 --- a/challenges/deadline-scheduler-easy/harness/harness.go +++ b/challenges/deadline-scheduler-easy/harness/harness.go @@ -19,7 +19,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "deadline-scheduler-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" // FixtureWorkloadID identifies explicit job fixtures used by public tests. diff --git a/challenges/durable-queue-easy/harness/harness.go b/challenges/durable-queue-easy/harness/harness.go index b1e597c..8a3eda6 100644 --- a/challenges/durable-queue-easy/harness/harness.go +++ b/challenges/durable-queue-easy/harness/harness.go @@ -20,7 +20,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "durable-queue-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" ) diff --git a/challenges/fair-gate-easy/harness/harness.go b/challenges/fair-gate-easy/harness/harness.go index e8a340f..4b65031 100644 --- a/challenges/fair-gate-easy/harness/harness.go +++ b/challenges/fair-gate-easy/harness/harness.go @@ -17,7 +17,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "fair-gate-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" // FixtureWorkloadID identifies explicit request fixtures used by tests. diff --git a/challenges/journal-recovery-easy/harness/harness.go b/challenges/journal-recovery-easy/harness/harness.go index cbfcf94..801c3e8 100644 --- a/challenges/journal-recovery-easy/harness/harness.go +++ b/challenges/journal-recovery-easy/harness/harness.go @@ -19,7 +19,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "journal-recovery-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" ) diff --git a/challenges/lossy-link-easy/harness/harness.go b/challenges/lossy-link-easy/harness/harness.go index cb24c76..1634f6e 100644 --- a/challenges/lossy-link-easy/harness/harness.go +++ b/challenges/lossy-link-easy/harness/harness.go @@ -14,7 +14,7 @@ import ( const ( ArenaID = "lossy-link-easy" - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" WorkloadID = "public-smoke-v1" // maxTicks bounds the virtual event loop. Reaching it without completing diff --git a/challenges/lsm-compactor-easy/harness/harness.go b/challenges/lsm-compactor-easy/harness/harness.go index c83e359..64f0a4b 100644 --- a/challenges/lsm-compactor-easy/harness/harness.go +++ b/challenges/lsm-compactor-easy/harness/harness.go @@ -16,7 +16,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "lsm-compactor-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" diff --git a/challenges/stream-windows-easy/harness/harness.go b/challenges/stream-windows-easy/harness/harness.go index 0f8b4d5..6bce19b 100644 --- a/challenges/stream-windows-easy/harness/harness.go +++ b/challenges/stream-windows-easy/harness/harness.go @@ -18,7 +18,7 @@ const ( // ArenaID is the stable public identifier for this variant. ArenaID = "stream-windows-easy" // ArenaVersion is the immutable public version for this variant. - ArenaVersion = "1.0.0" + ArenaVersion = "1.0.1" // WorkloadID identifies the deterministic public smoke workload. WorkloadID = "public-smoke-v1" // ticksPerLogicalSecond converts the injected virtual tick into a logical From 9eba17f4480b0323dfe0e5a901a2aab59e7ec41b Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:14:18 +0530 Subject: [PATCH 21/22] docs: record scaffold and protocol submission checks --- docs/challenge-audit-2026-10-15.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/challenge-audit-2026-10-15.md b/docs/challenge-audit-2026-10-15.md index 26ea9f1..4bb2837 100644 --- a/docs/challenge-audit-2026-10-15.md +++ b/docs/challenge-audit-2026-10-15.md @@ -29,6 +29,13 @@ lists the adversarial and boundary tests each challenge should ultimately own. all ten packages provide incomplete learner scaffolds while retaining the public baseline at `submission.entrypoint`. The scaffold path is validated separately from the file overwritten during a run. +- Version `1.0.1` of every harness now matches its manifest after the scaffold + release. A real submission of the easy-cache public baseline at this version + passed through the platform execution API; submitting the untouched learner + scaffold failed its public tests as expected. The platform's arena validator + checks Go harness and manifest versions for future drift. Browser checks + confirmed that all ten practice editors display unfinished code, and the + expanded visual and responsive suites passed. ## Challenge-by-challenge conclusion From a6c61a9cf247c91d76f0510e3714bae92a4e76a7 Mon Sep 17 00:00:00 2001 From: Omkar Tripathi <38633467+triggeredcode@users.noreply.github.com> Date: Sat, 17 Oct 2026 00:49:40 +0530 Subject: [PATCH 22/22] docs: describe current local challenge integration --- README.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index a236dd1..1079328 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,9 @@ interface, learner scaffold, deterministic simulator, public tests, smoke benchmark, public baseline, and metric contract. The Rough Stack platform is maintained separately. Challenge packages depend on -versioned contracts; the platform consumes validated, immutable challenge bundles -and does not import challenge source code. +versioned contracts. Today, a local platform checkout validates and reads a +separate challenge checkout from disk. Published immutable bundles are a future +distribution format, not a requirement for running or contributing locally. The current manifest filename and protocol fields retain the `arena` name for compatibility with the pre-release platform. They will change only through a @@ -38,11 +39,15 @@ Changed manifests must also pass the Rough Stack platform validator. Every smoke benchmark must emit one valid result object and reproduce the same canonical result when repeated with the same seed. -## Initial build order +## Local platform integration -The first public implementation slice is `cache-pressure-easy`. The remaining -families are added one at a time after the preceding challenge passes tests, -determinism checks, and the platform manifest validator. +Clone this repository beside `roughstack/web-platform` and follow the +[web-platform local setup](https://github.com/roughstack/web-platform#running-it). +The platform's `BYTEARENA_ARENA_ROOTS` setting should point to this repository's +`challenges/` directory. Its opt-in real-run check submits the easy-cache public +baseline and learner scaffold to the local Docker runner. Public baselines are +practice feasibility witnesses; the generated arenas do not yet provide ranked +scores or multi-language submissions. ## Contributing and security