From 3cb62ad4bd97a4e8d6a216480d7fa69bb06760a4 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 3 Oct 2026 03:45:46 +0800 Subject: [PATCH] The record of the round: what a context stands on, and the two defects it exposed The design and the execution plan are already in this tree; this is what happened. It carries the ecosystem verification script beside it --- run inside an xlings sandbox against the published index with the CN mirror selected --- and the transcript of that run, which is 0 assertions failed. The round delivered one declaration and five implementations that answer it, and it exposed two defects in the implementation beneath: 1. The thread-local image was copied eight bytes below the variables that name it, because the segment's alignment was clamped to sixteen where the block's SIZE is computed. Found by openkal-llvm-runtime's own probe against the released packages, not by anything here. 2. A context in the hosted arrangement was given storage laid out from an undescribed image, because only this implementation's own entry point ever described it. Both are repaired, the repair is released (0.16.1, 0.20.1, 0.15.4), and the record states what the round could not measure as plainly as what it did. --- ...-10-03-stack-bounds-of-a-context-record.md | 183 +++++++++++++ ...-10-03-stack-bounds-of-a-context-verify.sh | 246 ++++++++++++++++++ 2 files changed, 429 insertions(+) create mode 100644 .agents/docs/2026-10-03-stack-bounds-of-a-context-record.md create mode 100644 .agents/docs/2026-10-03-stack-bounds-of-a-context-verify.sh diff --git a/.agents/docs/2026-10-03-stack-bounds-of-a-context-record.md b/.agents/docs/2026-10-03-stack-bounds-of-a-context-record.md new file mode 100644 index 0000000..d0697dc --- /dev/null +++ b/.agents/docs/2026-10-03-stack-bounds-of-a-context-record.md @@ -0,0 +1,183 @@ +# Stack bounds of a context — record of the round + +Date: 2026-10-03. The design is `2026-10-02-stack-bounds-of-a-context-design.md`, +the task breakdown and the decisions are +`2026-10-02-stack-bounds-of-a-context-execution-plan.md`, and the ecosystem +verification against the published index is +`2026-10-03-stack-bounds-of-a-context-verify.sh` beside this file. + +What this round delivered: one declaration (`kal_task_stack`), its +specification, its observation in the conformance suite, five implementations +that answer it, the C library port that uses it in place of a refusal, and the +release, index and sandbox verification of the whole set. + +--- + +## 1. The releases + +| Repository | Version | Pull request | What it carries | +| --- | --- | --- | --- | +| `openkal` | 0.15.0 | #45 | the declaration, clause 11 entry 21, `SURFACE.txt`, the suite's observation | +| `openkal-linux` | 0.16.0 | #32 | the measured region; the TLS-image defect it exposed, fixed | +| `openkal-macos` | 0.13.0 | #26 | `_pthread_self` and the two `_np` enquiries; three names into `port/libSystem.tbd` | +| `openkal-windows` | 0.11.0 | #30 | `GetCurrentThreadStackLimits`, declared and exported | +| `openkal-emscripten` | 0.4.0 | #5 | the platform's per-context pair | +| `openkal-opensbi` | 0.8.2 | #19 | the pin, and nothing else | +| `openkal-musl` | 0.20.0 | #50 | the real answer for the calling thread, the refusal for another, the withdrawn absent row | +| `openkal-linux` | 0.16.1 | #33 | the thread-local image placed where the linker measured it (a repair; §3.1) | +| `openkal-musl` | 0.20.1 | #51 | the pin that reaches the repair | +| `openkal-llvm-runtime` | 0.15.4 | #34 | the pin that reaches the repaired C library | + +Every branch was named `openkal-0.15.0` in every repository, which is what makes +each repository's continuous integration test the other halves as written rather +than as published: the specification clones the implementation at that branch, +and the implementation clones the specification at it. + +## 2. What was measured, and where + +| Claim | Instrument | Result | +| --- | --- | --- | +| the region the kernel stops the stack at is `mapping_end - RLIMIT_STACK` | a descending write in a probe on 6.8.0 | exact: the floor writable, one page below it not, one page above it writable | +| the region below is bounded by one guard gap above the nearest mapping | the same probe with a page mapped by itself inside the reservation | exact: 256 pages above the planted page | +| the same, on the released implementation | the conformance suite's five observations, in `openkal-linux`'s own tests, and end to end through `openkal-musl` | held | +| the macOS names resolve | a cross link of a program that calls the enquiry, aarch64 and x86_64 | no undefined symbols; the four imports are exactly the ones the code names | +| the Windows name resolves | the declared-vs-exported gate, the generated `libkernel32.a`, and a cross build for `x86_64-windows-gnu` | pass; `__imp_GetCurrentThreadStackLimits` present | +| the WebAssembly pair answers per context | the task gate's program under node | exits 0; both contexts are told a region containing themselves | +| the C library answers for the caller and refuses another thread | `examples/stack-bounds` above `openkal-musl` 0.20.0 | first and started contained, another thread refused | + +## 3. The two defects the round exposed, and their fixes + +### 3.1 The thread-local image was placed eight bytes below the variables + +**This is the one that mattered, and 0.16.0 shipped it.** Every thread-local +address is `tp + st_value - tls_size`, where `tls_size` is the block the LINKER +laid out: `p_memsz` rounded up to the segment's alignment. `describe_tls` +clamped that alignment up to sixteen *before recording it*, so the clamp reached +the SIZE, which must not have it. Measured on a segment stating `p_align = 8, +p_memsz = 56`: the linker put the variables at `tp - 56` and the region was +built 64 bytes deep, so the image of the program's thread-local storage sat +eight bytes below the variables that name it. + +From above: two thread-local variables declared with different values read each +other's bytes, and a C++ program's `thread_local` object can find its guard byte +non-zero and never run its constructor at all. That is how it was found — not by +this round's own tests, which passed, but by `openkal-llvm-runtime`'s probe +against the released packages, on the runner: + +``` +FAIL: a thread_local is constructed in a spawned thread +FAIL: and its destructor runs when that thread ends +``` + +**Why it surfaced now, stated rather than glossed.** The clamp is as old as the +file. This implementation's own thread-local storage had been one four-byte +variable, which sits at the very end of the segment and was copied correctly by +luck; the twenty-four bytes 0.16.0 added moved the image far enough to be read +by the wrong variable. The kit test therefore passed throughout, and the test +0.16.0 added for exactly this area — `tests/conformance_task_tls.cpp`, with a +non-zero initialiser and a four-kilobyte block — did not catch it either, +because the alignment its own link happened to produce was sixteen. What did +catch it was a C++ `thread_local` object with a destructor, in another +repository's continuous integration, against the published packages. + +The repair: `describe_tls` keeps the segment's alignment as the loader stated +it; `make_tls` rounds the size by that and asks the ALLOCATOR for sixteen. +Verified against a probe of two initialised thread-local variables — in the +context the program was started on and in a started one — by the kit, by this +package's tests, by the conformance suite (197 held, 0 did not hold), and by the +runtime's own example, which now reports `-- failures: 0 --`. + +### 3.2 A context's storage was laid out from an undescribed image + +`openkal-linux` described its own thread-local image only from the entry point it +supplies, and a program that carries a runtime never runs that entry point. A +context was therefore given a storage region laid out from a segment of size +zero: its thread pointer sat at the START of the region rather than at its end, +and every thread-local variable of that context was addressed below the +allocation — into whatever the allocator kept beside it. It had been invisible +while this implementation's own storage was one four-byte variable, and it became +fatal when the storage for this round's operation made it twenty-four: the +specification package's own kit test died at the first instruction of a context. + +`make_tls` now describes the image if nothing has, from the vectors `env.cpp` +records in either arrangement, and refuses to lay out a region it cannot size +rather than laying out a wrong one. + +## 4. What this round could not measure + +- **macOS and Windows at run time.** Their mechanisms are exercised by the + conformance suite on their own runners; what was done here is the cross link + and the symbol gates. `GetCurrentThreadStackLimits`' reservation-versus-commit + question remains unpublished by the vendor and is stated at the declaration. +- **A machine with firmware and no operating system.** `openkal-opensbi` provides + no `openkal.task`, and the round's change to it is one version pin. +- **The `stack_guard_gap` a kernel is booted with.** The default is applied; a + kernel configured with a larger gap stops the stack above the base reported, + which is the direction a caller must not be wrong in, and it is stated at the + code. + +## 5. The verification transcript + +From `2026-10-03-stack-bounds-of-a-context-verify.sh`, run inside an `xlings` +sandbox (`xlings subos use okl015 --sandbox --cmd ...`) against the published +index with the CN mirror selected, using the released engine: + +``` +▸ entering subos okl015 (exit to leave) + +== A. identity and mirror == +ok: mcpp 2026.10.1.3 from /home/speak/.xlings/data/xpkgs/xim-x-mcpp/2026.10.1.3/bin/mcpp +ok: xlings mirror is CN + +== B. openkal 0.15.0 and openkal-linux 0.16.1 resolve, and a context is told its own region == +proot warning: ptrace(PEEKDATA): Bad address +ok: version 0.15.0 first e=0 contains=1 size=8376320 +ok: started contains=1 ran=1 +ok: the lock records openkal-linux 0.16.1 + +== C. openkal-musl 0.20.1: the calling thread is told its region, another thread is refused == +ok: musl first e=0 contains=1 size=3280896 +ok: musl started contains=1 size=262144 another thread refused=1 +ok: the lock records openkal-musl 0.20.1 +NOT RUN: macOS, Windows and WebAssembly: their implementations answer on their own runners (openkal-macos#26, openkal-windows#30, openkal-emscripten#5), and the two cross links are made on this machine by their own continuous integration +NOT RUN: a machine with firmware and no operating system: openkal-opensbi 0.8.2 moves a pin and provides no openkal.task + +0 assertion(s) failed +not run: + - macOS, Windows and WebAssembly: their implementations answer on their own runners (openkal-macos#26, openkal-windows#30, openkal-emscripten#5), and the two cross links are made on this machine by their own continuous integration + - a machine with firmware and no operating system: openkal-opensbi 0.8.2 moves a pin and provides no openkal.task +exit=0 +``` + +## 6. Review of the round, at the scale of the ecosystem + +**What is consistent.** One declaration, five implementations answering the same +question in the same shape, one suite observing it once for all of them, and one +C library asking it on behalf of the callers that cannot. Every implementation +that answers was measured answering on its own system: Linux here and in +`openkal-musl`'s example, macOS and Windows on their runners through the +conformance suite, WebAssembly under node in the emscripten gate. + +**What the round changed that was not this feature.** One latent defect in the +hosted arrangement of `openkal-linux` — a context's thread-local storage laid out +from an undiscribed image, which had been silently writing below its own +allocation since the implementation was written. It was found because this +feature made the implementation's own storage large enough to fault. That is the +honest account: the round's own change did not cause it, and the round could not +have been finished without it. + +**What a consumer must know.** A version written without an operator is an exact +pin in mcpp, so a consumer moves by naming the new versions, and any package that +locks `openkal` and can share a graph with the consumer must move too. Within +this round that is `openkal-musl` 0.20.1 and `openkal-llvm-runtime` 0.15.4; a +program that carries the C++ runtime names both, and the runtime's own release +is a consequence of the repair rather than of the feature. + +**What remains open, with the reason.** The macOS and Windows mechanisms cannot +be run from this machine, and their reservation-versus-commit question is +answered by the vendor's documentation on one system and by a reimplementation +on the other; both are stated at the declaration. The `stack_guard_gap` a kernel +is booted with cannot be read by an implementation, so the default is applied and +the direction of the error is stated. And `pthread_getattr_np` still refuses for +a thread that is not the caller: a context can only be asked about itself, which +is the shape the specification chose and not a gap left by the port. diff --git a/.agents/docs/2026-10-03-stack-bounds-of-a-context-verify.sh b/.agents/docs/2026-10-03-stack-bounds-of-a-context-verify.sh new file mode 100644 index 0000000..2aea218 --- /dev/null +++ b/.agents/docs/2026-10-03-stack-bounds-of-a-context-verify.sh @@ -0,0 +1,246 @@ +#!/usr/bin/env bash +# Ecosystem verification for the wave that lets a running context say where it +# stands (openkal 0.15.0, openkal-linux 0.16.1, openkal-macos 0.13.0, +# openkal-windows 0.11.0, openkal-emscripten 0.4.0, openkal-opensbi 0.8.2, +# openkal-musl 0.20.1), resolved from the published index only. +# +# xlings subos new okl015 +# cp ~/.xlings/subos/okl015/tmp/v.sh +# xlings subos use okl015 --sandbox --cmd "MCPP_VERIFY_VERSION= bash /tmp/v.sh" +# +# The script is copied into the sandbox's /tmp rather than passed on its command +# line: under proot a command line of this size was observed to come with +# `ptrace(PEEKDATA): Bad address` and failures of unrelated file operations. +# +# The subject is what a consumer gets by naming the released versions. A version +# written without an operator is an exact pin in mcpp (`is_constraint` in +# modules/versioning/src/version_req.cppm), so each manifest below is what a +# consumer writes. A step that cannot run here says so and is counted as not +# run, never as passed. +set -u + +VER="${MCPP_VERIFY_VERSION:?set MCPP_VERIFY_VERSION}" +STORE="${MCPP_VERIFY_BIN:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp}" +XL="${XLINGS_BIN:-$(command -v xlings)}" + +fails=0; skipped="" +fail() { printf 'ASSERT-FAIL: %s\n' "$1"; fails=$((fails + 1)); } +ok() { printf 'ok: %s\n' "$1"; } +section() { printf '\n== %s ==\n' "$1"; } +skip() { printf 'NOT RUN: %s\n' "$1"; skipped="$skipped + - $1"; } + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +# -- A. identity and mirror --------------------------------------------------- +section "A. identity and mirror" +got=$("$STORE" --version 2>&1 | head -1) +[ "$got" = "mcpp $VER" ] && ok "$got from $STORE" || fail "version is '$got' at $STORE" +"$STORE" self config --mirror CN >/dev/null 2>&1 || true +"$XL" config --mirror CN >/dev/null 2>&1 || true +xm=$(python3 -c "import json,os;print(json.load(open(os.path.expanduser('~/.xlings/.xlings.json'))).get('mirror',''))" 2>/dev/null) +[ "$xm" = "CN" ] && ok "xlings mirror is CN" || fail "xlings mirror is '$xm'" +"$STORE" index update >/dev/null 2>&1 || true + +# -- B. the implementation beneath answers the context that asks -------------- +# The manifest names the specification and one implementation, which is the +# published shape. The program carries no runtime, so the region it asks about +# is the one openkal-linux measured: a running context can always say where it +# stands, which is the whole argument for clause 6.4 admitting the operation. +section "B. openkal 0.15.0 and openkal-linux 0.16.1 resolve, and a context is told its own region" +mkdir -p "$work/ver/src" +cat > "$work/ver/mcpp.toml" <<'TOML' +[package] +name = "stackprobe" +version = "0.1.0" + +[dependencies] +openkal = "0.15.0" + +[target.'cfg(os = "linux")'.dependencies] +openkal-linux = "0.16.1" + +[targets.stackprobe] +kind = "bin" +main = "src/main.c" +TOML +cat > "$work/ver/src/main.c" <<'C' +#include +#include +#include + +static void say(const char* s, kal_uintptr n) { kal_stream_write(kal_stdout(), s, n); } +static void put(const char* s) { kal_uintptr n = 0; while (s[n]) n++; say(s, n); } +static void number(kal_uintptr v) { + char b[24]; int i = 0; + if (v == 0) { put("0"); return; } + while (v) { b[i++] = (char)('0' + (v % 10)); v /= 10; } + char o[24]; int j = 0; + while (i) o[j++] = b[--i]; + o[j] = 0; put(o); +} + +/* The property a caller relies on and not the shape of the numbers: the region + * contains a local of the context that asked. */ +static int holds(void* base, kal_uintptr size, void* here) { + const kal_uintptr b = (kal_uintptr)base; + const kal_uintptr at = (kal_uintptr)here; + return size != 0 && b + size > b && at >= b && at - b < size; +} + +static volatile int g_started = -1; + +static void entry(void* arg) { + char here = 0; + void* base = 0; + kal_uintptr size = 0; + const int e = kal_task_stack(&base, &size); + g_started = (e == kal_ok && holds(base, size, &here)) ? 1 : 0; + *(int*)arg = 1; +} + +int main(void) { + char here = 0; + void* base = 0; + kal_uintptr size = 0; + const int e = kal_task_stack(&base, &size); + + put("version "); + number((kal_uintptr)KAL_VERSION_MAJOR); put("."); + number((kal_uintptr)KAL_VERSION_MINOR); put("."); + number((kal_uintptr)KAL_VERSION_PATCH); + put(" first e="); number((kal_uintptr)e); + put(" contains="); number((kal_uintptr)(e == kal_ok && holds(base, size, &here)) ? 1u : 0u); + put(" size="); number(size); + put("\n"); + + int ran = 0; + struct kal_task t = {0}; + if (kal_task_start(entry, &ran, &t) == kal_ok) { + kal_task_join(t); + put("started contains="); number((kal_uintptr)(g_started == 1) ? 1u : 0u); + put(" ran="); number((kal_uintptr)ran); + put("\n"); + } else { + put("started refused\n"); + } + return 0; +} +C +out=$(cd "$work/ver" && "$STORE" run 2>&1); rc=$? +first=$(printf '%s\n' "$out" | grep -E '^version ' | head -1) +started=$(printf '%s\n' "$out" | grep -E '^started ' | head -1) +if [ $rc -ne 0 ] || [ -z "$first" ]; then + fail "the stack probe did not build or run"; printf '%s\n' "$out" | tail -8 +else + printf '%s\n' "$first" | grep -q '^version 0.15.0 first e=0 contains=1 ' \ + && ok "$first" || fail "the probe reported: $first" + [ "$started" = "started contains=1 ran=1" ] \ + && ok "$started" || fail "the started context reported: $started" +fi +lock=$(cat "$work/ver/mcpp.lock" 2>/dev/null) +printf '%s\n' "$lock" | grep -q '0.16.1' && ok "the lock records openkal-linux 0.16.1" \ + || fail "the lock does not record openkal-linux 0.16.1" + +# -- C. the C library above it answers, and refuses where it must ------------- +# `pthread_getattr_np' was the reason for the change: above openkal it described +# a range that was not the stack. It now answers for the calling thread with the +# region the implementation measured, and refuses for any other thread --- a +# refusal a program can read being the only alternative to a range it cannot +# check. The program is C, so nothing here needs the C++ runtime. +section "C. openkal-musl 0.20.1: the calling thread is told its region, another thread is refused" +mkdir -p "$work/musl/src" +cat > "$work/musl/mcpp.toml" <<'TOML' +[package] +name = "muslstack" +version = "0.1.0" + +[dependencies] +openkal-musl = "0.20.1" + +[targets.muslstack] +kind = "bin" +main = "src/main.c" +TOML +cat > "$work/musl/src/main.c" <<'C' +#define _GNU_SOURCE +#include +#include +#include +#include + +static int holds(void* base, size_t size, void* here) { + const uintptr_t b = (uintptr_t)base; + const uintptr_t at = (uintptr_t)here; + return size != 0 && b + size > b && at >= b && at - b < size; +} + +struct started { int contained; int refused; size_t size; }; + +static void* body(void* arg) { + struct started* s = arg; + char here = 0; + pthread_attr_t a; + int e = pthread_getattr_np(pthread_self(), &a); + void* base = 0; + size_t size = 0; + if (e == 0 && pthread_attr_getstack(&a, &base, &size) != 0) e = -1; + s->contained = (e == 0 && holds(base, size, &here)) ? 1 : 0; + s->size = size; + /* The identifier of another thread, asked from here: the port compares it + * and never reads it, so a refusal is the only thing this can produce. */ + pthread_attr_t other; + s->refused = pthread_getattr_np((pthread_t)(uintptr_t)0x1234, &other) == ENOSYS; + return 0; +} + +int main(void) { + char here = 0; + pthread_attr_t a; + int e = pthread_getattr_np(pthread_self(), &a); + void* base = 0; + size_t size = 0; + if (e == 0 && pthread_attr_getstack(&a, &base, &size) != 0) e = -1; + printf("musl first e=%d contains=%d size=%zu\n", e, + (e == 0 && holds(base, size, &here)) ? 1 : 0, size); + + struct started s = {0, 0, 0}; + pthread_t t; + if (pthread_create(&t, 0, body, &s) != 0 || pthread_join(t, 0) != 0) { + printf("musl started did not run\n"); + return 1; + } + printf("musl started contains=%d size=%zu another thread refused=%d\n", + s.contained, s.size, s.refused); + return 0; +} +C +out=$(cd "$work/musl" && "$STORE" build 2>&1); rc=$? +bin=$(find "$work/musl/target" -type f -name muslstack 2>/dev/null | head -1) +if [ $rc -ne 0 ] || [ -z "$bin" ]; then + fail "the musl probe did not build"; printf '%s\n' "$out" | tail -8 +else + run=$(timeout 120 "$bin" 2>&1); rc=$? + first=$(printf '%s\n' "$run" | grep -E '^musl first ' | head -1) + started=$(printf '%s\n' "$run" | grep -E '^musl started ' | head -1) + if [ $rc -ne 0 ] || [ -z "$first" ]; then + fail "the musl probe did not run"; printf '%s\n' "$run" | tail -8 + else + printf '%s\n' "$first" | grep -q '^musl first e=0 contains=1 ' \ + && ok "$first" || fail "the first context reported: $first" + printf '%s\n' "$started" | grep -q '^musl started contains=1 .*another thread refused=1$' \ + && ok "$started" || fail "the started context reported: $started" + fi + lk=$(cat "$work/musl/mcpp.lock" 2>/dev/null) + printf '%s\n' "$lk" | grep -q '0.20.1' && ok "the lock records openkal-musl 0.20.1" \ + || fail "the lock does not record openkal-musl 0.20.1" +fi + +# -- D. what this sandbox cannot observe -------------------------------------- +skip "macOS, Windows and WebAssembly: their implementations answer on their own runners (openkal-macos#26, openkal-windows#30, openkal-emscripten#5), and the two cross links are made on this machine by their own continuous integration" +skip "a machine with firmware and no operating system: openkal-opensbi 0.8.2 moves a pin and provides no openkal.task" + +printf '\n%s assertion(s) failed\n' "$fails" +[ -n "$skipped" ] && printf 'not run:%s\n' "$skipped" +exit "$fails"