diff --git a/.agents/docs/2026-10-03-stack-bounds-of-a-context-record.md b/.agents/docs/2026-10-03-stack-bounds-of-a-context-record.md new file mode 100644 index 0000000..d0697dc --- /dev/null +++ b/.agents/docs/2026-10-03-stack-bounds-of-a-context-record.md @@ -0,0 +1,183 @@ +# Stack bounds of a context — record of the round + +Date: 2026-10-03. The design is `2026-10-02-stack-bounds-of-a-context-design.md`, +the task breakdown and the decisions are +`2026-10-02-stack-bounds-of-a-context-execution-plan.md`, and the ecosystem +verification against the published index is +`2026-10-03-stack-bounds-of-a-context-verify.sh` beside this file. + +What this round delivered: one declaration (`kal_task_stack`), its +specification, its observation in the conformance suite, five implementations +that answer it, the C library port that uses it in place of a refusal, and the +release, index and sandbox verification of the whole set. + +--- + +## 1. The releases + +| Repository | Version | Pull request | What it carries | +| --- | --- | --- | --- | +| `openkal` | 0.15.0 | #45 | the declaration, clause 11 entry 21, `SURFACE.txt`, the suite's observation | +| `openkal-linux` | 0.16.0 | #32 | the measured region; the TLS-image defect it exposed, fixed | +| `openkal-macos` | 0.13.0 | #26 | `_pthread_self` and the two `_np` enquiries; three names into `port/libSystem.tbd` | +| `openkal-windows` | 0.11.0 | #30 | `GetCurrentThreadStackLimits`, declared and exported | +| `openkal-emscripten` | 0.4.0 | #5 | the platform's per-context pair | +| `openkal-opensbi` | 0.8.2 | #19 | the pin, and nothing else | +| `openkal-musl` | 0.20.0 | #50 | the real answer for the calling thread, the refusal for another, the withdrawn absent row | +| `openkal-linux` | 0.16.1 | #33 | the thread-local image placed where the linker measured it (a repair; §3.1) | +| `openkal-musl` | 0.20.1 | #51 | the pin that reaches the repair | +| `openkal-llvm-runtime` | 0.15.4 | #34 | the pin that reaches the repaired C library | + +Every branch was named `openkal-0.15.0` in every repository, which is what makes +each repository's continuous integration test the other halves as written rather +than as published: the specification clones the implementation at that branch, +and the implementation clones the specification at it. + +## 2. What was measured, and where + +| Claim | Instrument | Result | +| --- | --- | --- | +| the region the kernel stops the stack at is `mapping_end - RLIMIT_STACK` | a descending write in a probe on 6.8.0 | exact: the floor writable, one page below it not, one page above it writable | +| the region below is bounded by one guard gap above the nearest mapping | the same probe with a page mapped by itself inside the reservation | exact: 256 pages above the planted page | +| the same, on the released implementation | the conformance suite's five observations, in `openkal-linux`'s own tests, and end to end through `openkal-musl` | held | +| the macOS names resolve | a cross link of a program that calls the enquiry, aarch64 and x86_64 | no undefined symbols; the four imports are exactly the ones the code names | +| the Windows name resolves | the declared-vs-exported gate, the generated `libkernel32.a`, and a cross build for `x86_64-windows-gnu` | pass; `__imp_GetCurrentThreadStackLimits` present | +| the WebAssembly pair answers per context | the task gate's program under node | exits 0; both contexts are told a region containing themselves | +| the C library answers for the caller and refuses another thread | `examples/stack-bounds` above `openkal-musl` 0.20.0 | first and started contained, another thread refused | + +## 3. The two defects the round exposed, and their fixes + +### 3.1 The thread-local image was placed eight bytes below the variables + +**This is the one that mattered, and 0.16.0 shipped it.** Every thread-local +address is `tp + st_value - tls_size`, where `tls_size` is the block the LINKER +laid out: `p_memsz` rounded up to the segment's alignment. `describe_tls` +clamped that alignment up to sixteen *before recording it*, so the clamp reached +the SIZE, which must not have it. Measured on a segment stating `p_align = 8, +p_memsz = 56`: the linker put the variables at `tp - 56` and the region was +built 64 bytes deep, so the image of the program's thread-local storage sat +eight bytes below the variables that name it. + +From above: two thread-local variables declared with different values read each +other's bytes, and a C++ program's `thread_local` object can find its guard byte +non-zero and never run its constructor at all. That is how it was found — not by +this round's own tests, which passed, but by `openkal-llvm-runtime`'s probe +against the released packages, on the runner: + +``` +FAIL: a thread_local is constructed in a spawned thread +FAIL: and its destructor runs when that thread ends +``` + +**Why it surfaced now, stated rather than glossed.** The clamp is as old as the +file. This implementation's own thread-local storage had been one four-byte +variable, which sits at the very end of the segment and was copied correctly by +luck; the twenty-four bytes 0.16.0 added moved the image far enough to be read +by the wrong variable. The kit test therefore passed throughout, and the test +0.16.0 added for exactly this area — `tests/conformance_task_tls.cpp`, with a +non-zero initialiser and a four-kilobyte block — did not catch it either, +because the alignment its own link happened to produce was sixteen. What did +catch it was a C++ `thread_local` object with a destructor, in another +repository's continuous integration, against the published packages. + +The repair: `describe_tls` keeps the segment's alignment as the loader stated +it; `make_tls` rounds the size by that and asks the ALLOCATOR for sixteen. +Verified against a probe of two initialised thread-local variables — in the +context the program was started on and in a started one — by the kit, by this +package's tests, by the conformance suite (197 held, 0 did not hold), and by the +runtime's own example, which now reports `-- failures: 0 --`. + +### 3.2 A context's storage was laid out from an undescribed image + +`openkal-linux` described its own thread-local image only from the entry point it +supplies, and a program that carries a runtime never runs that entry point. A +context was therefore given a storage region laid out from a segment of size +zero: its thread pointer sat at the START of the region rather than at its end, +and every thread-local variable of that context was addressed below the +allocation — into whatever the allocator kept beside it. It had been invisible +while this implementation's own storage was one four-byte variable, and it became +fatal when the storage for this round's operation made it twenty-four: the +specification package's own kit test died at the first instruction of a context. + +`make_tls` now describes the image if nothing has, from the vectors `env.cpp` +records in either arrangement, and refuses to lay out a region it cannot size +rather than laying out a wrong one. + +## 4. What this round could not measure + +- **macOS and Windows at run time.** Their mechanisms are exercised by the + conformance suite on their own runners; what was done here is the cross link + and the symbol gates. `GetCurrentThreadStackLimits`' reservation-versus-commit + question remains unpublished by the vendor and is stated at the declaration. +- **A machine with firmware and no operating system.** `openkal-opensbi` provides + no `openkal.task`, and the round's change to it is one version pin. +- **The `stack_guard_gap` a kernel is booted with.** The default is applied; a + kernel configured with a larger gap stops the stack above the base reported, + which is the direction a caller must not be wrong in, and it is stated at the + code. + +## 5. The verification transcript + +From `2026-10-03-stack-bounds-of-a-context-verify.sh`, run inside an `xlings` +sandbox (`xlings subos use okl015 --sandbox --cmd ...`) against the published +index with the CN mirror selected, using the released engine: + +``` +▸ entering subos okl015 (exit to leave) + +== A. identity and mirror == +ok: mcpp 2026.10.1.3 from /home/speak/.xlings/data/xpkgs/xim-x-mcpp/2026.10.1.3/bin/mcpp +ok: xlings mirror is CN + +== B. openkal 0.15.0 and openkal-linux 0.16.1 resolve, and a context is told its own region == +proot warning: ptrace(PEEKDATA): Bad address +ok: version 0.15.0 first e=0 contains=1 size=8376320 +ok: started contains=1 ran=1 +ok: the lock records openkal-linux 0.16.1 + +== C. openkal-musl 0.20.1: the calling thread is told its region, another thread is refused == +ok: musl first e=0 contains=1 size=3280896 +ok: musl started contains=1 size=262144 another thread refused=1 +ok: the lock records openkal-musl 0.20.1 +NOT RUN: macOS, Windows and WebAssembly: their implementations answer on their own runners (openkal-macos#26, openkal-windows#30, openkal-emscripten#5), and the two cross links are made on this machine by their own continuous integration +NOT RUN: a machine with firmware and no operating system: openkal-opensbi 0.8.2 moves a pin and provides no openkal.task + +0 assertion(s) failed +not run: + - macOS, Windows and WebAssembly: their implementations answer on their own runners (openkal-macos#26, openkal-windows#30, openkal-emscripten#5), and the two cross links are made on this machine by their own continuous integration + - a machine with firmware and no operating system: openkal-opensbi 0.8.2 moves a pin and provides no openkal.task +exit=0 +``` + +## 6. Review of the round, at the scale of the ecosystem + +**What is consistent.** One declaration, five implementations answering the same +question in the same shape, one suite observing it once for all of them, and one +C library asking it on behalf of the callers that cannot. Every implementation +that answers was measured answering on its own system: Linux here and in +`openkal-musl`'s example, macOS and Windows on their runners through the +conformance suite, WebAssembly under node in the emscripten gate. + +**What the round changed that was not this feature.** One latent defect in the +hosted arrangement of `openkal-linux` — a context's thread-local storage laid out +from an undiscribed image, which had been silently writing below its own +allocation since the implementation was written. It was found because this +feature made the implementation's own storage large enough to fault. That is the +honest account: the round's own change did not cause it, and the round could not +have been finished without it. + +**What a consumer must know.** A version written without an operator is an exact +pin in mcpp, so a consumer moves by naming the new versions, and any package that +locks `openkal` and can share a graph with the consumer must move too. Within +this round that is `openkal-musl` 0.20.1 and `openkal-llvm-runtime` 0.15.4; a +program that carries the C++ runtime names both, and the runtime's own release +is a consequence of the repair rather than of the feature. + +**What remains open, with the reason.** The macOS and Windows mechanisms cannot +be run from this machine, and their reservation-versus-commit question is +answered by the vendor's documentation on one system and by a reimplementation +on the other; both are stated at the declaration. The `stack_guard_gap` a kernel +is booted with cannot be read by an implementation, so the default is applied and +the direction of the error is stated. And `pthread_getattr_np` still refuses for +a thread that is not the caller: a context can only be asked about itself, which +is the shape the specification chose and not a gap left by the port. diff --git a/.agents/docs/2026-10-03-stack-bounds-of-a-context-verify.sh b/.agents/docs/2026-10-03-stack-bounds-of-a-context-verify.sh new file mode 100644 index 0000000..2aea218 --- /dev/null +++ b/.agents/docs/2026-10-03-stack-bounds-of-a-context-verify.sh @@ -0,0 +1,246 @@ +#!/usr/bin/env bash +# Ecosystem verification for the wave that lets a running context say where it +# stands (openkal 0.15.0, openkal-linux 0.16.1, openkal-macos 0.13.0, +# openkal-windows 0.11.0, openkal-emscripten 0.4.0, openkal-opensbi 0.8.2, +# openkal-musl 0.20.1), resolved from the published index only. +# +# xlings subos new okl015 +# cp ~/.xlings/subos/okl015/tmp/v.sh +# xlings subos use okl015 --sandbox --cmd "MCPP_VERIFY_VERSION= bash /tmp/v.sh" +# +# The script is copied into the sandbox's /tmp rather than passed on its command +# line: under proot a command line of this size was observed to come with +# `ptrace(PEEKDATA): Bad address` and failures of unrelated file operations. +# +# The subject is what a consumer gets by naming the released versions. A version +# written without an operator is an exact pin in mcpp (`is_constraint` in +# modules/versioning/src/version_req.cppm), so each manifest below is what a +# consumer writes. A step that cannot run here says so and is counted as not +# run, never as passed. +set -u + +VER="${MCPP_VERIFY_VERSION:?set MCPP_VERIFY_VERSION}" +STORE="${MCPP_VERIFY_BIN:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp}" +XL="${XLINGS_BIN:-$(command -v xlings)}" + +fails=0; skipped="" +fail() { printf 'ASSERT-FAIL: %s\n' "$1"; fails=$((fails + 1)); } +ok() { printf 'ok: %s\n' "$1"; } +section() { printf '\n== %s ==\n' "$1"; } +skip() { printf 'NOT RUN: %s\n' "$1"; skipped="$skipped + - $1"; } + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +# -- A. identity and mirror --------------------------------------------------- +section "A. identity and mirror" +got=$("$STORE" --version 2>&1 | head -1) +[ "$got" = "mcpp $VER" ] && ok "$got from $STORE" || fail "version is '$got' at $STORE" +"$STORE" self config --mirror CN >/dev/null 2>&1 || true +"$XL" config --mirror CN >/dev/null 2>&1 || true +xm=$(python3 -c "import json,os;print(json.load(open(os.path.expanduser('~/.xlings/.xlings.json'))).get('mirror',''))" 2>/dev/null) +[ "$xm" = "CN" ] && ok "xlings mirror is CN" || fail "xlings mirror is '$xm'" +"$STORE" index update >/dev/null 2>&1 || true + +# -- B. the implementation beneath answers the context that asks -------------- +# The manifest names the specification and one implementation, which is the +# published shape. The program carries no runtime, so the region it asks about +# is the one openkal-linux measured: a running context can always say where it +# stands, which is the whole argument for clause 6.4 admitting the operation. +section "B. openkal 0.15.0 and openkal-linux 0.16.1 resolve, and a context is told its own region" +mkdir -p "$work/ver/src" +cat > "$work/ver/mcpp.toml" <<'TOML' +[package] +name = "stackprobe" +version = "0.1.0" + +[dependencies] +openkal = "0.15.0" + +[target.'cfg(os = "linux")'.dependencies] +openkal-linux = "0.16.1" + +[targets.stackprobe] +kind = "bin" +main = "src/main.c" +TOML +cat > "$work/ver/src/main.c" <<'C' +#include +#include +#include + +static void say(const char* s, kal_uintptr n) { kal_stream_write(kal_stdout(), s, n); } +static void put(const char* s) { kal_uintptr n = 0; while (s[n]) n++; say(s, n); } +static void number(kal_uintptr v) { + char b[24]; int i = 0; + if (v == 0) { put("0"); return; } + while (v) { b[i++] = (char)('0' + (v % 10)); v /= 10; } + char o[24]; int j = 0; + while (i) o[j++] = b[--i]; + o[j] = 0; put(o); +} + +/* The property a caller relies on and not the shape of the numbers: the region + * contains a local of the context that asked. */ +static int holds(void* base, kal_uintptr size, void* here) { + const kal_uintptr b = (kal_uintptr)base; + const kal_uintptr at = (kal_uintptr)here; + return size != 0 && b + size > b && at >= b && at - b < size; +} + +static volatile int g_started = -1; + +static void entry(void* arg) { + char here = 0; + void* base = 0; + kal_uintptr size = 0; + const int e = kal_task_stack(&base, &size); + g_started = (e == kal_ok && holds(base, size, &here)) ? 1 : 0; + *(int*)arg = 1; +} + +int main(void) { + char here = 0; + void* base = 0; + kal_uintptr size = 0; + const int e = kal_task_stack(&base, &size); + + put("version "); + number((kal_uintptr)KAL_VERSION_MAJOR); put("."); + number((kal_uintptr)KAL_VERSION_MINOR); put("."); + number((kal_uintptr)KAL_VERSION_PATCH); + put(" first e="); number((kal_uintptr)e); + put(" contains="); number((kal_uintptr)(e == kal_ok && holds(base, size, &here)) ? 1u : 0u); + put(" size="); number(size); + put("\n"); + + int ran = 0; + struct kal_task t = {0}; + if (kal_task_start(entry, &ran, &t) == kal_ok) { + kal_task_join(t); + put("started contains="); number((kal_uintptr)(g_started == 1) ? 1u : 0u); + put(" ran="); number((kal_uintptr)ran); + put("\n"); + } else { + put("started refused\n"); + } + return 0; +} +C +out=$(cd "$work/ver" && "$STORE" run 2>&1); rc=$? +first=$(printf '%s\n' "$out" | grep -E '^version ' | head -1) +started=$(printf '%s\n' "$out" | grep -E '^started ' | head -1) +if [ $rc -ne 0 ] || [ -z "$first" ]; then + fail "the stack probe did not build or run"; printf '%s\n' "$out" | tail -8 +else + printf '%s\n' "$first" | grep -q '^version 0.15.0 first e=0 contains=1 ' \ + && ok "$first" || fail "the probe reported: $first" + [ "$started" = "started contains=1 ran=1" ] \ + && ok "$started" || fail "the started context reported: $started" +fi +lock=$(cat "$work/ver/mcpp.lock" 2>/dev/null) +printf '%s\n' "$lock" | grep -q '0.16.1' && ok "the lock records openkal-linux 0.16.1" \ + || fail "the lock does not record openkal-linux 0.16.1" + +# -- C. the C library above it answers, and refuses where it must ------------- +# `pthread_getattr_np' was the reason for the change: above openkal it described +# a range that was not the stack. It now answers for the calling thread with the +# region the implementation measured, and refuses for any other thread --- a +# refusal a program can read being the only alternative to a range it cannot +# check. The program is C, so nothing here needs the C++ runtime. +section "C. openkal-musl 0.20.1: the calling thread is told its region, another thread is refused" +mkdir -p "$work/musl/src" +cat > "$work/musl/mcpp.toml" <<'TOML' +[package] +name = "muslstack" +version = "0.1.0" + +[dependencies] +openkal-musl = "0.20.1" + +[targets.muslstack] +kind = "bin" +main = "src/main.c" +TOML +cat > "$work/musl/src/main.c" <<'C' +#define _GNU_SOURCE +#include +#include +#include +#include + +static int holds(void* base, size_t size, void* here) { + const uintptr_t b = (uintptr_t)base; + const uintptr_t at = (uintptr_t)here; + return size != 0 && b + size > b && at >= b && at - b < size; +} + +struct started { int contained; int refused; size_t size; }; + +static void* body(void* arg) { + struct started* s = arg; + char here = 0; + pthread_attr_t a; + int e = pthread_getattr_np(pthread_self(), &a); + void* base = 0; + size_t size = 0; + if (e == 0 && pthread_attr_getstack(&a, &base, &size) != 0) e = -1; + s->contained = (e == 0 && holds(base, size, &here)) ? 1 : 0; + s->size = size; + /* The identifier of another thread, asked from here: the port compares it + * and never reads it, so a refusal is the only thing this can produce. */ + pthread_attr_t other; + s->refused = pthread_getattr_np((pthread_t)(uintptr_t)0x1234, &other) == ENOSYS; + return 0; +} + +int main(void) { + char here = 0; + pthread_attr_t a; + int e = pthread_getattr_np(pthread_self(), &a); + void* base = 0; + size_t size = 0; + if (e == 0 && pthread_attr_getstack(&a, &base, &size) != 0) e = -1; + printf("musl first e=%d contains=%d size=%zu\n", e, + (e == 0 && holds(base, size, &here)) ? 1 : 0, size); + + struct started s = {0, 0, 0}; + pthread_t t; + if (pthread_create(&t, 0, body, &s) != 0 || pthread_join(t, 0) != 0) { + printf("musl started did not run\n"); + return 1; + } + printf("musl started contains=%d size=%zu another thread refused=%d\n", + s.contained, s.size, s.refused); + return 0; +} +C +out=$(cd "$work/musl" && "$STORE" build 2>&1); rc=$? +bin=$(find "$work/musl/target" -type f -name muslstack 2>/dev/null | head -1) +if [ $rc -ne 0 ] || [ -z "$bin" ]; then + fail "the musl probe did not build"; printf '%s\n' "$out" | tail -8 +else + run=$(timeout 120 "$bin" 2>&1); rc=$? + first=$(printf '%s\n' "$run" | grep -E '^musl first ' | head -1) + started=$(printf '%s\n' "$run" | grep -E '^musl started ' | head -1) + if [ $rc -ne 0 ] || [ -z "$first" ]; then + fail "the musl probe did not run"; printf '%s\n' "$run" | tail -8 + else + printf '%s\n' "$first" | grep -q '^musl first e=0 contains=1 ' \ + && ok "$first" || fail "the first context reported: $first" + printf '%s\n' "$started" | grep -q '^musl started contains=1 .*another thread refused=1$' \ + && ok "$started" || fail "the started context reported: $started" + fi + lk=$(cat "$work/musl/mcpp.lock" 2>/dev/null) + printf '%s\n' "$lk" | grep -q '0.20.1' && ok "the lock records openkal-musl 0.20.1" \ + || fail "the lock does not record openkal-musl 0.20.1" +fi + +# -- D. what this sandbox cannot observe -------------------------------------- +skip "macOS, Windows and WebAssembly: their implementations answer on their own runners (openkal-macos#26, openkal-windows#30, openkal-emscripten#5), and the two cross links are made on this machine by their own continuous integration" +skip "a machine with firmware and no operating system: openkal-opensbi 0.8.2 moves a pin and provides no openkal.task" + +printf '\n%s assertion(s) failed\n' "$fails" +[ -n "$skipped" ] && printf 'not run:%s\n' "$skipped" +exit "$fails"