Make Python macros BOUND by default with explicit RAW authority - #15
Merged
Merged
Conversation
Give macros one compiler-owned authority marker: absence means BOUND and the exact header line '# neuron: raw' opts into unrestricted Python. Malformed or duplicate Neuron directives are policy errors rather than guessed settings, and the Workshop toggle edits that source line instead of storing a second preference. Promote the visual constructor from a body-only Vec<MacroNode> to MacroDocument. Parsing now carries the module prefix, the actual entry wrapper, typed body and suffix together, so a block edit keeps imports, NEURON_OPTIONS, helpers, comments, main-vs-macro naming and ordinary multi-line signatures instead of manufacturing a new bare macro() module. The catalog exposes BOUND/RAW from the same source-derived model. parse_macro stays as a compatibility projection over the whole-document parser, so existing callers do not need a second Python grammar implementation.
BOUND changes the default for new source, not the meaning of code users already trusted. On the first observation of an install's macro directory, atomically stamp every pre-feature Python file with '# neuron: raw' and write the migration marker last. A partial IO failure therefore retries rather than silently reclassifying an old unrestricted macro. Run that migration before seeding bundled examples. Existing user macros keep RAW, while examples created by the new build are new source and remain BOUND. Once the marker exists, later files are never rewritten, which makes absence of the directive the stable new-code default rather than a version-dependent guess.
Turn source policy into process topology. RAW remains the compiler/control lane and unrestricted runtime; BOUND gets its own independently supervised CPython process. Registration, ephemeral tests, blocking invokes and live fires all route from the source-derived mode, while the 1000 Hz path performs only an in-memory mode lookup. Each lane owns its own crash breaker and warm latch. A control-plane failure retires only the process that served it, arm state is mirrored to both, and prompt ids use disjoint namespaces so two independent sidecars can share the existing beacon UI without pid collisions. The child handshake now names its execution domain and Rust refuses a mismatched process. No containment is claimed by this commit yet; it establishes the structural boundary that the BOUND capability policy can actually rely on.
Give the BOUND interpreter a deliberately small Python surface: curated imports, no file/process/eval primitives, no private/dunder reflection, and no executable module initialization beyond literals, imports and plain function definitions. RAW keeps ordinary CPython unchanged. This is policy containment for trusted and agent-authored macros, not a claim that CPython is a hostile-code security sandbox. The familiar neuron helpers keep their names in both worlds, but BOUND input, clipboard and focus operations cross the framed host boundary and reuse Neuron's native primitives. Arbitrary neuron.run is explicitly RAW-only. Move the final SAFE/mock decision to Rust as well. The helper still refuses early, but every effectful act frame is checked again at execution time before native input, audio, device, clipboard, focus or external-integration work can land. The sidecar supplies macro identity and per-fire mock state so that check cannot be bypassed by simply reaching the protocol directly.
BOUND macros should not need ambient filesystem authority just to remember a counter or preference. Route store/load/forget/stored through the existing act protocol and keep the JSON files in the same per-macro state directory and shape RAW already uses, so switching modes does not strand data. Rust owns the per-macro state lock, JSON parsing and atomic replacement for BOUND. Locks are keyed by macro id so unrelated namespaces never serialize, and Windows sharing violations get the same bounded retry budget as the Python RAW path. State verbs are deliberately not SAFE-gated: they mutate Neuron-owned macro state, not the user's input/device/clipboard/system state. The host still validates the macro id and JSON boundary before touching disk.
A split runtime must not turn neuron.invoke into an authority tunnel. Same-domain calls keep their existing direct Python semantics. When a target lives in the other interpreter, invocation returns to Rust: RAW may call a BOUND macro, which still executes inside BOUND, while BOUND -> RAW is refused before any target fire is sent. The bridge forwards the trigger-time context, options, mock state and target generation. Async calls remain queued on the callee's serial actor; synchronous cross-domain calls wait on the callee result without moving its code into the caller's interpreter. Same-domain synchronous return values remain native Python; cross-domain values are the sidecar protocol's string result. This makes the authority rule structural: a callee can never acquire more ambient authority merely because a stronger or weaker macro invoked it.
Exercise the real bundled interpreters as two domains in one integration binary. BOUND must fail ambient file access and os import through its normal language surface and report neuron.run as RAW-only; RAW must retain ordinary import/open power. Same-domain invoke keeps native Python values, RAW -> BOUND crosses the Rust bridge, and BOUND -> RAW returns no callable. The same proof carries BOUND state through a deliberate switch to RAW and checks the asynchronous RAW -> BOUND path by observing the callee's persisted marker, not by firing the target from the test itself. Also pin the actual authority boundary directly in Rust: a forged effectful act frame is refused while the process is disarmed before clipboard/native execution is reached. That test does not depend on Python's convenience gate.
The BOUND/RAW router exposed one transactional gap: after the source write it updated mode/generation in the host before the prepared callable's final commit ack. A concurrent press could therefore route to a revision that was durable but not yet live. Keep the old routing truth until COMMIT succeeds. Only then publish source, mode, generation and options, and only then retire an old execution lane after a mode switch. If the final acknowledgement is lost, retire the candidate session and restore the previous source atomically instead of guessing whether the new callable won. The next use therefore reconstructs last-known-good disk + manifest truth.
Keep ordinary macro behavior on the new BOUND default, but make every test that deliberately needs ambient os/sys/ctypes access or a process-kill crash opt into RAW explicitly. This preserves the old test intent instead of letting BOUND import policy turn runtime/crash tests into registration failures. Rework the act stress around the new host authority boundary. Private _act probes run RAW, forged effectful frames are expected to fail closed while disarmed, and rid/timeout/concurrency correlation uses host-owned state replies that remain safe to exercise without hardware. Separate RAW pid and BOUND module-state canaries prove neither sidecar silently respawns during its half of the stress. Also fix curated from-import validation: BOUND authorizes the imported module root, not the imported symbol name. The authority e2e now covers that form and proves a RAW process crash leaves the independent BOUND interpreter warm.
With BOUND and RAW in separate warm interpreters, the old RetireAll event became incorrect: a RAW crash could withdraw an unrelated BOUND ask that was still alive and waiting for the user. Make process death emit RetireDomain(mode). Prompt ids already occupy disjoint high-bit namespaces, so the GUI can remove/stop only prompts owned by that domain. The CLI ignores retirement as before. Update crash stress accordingly: two BOUND asks remain answerable across a RAW process death, while the RAW lane emits its own retirement event and then proves it can respawn and serve a fresh ask. RAW-only queue/runtime crash tests now wait for RAW-domain retirement explicitly.
Make the Rust capability layer fail closed instead of merely approximating the old Python helpers. macro_key/hotkey/click now re-check the process arm gate inside the native adapter as defense in depth, unknown mouse button names are rejected instead of silently becoming a left click, and hotkey preserves the Python helper's arbitrary key-chord semantics rather than treating every chord as a modifier-prefixed single key. Clamp brokered mouse coordinates before narrowing i64 protocol values to i32 so giant accidental values cannot wrap into unrelated cursor motion. A parser unit test pins invalid mouse-button refusal without touching real input.
Bring the public and internal docs in line with the runtime that now exists. New macros are BOUND by default, one source-owned '# neuron: raw' directive selects unrestricted Python, and the two tiers never share an interpreter. Be explicit about the threat model: BOUND is capability-oriented policy containment for ordinary and agent-authored macros, not a promise that CPython can safely execute hostile code. RAW remains full user-authority Python by design and can bypass Neuron's helpers. Existing macros migrate to RAW once so an upgrade does not silently revoke authority. Also correct the runtime/network wording, invocation direction, host-side arm enforcement, prompt-domain isolation, and the old one-sidecar comments so the README, SECURITY, GDD, TDD, STATUS and source headers all describe one system.
The pyruntime respawn phase killed the RAW interpreter while its outstanding ask had accidentally remained BOUND. That made a RAW RetireDomain event arrive while the prompt under test correctly survived in the other process. Make both the pre-crash and post-respawn asks explicitly RAW so the phase proves what it says: one RAW crash retires RAW prompts and the rebuilt RAW lane serves a fresh beacon. Also update the resident-budget comments for multiple warm macro-host helpers. The allowlist already permits python.exe descendants without assuming a count.
The whole-document parser already kept multi-line main/macro wrappers, but a one-line suite such as `def main(ctx, n=3): return n` fell back to the canonical `def macro(ctx):` wrapper on the first canvas edit. That made an advertised source-preservation edge silently rename the entry and discard its signature. Use the first body statement's AST location to split the physical line at the suite boundary. AST columns are UTF-8 byte offsets, so slice encoded bytes before decoding the preserved prefix. The visual body may expand onto normal indented lines, but main-vs-macro, async/signature text, decorators and leading header source survive. Pin the case against the real sidecar with a RAW one-line `main` document.
BOUND already failed closed at runtime, but authoring still let restricted code look syntactically valid until Save or execution. Promote the containment rules into the pure AST Check path so authority is visible before user code runs. The BOUND policy pass now walks the whole module, including function bodies: forbidden imports, ambient file/eval/reflection builtins, private/dunder access and neuron.run report line-numbered "requires RAW" errors. Module initialization remains declarative, while future-annotations keeps non-executing annotations available without unnecessary escalation. Rust derives the mode from source and sends it to the zero-exec compiler check; durable registration runs the same check before preparing a candidate. Workshop copy now calls this a compiler check instead of syntax-only validation. Keep the legacy process-spawn stress explicitly RAW, since it is testing RAW's arm gate. Add real-sidecar assertions that BOUND run/os/open fail at Check with source lines and that the exact '# neuron: raw' directive makes the same source compiler-valid.
The inherited last-known-good e2e intentionally fails during module top-level execution after the candidate reaches the runtime. BOUND now rejects executable top-level statements in the pure compiler-policy pass, which would make that fixture pass for the wrong reason. Mark the baseline and broken replacement RAW so the test continues to exercise the prepare/execute failure path, proving the old callable and durable file survive a real candidate-load exception rather than only a policy rejection.
worflor
force-pushed
the
macros/bound-raw-v2
branch
from
September 19, 2026 20:35
0e7c14b to
0e193d9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack
This is PR 2 of 2 and is intentionally stacked on #14 (
macros/runtime-truth). Review #14 first; this PR assumes its transactional registration, generation pinning, ephemeral test path, canonical ids, worker retirement, and control-plane recovery.What changed
Make Python macro authority source-owned and explicit without removing the project's full-power Python philosophy.
Source/compiler contract
no directive means BOUND
exactly
# neuron: rawmeans RAWmalformed/duplicate Neuron directives are compiler-policy errors instead of guessed settings
the Workshop RAW toggle adds/removes that exact source line; there is no hidden sidecar preference
catalog cards derive BOUND/RAW from source
promote the visual compiler from a body-only
Vec<MacroNode>to a wholeMacroDocument, preserving module prefix/suffix, the actual entry wrapper, options/imports/helpers/comments around the entry, body nodes, and execution mode through block editsBackward compatibility
existing pre-feature
.pymacros are atomically stamped RAW exactly once before the new default takes effectthe migration marker is written last, so interruption retries safely
bundled/new macros created after migration remain BOUND by absence of the pragma
BOUND and RAW persistent state use the same per-macro JSON namespace, so switching mode does not fork memory
Runtime authority
split MacroHost into two independently supervised warm CPython domains; RAW and BOUND never share an interpreter
each domain has its own crash breaker, warm latch, registry population, prompt lifetime, and recovery
prompt ids use disjoint namespaces; a dead RAW sidecar no longer withdraws live BOUND prompts (and vice versa)
registered dispatch chooses the domain from an in-memory source-derived mode map; no disk or policy parse moves onto the 1000 Hz path
RAW remains ordinary unrestricted CPython
BOUND receives curated imports/builtins plus the public
neuroncapability proxy and refuses ambient file/process/network/native-FFI/reflection surfaces through the supported language patharbitrary
neuron.runrequires RAWBOUND is intentionally described as policy containment, not a hostile-code Python sandbox. The feature is for safer/default and agent-authored macros, not for executing adversarial untrusted Python.
Host-owned effects
broker BOUND key/chord/text/ghost typing, mouse, clipboard, focus, device/audio/OBS effects, and persistent macro state through Rust
Rust independently re-checks arm/mock authority before effectful work; Python's helper check is no longer the final security decision
native adapters fail closed on invalid buttons/keys and clamp protocol-sized mouse coordinates before narrowing
state verbs are host-owned but not SAFE-gated because they mutate Neuron macro state, not user input/device/system state
Composition
same-domain synchronous invoke keeps existing native-Python subroutine semantics
cross-domain calls go through Rust
RAW → BOUND is allowed and executes the callee inside BOUND
BOUND → RAW is refused before dispatch, so
invoke()cannot become an authority-escalation tunnelasync cross-domain calls still enter the target's serial worker
Verification
Linux (WSL):
cargo test --workspace --exclude neuron-app --lockedpassed on the final stacked branch.Windows:
.\validate.ps1and.\validate.ps1 -Mode fullpassed on the final branch. Full mode covered the feature matrix, release build, and CI-safe ignored tests; its reference emitter now receives its required scratch directory..\validate.ps1 -Mode lintpassed with advisory rustfmt findings; full mode reported advisory clippy findings.Built and launched
neuron-app --safewith a separate run directory. Workshop displayed the RAW toggle and the seeded macro card displayed its BOUND label.Bundled-CPython BOUND/RAW authority, migration, whole-document round trip, and macro stress tests passed locally.
GitHub Actions cannot start the
what changedrunner because of the account billing or spending-limit annotation; platform test jobs are skipped.Live input injection and hardware-write behavior were not exercised.
Threat-model note
RAW is still the deliberate full-power escape hatch. A RAW macro has the user's normal Python/process authority and can bypass Neuron's helpers. BOUND meaningfully reduces ambient authority and structurally isolates itself from RAW, but it is not presented as adversarial-code containment.
Before this can merge
If the pull request contains work by more than one author, every author must
record the same acceptance in the pull request, unless the submitter has
authority to accept for them.