Document the protocols and state machines

docs/architecture.md

@@ -127,9 +127,9 @@ In isolated mode there is one path: a status-3 frame on the socket, answered
 by a process the `py_isolated` state machine spawns; nested calls into the
 same context are dispatched immediately because they come from that process.
 
-The frame format shared by the pipe and the socket, and the ETF conventions,
-will get their own page (protocols); until then `c_src/py_convert.c` (type
-mapping) and `priv/_erlang_impl/_etf.py` are the reference.
+Every message and frame, with the rules for changing them, is in
+[protocols](protocols.md); the states each process and thread moves through
+are in [state machines](state-machines.md).
 
 ## asyncio
 

docs/protocols.md

@@ -0,0 +1,247 @@
+# Protocols
+
+Every message and frame that crosses a boundary in erlang_python: between
+an Erlang process and the context process, between the context thread in C
+and Erlang, and between the VM and an isolated child. Read this when you
+change a message shape, add a request kind, or debug a hang with
+`sys:trace/2` or `strace`. The code is the reference; this page tells you
+where each piece is and what must stay consistent.
+
+## Boundaries
+
+```
+ caller process  --(1) Erlang messages-->  py_context / py_isolated process
+                                                  |
+                       embedded modes             |          isolated mode
+                (2) NIF request queue             |     (4) Unix socket frames
+                (3) callback pipe / thread pipe   |
+                                                  v
+                          context thread in C            child OS process
+```
+
+1. Erlang messages: `src/py_context.erl` (API side and embedded loop),
+   `src/py_isolated.erl` (isolated loop).
+2. Request queue: `ctx_request_t` in `c_src/py_nif.h`, enqueued by
+   `nif_context_call_async`, answered with `{py_result, Ref, Result}`.
+3. Callbacks: `erlang_call_impl` in `c_src/py_callback.c`,
+   `c_src/py_thread_worker.c`, `src/py_thread_handler.erl`.
+4. Socket: `src/py_isolated.erl`, `priv/_erlang_impl/_isolated.py`,
+   `priv/_erlang_impl/_etf.py`.
+
+## 1. Caller to context process
+
+The public API (`py:call/3`, `py_context:call/4`, ...) is a plain message
+plus a monitor. Both context loops accept the same messages, so a caller
+never knows the mode.
+
+```erlang
+%% py_context:submit/5 and friends
+MRef = erlang:monitor(process, Ctx),
+Ctx ! {call, self(), MRef, Module, Func, Args, Kwargs},
+%% reply
+{MRef, Result}          %% {ok, Term} | {error, Reason}
+```
+
+| Message | Reply | Notes |
+|---|---|---|
+| `{call, From, MRef, Module, Func, Args, Kwargs}` | `{ok, R}` / `{error, E}` | 8-tuple variant adds `EnvRef` for process-local envs |
+| `{eval, From, MRef, Code, Locals}` | same | 6-tuple variant adds `EnvRef` |
+| `{exec, From, MRef, Code}` | `ok` / `{error, E}` | 5-tuple variant adds `EnvRef` |
+| `{submit, From, MRef, TaskRef, Module, Func, Args, Kwargs}` | `ok`, then `{async_result, TaskRef, Result}` to `From` | asyncio coroutine on the context loop |
+| `{start_loop, From, MRef, Owner}` | `ok` / `{error, already_running}` | runs `run_forever` on the context thread; `Owner` is monitored and gets `{py_loop_exit, Ctx, Result}` when the loop ends |
+| `{stop_loop, From, MRef, GraceMs}` | `ok` / `{error, no_loop}` | cooperative stop, interrupt after `GraceMs`, then kill |
+| `{loop_ref, From, MRef}` | `{ok, LoopRef}` / `{error, no_loop}` | isolated: `{error, not_supported_in_isolated}` |
+| `{interrupt, From, MRef}` | `ok` / `not_running` | |
+| `{kill, From, MRef}` | `ok` | isolated only: SIGKILL, answered once the new child is up |
+| `{pass_fd, From, MRef, Fd}` | `ok` / `{error, E}` | isolated only: SCM_RIGHTS |
+| `{child_info, From, MRef}` | `{ok, Map}` | isolated only |
+| `{stop, From, MRef}` | `ok` | |
+
+Timeouts are the caller's business: `await_reply/3` waits `Timeout`, then
+sends `{interrupt_request, MRef}` so the request executing now is
+interrupted, waits a short grace for the late reply, and returns
+`{error, timeout}`. Loop control messages use `await_ctrl_reply/3`, which
+must not interrupt (it would stop the loop it manages) and instead sends
+`{cancel_ctrl, MRef}` so an isolated context drops the pending entry.
+
+Rules:
+
+- One request at a time per context. The embedded loop blocks in the
+  request; `py_isolated` postpones with `gen_statem` `postpone` while in
+  `{busy, Id}`.
+- A request from a process that is running a callback for this context is
+  nested and served at once (both loops track those pids).
+- Replies always go to `From`, tagged with `MRef`; a late reply after a
+  timeout is flushed by `demonitor(MRef, [flush])`.
+
+## 2. Request queue (embedded modes)
+
+`nif_context_call_async(Ctx, Kind, Data, RequestId)` allocates a
+`ctx_request_t`, copies the terms into `request_env`, appends it under
+`queue_mutex` and returns. The context thread dequeues, runs the request,
+and sends:
+
+```erlang
+{py_result, RequestId, Result}
+```
+
+to `caller_pid` through `msg_env`. Cancelled requests (context destroyed
+while queued) are answered with `{py_result, Id, {error, cancelled}}`.
+Request kinds are `ctx_request_type_t` in `c_src/py_nif.h`; the execute
+functions read them from the request mirror on `py_context_t`
+(`request_type`, `request_term`, ...).
+
+A result that needs a callback served before completion comes back as
+`{suspended, CallbackId, StateRef, {Name, Args}}` in place of the final
+result (see 3.1); the process resumes with `resume_callback/2` and waits
+for the next `py_result`.
+
+## 3. Python calling Erlang
+
+`erlang.call(Name, Args...)` picks a path in `erlang_call_impl`
+(`c_src/py_callback.c`); the comment above that function is authoritative.
+Precedence: suspension when the calling thread is a context thread with
+suspension enabled, then the context's blocking pipe when a handler is set,
+then the thread worker for every other Python thread.
+
+### 3.1 Suspension
+
+The Python call raises `SuspensionRequired` and sets thread-local pending
+state; the execute function sees the flag before the exception type and
+returns the `{suspended, ...}` result above. `py_context` runs the fun
+(`execute/2` in `py_callback`), serving nested requests meanwhile
+(`wait_for_callback/2`), and calls `resume_callback(StateRef, Response)`
+with the response body of 3.4.
+
+### 3.2 Context callback pipe
+
+Set up with `context_set_callback_handler(Ref, Pid)`. The context thread
+sends:
+
+```erlang
+{erlang_callback, CallbackId, FuncName, Args}
+```
+
+to the handler process and blocks (GIL released) reading the pipe with a
+30 s timeout. The handler runs the fun and answers with
+`context_write_callback_response(Ref, Body)`, which writes
+`<<Len:32/native, Body/binary>>` on the pipe. There is no id on this pipe:
+one context thread, one outstanding call.
+
+### 3.3 Thread worker pipe
+
+Any other Python thread (a `threading.Thread`, an executor worker) sends:
+
+```erlang
+{thread_callback, WorkerId, CallbackId, FuncName, Args}
+```
+
+to the `py_thread_handler` coordinator, which gives each `WorkerId` a
+handler process (`{thread_worker_spawn, WorkerId, WriteFd}`) and replies
+with `thread_worker_write_response(Fd, CallbackId, Body)`. The frame on the
+response pipe is:
+
+```
+<<CallbackId:64/native, Len:32/native, Body:Len/binary>>
+```
+
+The reader discards frames whose id is not the one it waits for; a short
+read or a timeout poisons the worker so the pipe is never read out of
+phase. `erlang.async_call` uses the same frame on a per-interpreter
+non-blocking pipe (`async_callback_pipe`, `{async_callback, CallbackId,
+FuncName, Args, WriteFd}`), parsed incrementally by the reader tick.
+
+### 3.4 Response body
+
+Every path answers with the same body:
+
+```
+<<2, ETF/binary>>        %% ok, term_to_binary(Result)
+<<1, Message/binary>>    %% error, UTF-8 text, raised as RuntimeError
+```
+
+Built in `handle_blocking_callback/3` (`py_context`), `py_thread_handler`
+and the isolated loop; parsed by `parse_callback_response` in C and
+`_isolated.py` in the child. Keep the three writers and two parsers in
+step.
+
+## 4. Isolated socket
+
+The child (`priv/py_isolated_child.py`) connects to a Unix socket in the
+private directory (`py_isolated:sock_dir/0`) and both sides exchange frames:
+
+```
+<<Id:64/native, Len:32/native, Body:Len/binary>>
+Body = <<Status:8, ETF/binary>>
+```
+
+| Status | Direction | Meaning | Payload |
+|---|---|---|---|
+| 0 | Erlang to child | request | `{call, M, F, Args, Kwargs}`, `{eval, Code, Locals}`, `{exec, Code}`, `{submit, TaskRef, ...}`, `start_loop`, `{pass_fd, ...}`, `ping`, `shutdown`, `{init, Opts}` |
+| 1 | either | error reply | `{Class, Message, Traceback}` or an atom |
+| 2 | either | ok reply | the result term |
+| 3 | child to Erlang | request from Python | `{call, Name, Args}`, `{send, Pid, Msg}`, `{whereis, Name}` |
+| 4 | child to Erlang | event, `Id` = 0 | `{ready, Info}`, `{startup_error, Problems}`, `{memory_limit, Rss}`, `{log, Level, Msg}`, `{async_result, TaskRef, R}`, `{loop_exit, R}` |
+| 5 | Erlang to child | control, `Id` = 0 | `{interrupt, Target}`, `{cancel, Id}`, `stop_loop`, `{shm_close, Id}` |
+
+Ids: Erlang numbers its requests from 1 (`next_id`); the child numbers its
+status-3 requests independently; replies carry the id of the request they
+answer. Control and events use id 0 and never get a reply.
+
+ETF: `_etf.py` encodes what `py_convert.c` would produce for the same
+Python value, with two differences worth knowing: `Pid`, `Ref` and `Port`
+are opaque and keep their raw bytes, and the child can create atoms
+(`binary_to_term/1` is not called with `safe`). Shared handles
+(`{'$py_shm', Id, Path, Size}`, `{'$py_buffer', Id, Path, Ring}`) are plain
+tuples on the wire and become `SharedMemory` / `SharedBuffer` on arrival
+(`_shm.from_term`); NIF resources cannot cross and the API answers
+`{error, not_supported_in_isolated}`.
+
+Handshake (`py_isolated:handshake/1`): the child sends `{ready, Info}`,
+Erlang sends `{init, Opts}` (status 0, id 1) and the preload `exec`,
+serving status-3 requests that arrive meanwhile with a bounded callback
+runner. Anything else during the handshake fails the start.
+
+Interrupt: `{interrupt, Target}` is read by the child's reader thread, which
+sends SIGUSR1 to the main thread when `Target` is the request executing
+now (top of `_exec_stack`); the signal handler raises `_Interrupted`
+(a `KeyboardInterrupt` subclass) only while a request runs. `{cancel, Id}`
+marks a queued request so it is answered `{error, cancelled}` instead of
+run. Erlang arms a SIGKILL backstop bound to the request id when it sends
+an interrupt; a reply for that id cancels it.
+
+Flow control: the socket buffers are 1 MB each way; a frame larger than
+that is written in pieces by both sides. Bulk data goes through shared
+memory, not the socket (see [isolated](isolated.md)).
+
+## 5. Shared buffer ring
+
+`py_buffer:new(#{shared => true})` allocates a `py_shm` region used as a
+ring. The first page is a header written by Erlang after each `write/2`
+(write position, closed flag, ring size) and read by Python; the
+`py_shm` server keeps the consumed position from the callbacks below, not
+from the header. Notifications are callbacks, so they work in every mode:
+
+| Callback | Called by | Returns |
+|---|---|---|
+| `_py_buffer_wait(Id, ReadPos)` | reader, before blocking | `{WPos, Closed}` once `WPos > ReadPos` or the buffer is closed |
+| `_py_buffer_consumed(Id, N)` | reader, after a read | `ok`; a writer blocked on space wakes up |
+| `_py_buffer_state(Id)` | reader, on (re)map | `{WPos, Closed}` without waiting |
+
+All three answer `{error, closed}` for an unknown id. The header is only
+read after a callback returned, so ordering follows the round trip and the
+Python side needs no fence.
+
+## Changing a protocol
+
+- Add a request kind: the message in `py_context` (API and embedded loop),
+  the `ctx_request_type_t` and execute function in C, the `request/6`
+  clause in `py_isolated`, and `_on_request` in `_isolated.py`.
+- Add a control: `send_frame(Child, 0, ?STATUS_CONTROL, Term)` in
+  `py_isolated` and `_on_control` in `_isolated.py`; controls must be safe
+  to handle on the reader thread.
+- Add a child event: `self.event(...)` in `_isolated.py` and an
+  `{ok, {0, ?STATUS_EVENT, ...}}` clause in `drain_socket` or `handshake`.
+- Change the response body: three writers and two parsers listed in 3.4.
+- Never change the frame header: the child, the callback pipe reader and
+  the async pipe parser share it.