From 0d36591a1baf601963308445d345c90cc504a91b Mon Sep 17 00:00:00 2001 From: Benoit Chesneau Date: Sat, 29 Aug 2026 20:10:29 +0200 Subject: [PATCH] Check the code map against the tree A script, run by make and CI, that fails when a source file is missing from docs/code-map.md, an Erlang module has no moduledoc, or a module has no row in the new Modules table of the coverage audit. Running it once found two modules, seven headers and two Python files the map had missed. --- .github/workflows/ci.yml | 3 +++ CHANGELOG.md | 6 ++++++ Makefile | 7 ++++++- docs/code-map.md | 9 ++++++--- docs/contributing.md | 4 +++- scripts/check_code_map.sh | 21 ++++++++++++++++++++ test/coverage_audit.md | 40 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 85 insertions(+), 5 deletions(-) create mode 100755 scripts/check_code_map.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2d532fa..a2e6ee9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -326,6 +326,9 @@ jobs: - name: Run dialyzer run: rebar3 dialyzer + - name: Check code map + run: make check-code-map + docs: name: Documentation runs-on: ubuntu-24.04 diff --git a/CHANGELOG.md b/CHANGELOG.md index 37bdd1f..1514ebf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,6 +65,12 @@ virtual environments and shared dicts to `py_stream`, `py_venv` and `py_shared_dict`. The public API is unchanged. +### Documentation + +- `make check-code-map` (also run by CI) verifies that every source file is + in `docs/code-map.md`, every Erlang module has a moduledoc and a row in + the Modules table of `test/coverage_audit.md`. + ### Removed - The legacy worker API (`py_nif:worker_new/0,1`, `worker_call`, `worker_eval`, diff --git a/Makefile b/Makefile index 925c1ea..b44db69 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: all compile test lint-docs clean +.PHONY: all compile test lint-docs check-code-map clean all: compile @@ -16,5 +16,10 @@ test: lint-docs: compile escript scripts/lint_doc_snippets.escript +# Every source file in docs/code-map.md, every module with a moduledoc and +# a row in the Modules table of test/coverage_audit.md. +check-code-map: + sh scripts/check_code_map.sh + clean: rebar3 clean diff --git a/docs/code-map.md b/docs/code-map.md index 534ce81..9da9c9b 100644 --- a/docs/code-map.md +++ b/docs/code-map.md @@ -23,7 +23,8 @@ exercised by suites). Guides are in `docs/`, suites in `test/`. Start with | `py_thread_handler` | Coordinator that gives each Python thread calling Erlang a handler process and a pipe | live | threading | `py_thread_callback_SUITE`, `py_reentrant_SUITE` | | `py_event_loop` | Main-interpreter asyncio loop: `run`, `create_task`, `await`, and the loop callbacks Python needs | live | asyncio | `py_event_loop_SUITE`, `py_async_task_SUITE` | | `py_event_loop_pool` | Several main-interpreter loops with process affinity | live | asyncio | `py_event_loop_pool_SUITE` | -| `py_event_worker`, `_sup`, `_registry` | One process per running loop receiving `enif_select` readiness and timers | live | event_loop_architecture | `py_event_loop_SUITE`, `py_fd_ops_SUITE` | +| `py_event_worker` | One process per running loop receiving `enif_select` readiness and timers | live | event_loop_architecture | `py_event_loop_SUITE`, `py_fd_ops_SUITE` | +| `py_event_worker_sup`, `py_event_worker_registry` | Supervisor and name registry of the event workers | live | event_loop_architecture | `py_event_loop_SUITE` | | `py_reactor_context` | FD-owning context for the protocol-based reactor | live | reactor | `py_reactor_SUITE` | | `py_channel`, `py_byte_channel` | Term and byte queues between Erlang and Python coroutines (NIF resources) | live | channel | `py_channel_SUITE`, `py_byte_channel_SUITE` | | `py_buffer` | Native streaming input buffer; shared variant delegates to `py_shm` | live | buffer, isolated | `py_buffer_SUITE`, `py_isolated_buffer_SUITE` | @@ -56,6 +57,7 @@ files. Editing `py_convert.c` alone does not compile it alone; build with | `py_logging.c` | Logging and tracing NIFs | live | | `py_mem_limit.c` | Per-interpreter memory caps (owngil) | live | | `py_util.c/.h` | Macros and helpers | live | +| `py_nif.h`, `py_event_loop.h`, `py_channel.h`, `py_buffer.h`, `py_reactor_buffer.h`, `py_subinterp_thread.h`, `py_util.h` | Declarations shared between the included files; the struct comments in `py_nif.h` and `py_event_loop.h` carry the lock contracts | live | The only code not on a live path is the "Test Helper Functions" section of `py_event_loop.c` (fd, pipe, TCP and UDP helpers the suites use). @@ -79,7 +81,7 @@ loop, channels and servers. | `_erlang_impl/_isolated.py` | Child runtime: socket frames, reader thread, re-entrant main loop, interrupt signal, asyncio loop, the `erlang` shim | isolated child | | `_erlang_impl/_shm.py` | `SharedMemory` and `SharedBuffer` wrappers over mmap | all | | `py_isolated_child.py` | Child launcher: rlimits, parent-death signal, cgroup join, connect | isolated child | -| `test_erlang_loop.py`, `tests/` | Python-side tests of the loop | test | +| `test_erlang_loop.py`, `test_async_task.py`, `test_channel_ref.py`, `tests/` | Python-side tests of the loop, tasks and channels | test | ## Tests (`test/`) @@ -87,7 +89,8 @@ Suites named `py__SUITE`. Cross-mode suites run the same cases in `worker` and `isolated` groups (`py_isolated_SUITE`, `py_isolated_vm_SUITE`, `py_isolated_shm_SUITE`, `py_isolated_buffer_SUITE`). Python helpers used by suites are `test/py_test_*.py`. `test/coverage_audit.md` maps public APIs to -cases. `test/test.config` holds node-wide settings (memory limits flag). +cases and every module to its suites; `make check-code-map` verifies this +page and that table against the tree. `test/test.config` holds node-wide settings (memory limits flag). ## Build and docs diff --git a/docs/contributing.md b/docs/contributing.md index 756ed2e..44edc01 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -33,6 +33,7 @@ rebar3 ct --suite test/py_isolated_SUITE # one suite rebar3 ct --suite test/py_context_SUITE --case test_call # one case rebar3 dialyzer && rebar3 xref # required before a PR make lint-docs # snippets in README and docs/ +make check-code-map # every file in the code map ``` Notes: @@ -182,7 +183,8 @@ supervisor to carry it. 4. Skip, do not fail, when a platform or interpreter cannot run a case: `{skip, Reason}` with the reason a human can act on. 5. Add the suite to the table in `docs/code-map.md` and the cases that - cover a documented API to `test/coverage_audit.md`. + cover a documented API to `test/coverage_audit.md`; a new module also + needs a row in its Modules table. `make check-code-map` verifies both. 6. Cases that measure time or memory print their numbers with `ct:pal` and assert only on invariants, never on absolute timings. diff --git a/scripts/check_code_map.sh b/scripts/check_code_map.sh new file mode 100755 index 0000000..14e44be --- /dev/null +++ b/scripts/check_code_map.sh @@ -0,0 +1,21 @@ +#!/bin/sh +# Every source file must be listed in docs/code-map.md, every Erlang module +# must have a moduledoc, and every Erlang module must have a row in the +# "Modules" table of test/coverage_audit.md. Exit code is the number of +# failures. Run: sh scripts/check_code_map.sh +cd "$(dirname "$0")/.." || exit 2 +fail=0 +note() { echo "check_code_map: $1"; fail=$((fail + 1)); } + +for f in src/*.erl; do + m=$(basename "$f" .erl) + grep -q "\`$m\`" docs/code-map.md || note "$m is not in docs/code-map.md" + grep -q '^%%%\{0,1\} @doc' "$f" || note "$f has no @doc moduledoc" + grep -q "^| \`$m\` |" test/coverage_audit.md || note "$m has no row in the Modules table of test/coverage_audit.md" +done +for f in c_src/*.c c_src/*.h priv/_erlang_impl/*.py priv/*.py; do + b=$(basename "$f") + grep -q "$b" docs/code-map.md || note "$b is not in docs/code-map.md" +done +[ "$fail" -eq 0 ] && echo "check_code_map: clean" +exit $fail diff --git a/test/coverage_audit.md b/test/coverage_audit.md index 2502366..bb8a8ca 100644 --- a/test/coverage_audit.md +++ b/test/coverage_audit.md @@ -5,6 +5,46 @@ README and `docs/*.md` to at least one `*_SUITE.erl` test that exercises it. Update this table whenever a documented API is added, renamed, or removed. +## Modules + +Every Erlang module and the suites that exercise it, so a module without a +suite is visible. `scripts/check_code_map.sh` requires a row per module. + +| Module | Suites | +|---|---| +| `py` | `py_SUITE`, `py_api_SUITE`, `py_stream_SUITE`, `py_venv_SUITE` | +| `py_context` | `py_context_SUITE`, `py_context_process_SUITE`, `py_interrupt_SUITE`, `py_worker_loop_SUITE` | +| `py_context_embedded` | `py_context_SUITE`, `py_context_process_SUITE`, `py_interrupt_SUITE`, `py_worker_loop_SUITE` | +| `py_stream` | `py_stream_SUITE` | +| `py_venv` | `py_venv_SUITE` | +| `py_shared_dict` | `py_SUITE` | +| `py_isolated` | `py_isolated_*_SUITE` | +| `py_context_router` | `py_context_router_SUITE`, `py_pool_SUITE` | +| `py_context_sup` | (through the above) | +| `py_context_init` | (through the above) | +| `py_nif` | all | +| `py_callback` | `py_callback_encoding_SUITE`, `py_thread_callback_SUITE` | +| `py_thread_handler` | `py_thread_callback_SUITE`, `py_reentrant_SUITE` | +| `py_event_loop` | `py_event_loop_SUITE`, `py_async_task_SUITE` | +| `py_event_loop_pool` | `py_event_loop_pool_SUITE` | +| `py_event_worker` | `py_event_loop_SUITE`, `py_fd_ops_SUITE` | +| `py_event_worker_sup` | `py_event_loop_SUITE` | +| `py_event_worker_registry` | `py_event_loop_SUITE` | +| `py_reactor_context` | `py_reactor_SUITE` | +| `py_channel` | `py_channel_SUITE`, `py_byte_channel_SUITE` | +| `py_byte_channel` | `py_channel_SUITE`, `py_byte_channel_SUITE` | +| `py_buffer` | `py_buffer_SUITE`, `py_isolated_buffer_SUITE` | +| `py_shm` | `py_isolated_shm_SUITE` | +| `py_import` | `py_import_SUITE` | +| `py_preload` | `py_preload_SUITE` | +| `py_state` | `py_state_SUITE` | +| `py_semaphore` | (through `py_SUITE`) | +| `py_logger` | `py_logging_SUITE` | +| `py_util` | (through every suite) | +| `py_tracer` | `py_logging_SUITE` | +| `erlang_python_app` | all | +| `erlang_python_sup` | all | + ## Erlang public API (`src/py.erl` exports) | API | Documented in | Test suite | Test case |